The discussion highlights the challenges of calculating the empirical mean in high-dimensional data, particularly when faced with outliers. While the median is a robust solution in one dimension, its application becomes complex in higher dimensions, especially when a significant portion of the data consists of outliers. This insight emphasizes the need for advanced statistical methods to accurately estimate expectations in such scenarios.