First, since it’s short, let’s quote a comment from an astronomer that might reflect the notion of robust statistics in astronomy.

Bayesian is robust.

Is every Bayesian method robust and its counter part from classical statistics is not robust? I know that popular statistics in astronomy are not, generally speaking, robust and those popular statistics were borne before the notion of robustness in statistics were recognized and discussed.

I do understand why such partisan comment was produced. Astronomers always reports their data analysis results by best fits, error bars, probability, or some significance levels (they don’t say explicitly, p-values, powers, type I or type II errors, unbiased estimates, and other statistical jargon in inference problems) and those classical methods of frequent use have caused frustrations due to their lack of robustness. On the contrary, MCMC algorithms for estimating posterior distributions produce easy interpretable results to report best fit (mode) and error bar (HPD).

My understanding of robustness as a statistician does not draw a line between Bayesian and frequenstists. The following is quoted from the Katholieke Universiteit Leuven website of which mathematics department has a focus group for robust statistics.

Robust statistical methods and applications.
The goal of robust statistics is to develop data analytical methods which are resistant to outlying observations in the data, and hence which are also able to detect these outliers. Pioneering work in this area has been done by Huber (1981), Hampel et al. (1986) and Rousseeuw and Leroy (1987). In their work, estimators for location, scale, scatter and regression play a central role. They assume that the majority of the data follow a parametric model, whereas a minority (the contamination) can take arbitrary values. This approach leads to the concept of the influence function of an estimator which measures the influence of a small amount of contamination in one point. Other measures of robustness are the finite-sample and the asymptotic breakdown value of an estimator. They tell what the smallest amount of contamination is which can carry the estimates beyond all bounds.

Nowadays, robust estimators are being developed for many statistical models. Our research group is very active in investigating estimators of covariance and regression for high-dimensional data, with applications in chemometrics and bio-informatics. Recently, robust estimators have been developed for PCA (principal component analysis), PCR (principal component regression), PLS (partial least squares), classification, ICA (independent component analysis) and multi-way analysis. Also robust measures of skewness and tail weight have been introduced. We study robustness of kernel methods, and regression quantiles for censored data.

My understanding of “robustness” from statistics education is pandemic, covers both Bayesian and frequentist. Any methods and models that are insensitive or immune to outliers, are robust methods and statistics. For example, median is more robust than mean since the breakpoint of median is 1/2 and that of mean is 0, asymptotically. Both mean and median are estimable from Bayesian and frequentist methods. Instead of standard deviation, tactics like lower and upper quartiles to indicate error bars or Winsorization (or trim) to obtain a standard deviation for the error bar, are adopted regardless of Bayesian or frequenstist. Instead of the chi square goodness-of-fit tests, which assume Gaussian residuals, nonparametrics tests or distribution free tests can be utilized.

The notion that frequentist methods are not robust might have been developed from the frustration that those chi-square related methods in astronomy do not show robust characteristics. The reason is that data are prone to the breaks of the normality (Gaussianity) assumption. Also, the limited library of nonparametric methods in data analysis packages and softwares envisions that frequentist methods are not robust. An additional unattractive aspect about frequentist methods is that the description seems too mathematical, too abstract, and too difficult to be codified with full of unfriendly jargon whereas the Bayesian methods provide step by step modeling procedures with explanation why they chose such likelihood and such priors based on external knowledge from physics and observations (MCMC algorithms in the astronomical papers are adaptation of already proven algorithms from statistics and algorithm journals).

John Tukey said:

Robustness refers to the property of a procedure remaining effective even in the absence of usual assumptions such as normality and no incorrect data values. In simplest terms the idea is to improve upon the use of the simple arithmetic average in estimating the center of a distribution. As a simple case one can ask: Is it ever better to use the sample median than the samle mean, and if so, when?

I don’t think the whole set of frequentist methods is the complement set of Bayesians. Personally I feel quite embarrassed whenever I am told that frequentist methods are not robust compared to Bayesian methods. Bayesian methods become robust when a priori knowledge (subjective priors) allows the results to be unaffected by outliers with a general class of likelihood. Regardless of being frequentist or Bayesian, statistics have been developed to be less sensitive to outliers and to do optimal inferences, i.e. to achieve the goal, robustness.

Ah…there are other various accounts for robustness methods/statistics in astronomy not limited to “bayesian is robust.” As often I got confused whenever I see statistically rigorous while the method is a simple chi-square minimization in literature, I believe astronomers can educate me the notion of robustness and the definition of robust statistics in various ways. I hope for some sequels from astronomers about robustness and robust statistics to this post of limited view.

Even though most of my memories about his writings are gone – how many people can beat the time and fading memories! – like other rudimentary astronomy and physics stuffs that I used to know, statistics brought up his name above the surface before it sinks completely to the abyss.

Recently, I heard that the reason Prof. Feynman became famous outside of the physics community is

In this video, Prof. Feynman demonstrates the cause of the explosion while others, based on what I heard not the video, were trying to prove things with mathematical equations.

I was in the process of writing on “model uncertainty” to understand why the same term is used differently in astronomy and in statistics and planned to include some papers to show how statisticians handle uncertainty. One of them is Assessment and Propagation of Model Uncertainty by David Draper in JRSS B Vol. 57, No. 1 (1995), pp. 45-97. Draper used the O-ring data set, one of the most frequently cited data sets in statistics textbooks. (UCI archive has the data)

One of my favorite statisticians is John Tukey, of Fast Fourier Transform, higher criticisms, and exploratory data analysis which are all well known to astronomers. There is an interesting anecdotes of Feynman and Tukey, two most intellectual individuals, each of whom represented his field. It is an excerpt from Notices of the American Mathematical Society February 2002 Volume 49 Issue 2 pp.193-201 (pdf)

While living in the Graduate College, John came to know the physicist Richard Feynman, and he appears in various of the books by and about Feynman. One special story relates to keeping time. Feynman knew that he could keep track of time while reading, but not while speaking. He presented this as a challenge. Rising to it, JWT showed that he could speak and keep track of time simultaneously. Of this Feynman remarks: “Tukey and I discovered that what goes on in different people’s heads when they think they’re doing the same thing—something as simple as counting—is different for different people.” This may also be the source of JWT’s remark, “People are different.”

In 1939 Feynman and Tukey, together with Bryant Tuckerman and Arthur Stone, were members of the Flexagon Committee. This group formed directly following the discovery of certain origami like objects by Stone. Flexagons are folded from strips of paper and reveal different faces as they are flexed. A theory of flexagons was worked out by Tukey and Feynman, the theory being a hybrid of topology and network theory. Feynman created a diagram that showed all the possible paths through a hexaflexagon. The Feynman-Tukey theory was never published, but parts were later rediscovered.

After more than half a century since such conversation is made, after an almost quarter century since the explosion of Challenger, I wonder how many conversations have occurred between astronomers and statisticians casually. Did conversations ever happened, I wonder what kind problems scientifically intriguing and common to both fields were solved. Contradicting to Prof. Feynman’s simplicity of explaining causes, for two decades, there were statisticians and statistic practitioners spending time to explain their models and to argue that what they found was not discovered by predecessors’ model based on this O-ring data set. I wonder if such statistical modeling efforts help to prevent any disastrous results in space missions while contracting the Feynman’s report about the cause of the explosion.

I’d like to believe that there are occasionally many conversations among astronomers and statisticians in the world, which would produce more fruitful results than astronomers’ applying a century old statistics blindly without knowing its foundation and statisticians’ not talking astronomers because of complexity in physics.