To be sure, there are other possible reasons for this underdispersion, such as correlations in how the data are gathered and analyzed, and an overly conservative estimation of error bars, etc. In fact, the most benign explanation is probably in how people carry out “sanity checks” and tend to discard or explain away or correct the data that give odd results.

While this is indeed worrisome, I am inclined to think that this is not wrong per se, but rather a case where a fully Bayesian analysis would give the “right” coverage. After all, there does exist a strong prior that people are bringing into the analysis, but are not including in the calculations of the widths of the posterior probability distributions. Including such a highly informative prior will of course shrink the sizes of the error bars and make everything consistent. i.e., I think that the assumption needs to be explicit, that is all. Is that bias? bandwagon? or prior belief?

We recently submitted that paper to AJ, and rather ironically, I did the analysis during the same time frame as this discussion was going on, about how astronomers cannot rely on repeating observations. Ironic because the result reported there hinges on the existence of small, but persistent signal that is found in repeated observations of the same source. Doubly ironic in fact, in that just as we were backing and forthing about cultural differences I seemed to have gone and done something completely contrary to my heritage!

btw, this paper is interesting because Capella is a strong X-ray source, and “everybody believes” that such sources should exhibit some variability, so finding such shouldn’t be a big deal, and yet Capella itself has been remarkably stable and had all this while defied the characterization and even the detection of such variability. Even now, the estimated magnitude of the variability fraction is rather small. It’s a good thing that we had some 22 counts/sec over 205 kiloseconds to play with.