I have to check the paper carefully, but I think though that the 1% number in Roelofs et al does not refer to offsets of “this magnitude or larger”, but rather to a small range of offsets defined by the error on the measured offset (see their Fig 3).

]]>The quote above was a paraphrase, btw. The exact quote is *“Extensive simulations of the Chandra data show that the probability of finding an offset of this magnitude is ~1%, equal to the (trial-corrected) probability of a chance alignment with any X-ray source in the field.”*

I couldn’t resist commenting on it, however, because I came up with a Bayesian approach for assessing directional (and more generally, spatio-temporal) coincidences quite a few years ago (inspired by a GRB problem), and I’ll be using it as a pedagogical example at the CASt summer school in just a few weeks. The exercise compares the behavior of these two quantities (a p-value for a hypothesis test, and the posterior odds for coincidence vs. no coincidence). I’m also waiting (on pins and needles—news should be imminent) to see if an NSF proposal that, in part, seeks to develop MCMC-flavored algorithms for implementing Bayesian coincidence assessment with large data sets will get funded. We’ll see….

Anyone, one of the lessons of the toy computation for CASt is that a p-value can reject the null hypothesis of no true association (i.e., conclude there *is* an association) where the Bayesian calculation favors the null. The reason is that some data may be rather improbable under the null (thus leading to rejection in a significance test), yet similarly improbable under the alternative (here there is a definite alternative: association); a Bayes factor can thus say that data with a small p-value nevertheless does not significantly favor the alternative. An explicit (and often messy) power calculation might spare the significance test fans embarassment, but no one does them. The Bayes factor nicely puts all you need into a single quantity, with the usual “Occam factor” machinery coming in to play to help things out.