Is 8-sigma significant enough for you?

There is a new report from Bernabei et al. (arXiv:0804.2741) of the direct detection of the effects of Dark Matter that is causing a lot of buzz. (The Bad Astronomer has a good summary.) They find yearly modulation in their detected scintillation rate that matches what you would expect if the Earth were rushing through Galactic Dark Matter as it goes around the Sun. They have worked out the significance of the modulation to be 8.2 sigma. Significant! But significant of what?

I am no expert on this in a hundred and one different ways. But I feel kinda sorry for the DAMA group. Certainly, at 8-sigma, it is easy to accept that there is modulation. But is this modulation proof of Dark Matter? Astronomers in general are extremely suspicious of any yearly modulations, as we have learnt from hard experience that it is an extremely common source of systematic error. Essentially, the Earth is a poorly calibrated detector, and it has diurnal and annual cycles, and these invariably show up in everything. So when the signal you are looking for goes exactly like the first thing you try to catch and eliminate, what price statistical significance?

2 Comments
  1. Count Iblis:

    To me it’s not really the 8 sigmas that I find impressive, because you can have some systematic effect that you’ve overlooked. DAMA uses a model independent approach which makes their detecton method relatively unbiased against possible dark matter candidates compared to other searches like CDMS, but it also exposes them to the possibility of some unforseen systematic effects.

    So, we should instead focus on the likelyhood of systematic effects. We can note the following points:

    1) The latest results are from DAMA/LIBRA which confirmed a previous result by DAMA/NaI. The old detector was decomissioned, a new detector was build and installed. The 8.2 sigma result is the combined result from both experiments. What is perhaps more significant is that the separate DAMA/LIBRA results and DAMA/NaI results are within their error margins, except for one 2 sigma deviation in one energy bin.

    2) In the near future other features must become visible in the signal, like a diurnal modulation caused by the rotation of the Earth. This will have athe period of one siderial day and not one solar day. I’m guessing that the DAMA team have already seen this effect and are just waiting for it to becone statistically significant.

    I actually pretty much knew that DAMA/LIBRA was going to confirm DAMA/NAI already two years ago, because why else were the DAMA team writing so many papers the last few years? If DAMA/LIBRA had a null signal, that would have been known in 2005 or so. The same is probably also true for the diurnal modulation now. :)

    04-24-2008, 8:39 pm
  2. vlk:

    Indeed, it is the likelihood of systematic effects that will control our belief in the result. I am not sure, Count Iblis, that [1] is all that persuasive — all it may mean is that the relevant effect was not identified with either experiment. That is, what if both of them are simply watching the Earth breathing, so to speak? Short of moving the experiment off the planet, I am unsure that that can be definitively settled. Point [2] is however interesting. If a distinction can be made between modulations at the sidereal vs solar day periods, that will at least eliminate solar effects as a cause!

    04-25-2008, 9:41 am
Leave a comment