Comments on: tests of fit for the Poisson distribution http://hea-www.harvard.edu/AstroStat/slog/2008/tests-of-fit-for-the-poisson-distribution/ Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 01 Jun 2012 18:47:52 +0000 hourly 1 http://wordpress.org/?v=3.4 By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/tests-of-fit-for-the-poisson-distribution/comment-page-1/#comment-267 hlee Wed, 02 Jul 2008 23:42:37 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=280#comment-267 <blockquote>Best, D. J. and Rayner, J. C. W. (2005) <a href="http://www.ingentaconnect.com/content/tandf/lssp/2005/00000034/00000001/art00006" rel="nofollow">Improved Testing for the Poisson Distribution Using Chisquared Components with Data Dependent Cells</a> (subscription required) <i>Communications in Statistics: Simulation and Computation</i>, Volume 34, Number 1, pp. 85-96(12)</blockquote> <strong>Abstract:</strong> A power study suggests that a good test of fit analysis for the Poisson distribution is provided by a data dependent Chernoff-Lehmann X 2 test with class expectations greater than unity, and its components. These data dependent statistics involve arithmetically simple parameter estimation, convenient approximate distributions, and provide a comprehensive assessment of how well the data agree with a Poisson distribution. We suggest that a well-performed single test of fit statistic is the Anderson-Darling statistic. Three examples are discussed. ------------------------------------------------------------------------------------------------------------------------------------------------------------------ <blockquote>Haschenburger, J.K. and Spinelli, J.J. (2005) <a href="http://www.springerlink.com/content/qj28841676171974/" rel="nofollow"> Assessing the Goodness-of-Fit of Statistical Distributions When Data Are Grouped</a> (subscription required) <i>Mathematical Geology,</i> Volume 37, Number 3, pp. 261-276 </blockquote> <strong>Abstract:</strong> Modeling statistical distributions of phenomena can be compromised by the choice of goodness-of-fit statistics. The Pearson chi-square test is the most commonly used test in the geosciences, but the lesser known empirical distribution function (EDF) statistics should be preferred in many test situations. Using a data set from geomorphology, the Anderson–Darling test for grouped exponential distributions is employed to illustrate ease of use and statistical advantages of this EDF test. Attention to the issues discussed will result in more informed statistic selection and increased rigor in the identification of distribution functions that describe random variables. ------------------------------------------------------------------------------------------------------------------------------------------------------------------ Chernoff and Lehmann &#967<sup>2</sup> in these papers reminds me Lucy (2000) that recommends improved &#967<sup>2</sup> instead of Pearson &#967<sup>2</sup>. <blockquote>Lucy, L.B. (2000) <a href="http://adsabs.harvard.edu/abs/2000MNRAS.318...92L" rel="nofollow">Hypothesis testing for meagre data sets</a> <u>MNRAS</u>, Volume 318, Issue 1, pp. 92-100. </blockquote> <strong>Abstract:</strong>Improved χ2 statistics are defined for both Poisson and multinomial data sets. Numerical experiments with the Nousek-Shue test problem from X-ray spectroscopy indicate that, in contrast to Pearson's statistic X2, these modified statistics remain approximately valid at the 2σ level of significance even when the mean counts per bin is < <1 provided that the total counts N>~30. The simplest of the proposed statistics is formally equivalent to carrying out a goodness-of-fit test with Pearson's statistic but with modified confidence limits, and this procedure is recommended.

Best, D. J. and Rayner, J. C. W. (2005)
Improved Testing for the Poisson Distribution Using Chisquared Components with Data Dependent Cells (subscription required)
Communications in Statistics: Simulation and Computation, Volume 34, Number 1, pp. 85-96(12)

Abstract: A power study suggests that a good test of fit analysis for the Poisson distribution is provided by a data dependent Chernoff-Lehmann X 2 test with class expectations greater than unity, and its components. These data dependent statistics involve arithmetically simple parameter estimation, convenient approximate distributions, and provide a comprehensive assessment of how well the data agree with a Poisson distribution. We suggest that a well-performed single test of fit statistic is the Anderson-Darling statistic. Three examples are discussed.
——————————————————————————————————————————————————————

Haschenburger, J.K. and Spinelli, J.J. (2005)
Assessing the Goodness-of-Fit of Statistical Distributions When Data Are Grouped (subscription required)
Mathematical Geology, Volume 37, Number 3, pp. 261-276

Abstract: Modeling statistical distributions of phenomena can be compromised by the choice of goodness-of-fit statistics. The Pearson chi-square test is the most commonly used test in the geosciences, but the lesser known empirical distribution function (EDF) statistics should be preferred in many test situations. Using a data set from geomorphology, the Anderson–Darling test for grouped exponential distributions is employed to illustrate ease of use and statistical advantages of this EDF test. Attention to the issues discussed will result in more informed statistic selection and increased rigor in the identification of distribution functions that describe random variables.
——————————————————————————————————————————————————————
Chernoff and Lehmann χ2 in these papers reminds me Lucy (2000) that recommends improved χ2 instead of Pearson χ2.

Lucy, L.B. (2000)
Hypothesis testing for meagre data sets
MNRAS, Volume 318, Issue 1, pp. 92-100.

Abstract:Improved χ2 statistics are defined for both Poisson and multinomial data sets. Numerical experiments with the Nousek-Shue test problem from X-ray spectroscopy indicate that, in contrast to Pearson’s statistic X2, these modified statistics remain approximately valid at the 2σ level of significance even when the mean counts per bin is < <1 provided that the total counts N>~30. The simplest of the proposed statistics is formally equivalent to carrying out a goodness-of-fit test with Pearson’s statistic but with modified confidence limits, and this procedure is recommended.

]]>