The AstroStat Slog

Physicists believe that the Gaussian law has been proved in mathematics while mathematicians think that it was experimentally established in physics — Henri Poincare

Continue reading ‘Why Gaussianity?’ »

10:00am local time, Sept. 10th, 2008
As the first light from Fermi or GLAST, LHC First Beam is also a big moment for particle physicists. Find more from http://lhc-first-beam.web.cern.ch/lhc-first-beam/Welcome.html. Continue reading ‘LHC First Beam’ »

The problem with data analysis is of course that it is a performing art. It is not something you easily write a paper on; rather, it is something you do. And so it is difficult to publish.

quoted from this conversation Continue reading ‘A Conversation with Peter Huber’ »

My greatest concern was what to call it. I thought of calling it “information”, but the word was overly used, so I decided to call it “uncertainty”. When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, “You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, nobody knows what entropy really is, so in a debate you will always have the advantage.”

Continue reading ‘An anecdote on entrophy’ »

The whole story can be found from the page 8 of IMS Bulletin, Vol.37 Issue 7. (click for the pdf file) Continue reading ‘Irksome’ »

A question to astronomers. What do you think WLOG is? It has nothing to do with our BLOG or SLOG. Statisticians, please do not say a word. Continue reading ‘WLOG’ »

Talking about limits in Numerical Recipes in my PyIMSL post, I couldn’t resist checking materials, particularly updates in the new edition of Numerical Recipes by Press, et al. (2007). Continue reading ‘NR, the 3rd edition’ »

PyIMSL is a collection of Python wrappers to the math and statistical algorithms in the IMSL C Numerical Library^[1]. I recall the days of digging in IMSL (International Mathematics and Statistics Library) user manuals and learning Fortran and C to use this vast library (Splus was to slow at that time). Upon knowing that Python is very favored among astronomers (click here to see the slog posts about Python) and that limits exist in Numerical Recipes (I didn’t check the latest version published last year, though), probably IMSL is useful for mathematical and statistical analysis for astronomers.

To know more, Continue reading ‘PyIMSL’ »

1. cited from http://en.wikipedia.org/wiki/IMSL[↩]

I didn’t realize this post was sitting for a month during which I almost neglected the slog. As if great books about probability and information theory for statisticians and engineers exist, I believe there are great statistical physics books for physicists. On the other hand, relatively less exist that introduce one subject to the other kind audience. In this regard, I thought the lecture note can be useful.

[arxiv:physics.data-an:0808.0012]
Lectures on Probability, Entropy, and Statistical Physics by Ariel Caticha
Abstract: Continue reading ‘A lecture note of great utility’ »

Astronomers confront with various censored and truncated data. Often these types of data are called after famous scientists who generalized them, like Eddington bias. When these censored or truncated data become the subject of study in statistics, instead of naming them, statisticians try to model them so that the uncertainty can be quantified. This area is called survival analysis. If your library has The American Statistician subscription and you are an astronomer handles censored or truncated data sets, this primer would be useful for briefly conceptualizing statistics jargon in survival analysis and for characterizing uncertainties residing in your data. Continue reading ‘Survival Analysis: A Primer’ »

If getting the first derivative (score function) and the second derivative (empirical Fisher information) of a (pseudo) likelihood function is feasible and checking regularity conditions is viable, a test for global maximum (Li and Jiang, JASA, 1999, Vol. 94, pp. 847-854) seems to be a useful reference for verifying the best fit solution. Continue reading ‘A test for global maximum’ »

All models are wrong, but some are useful. –George Box

Continue reading ‘All models are wrong, but some are useful’ »

What if R. A. Fisher was hired by the Royal Observatory in spite that his interest was biology and agriculture, or W. S. Gosset^[1] instead of brewery? An article by E.L. Lehmann made me think this what if. If so, astronomers could have handled errors better than now. Continue reading ‘On the history and use of some standard statistical models’ »

1. Gosset’s pen name was Student, from which the name, Student-t in t-distribution or t-test was spawned.[↩]

A conference that I wanted to go but never made, started today. With relief, they have presentation files from the previous workshop
http://www.stanford.edu/group/mmds and I expect the same for this year. The workshop title may not attract astronomers but the contents, tools, methodologies, and theory are modern astronomy friendly. Astronomers can motivate, initiate, and push further these researchers at the workshop, which I believe currently happening without broad recognitions (foremost interdisciplinary works tend to stay within research groups).

Now it’s time for me to write my own astrostat papers instead of spending time for sieving them from [arXiv]. It has been an irresistible temptation scanning daily [arXiv] preprints to look for astronomy and sometimes statistics papers that 1. adopt statistics, 2. contain statistically challenging problems, 3. could be improved by more rigorous statistical applications, 4. look like abusing statistics, 5. may inspire statisticians by the data sets, or 6. might be useful for astronomers’ advancement in the data analysis. The temptation grew too much to be handled. The amount of papers belong to the above selection criteria seems to grow as my understanding widens. Also the mesh gets loose and starts to show holes. Continue reading ‘Discontinuation of weekly [arXiv] series’ »