The AstroStat Slog » E. L. Lehmann http://hea-www.harvard.edu/AstroStat/slog Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 09 Sep 2011 17:05:33 +0000 en-US hourly 1 http://wordpress.org/?v=3.4 Erich Lehmann http://hea-www.harvard.edu/AstroStat/slog/2009/erich-lehmann/ http://hea-www.harvard.edu/AstroStat/slog/2009/erich-lehmann/#comments Tue, 08 Dec 2009 04:46:34 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/2009/erich-lehmann/ He was one of the frequently cited statisticians in this slog because of his influence in statistics. It is extremely difficult to avoid his textbooks and his establishment of theoretical statistics when one begins to comprehend and to appreciate the modern theoretical statistics. To me, Testing Statistical Hypotheses and Theory of Point Estimation are two pillars of graduate statistical education. In addition, Elements of Large Sample Theory and Nonparametrics: Statistical Methods Based on Ranks are also eye openers.

It has not been long since I read Reminiscences of a Statistician: The Company I Kept. I quoted this book and an arXiv paper here :see the posts. I became very grateful to him because of his contributions to the statistical science. I feel so sad to see his obituary, particularly when I’m soon going to have time for reading his books more carefully.

]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/erich-lehmann/feed/ 0
[Book] The Physicists http://hea-www.harvard.edu/AstroStat/slog/2009/book-the-physicists/ http://hea-www.harvard.edu/AstroStat/slog/2009/book-the-physicists/#comments Wed, 22 Apr 2009 19:02:44 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=2157 I was reading Lehmann’s memoir on his friends and colleagues who influence a great deal on establishing his career. I’m happy to know that his meeting Landau, Courant, and Evans led him to be a statistician; otherwise, we, including astronomers, would have had very different textbooks and statistical thinking would have been different. On the other hand, I was surprised to know that he chose statistics over physics due to his experience from Cambridge (UK). I thought becoming a physicist is more preferred than becoming a statistician during the first half of the 20th century. At least I felt that way, probably it’s because more general science books in physics and physics related historic events were well exposed so that I became to think that physicists are more cooler than other type scientists.

The Physicists by Friedrich Durrenmatt

This short play (wiki link) is very charming and fun to read. Some statisticians would enjoy it. A subset of readers might embrace physics instead of repelling it. At least, it would show different aspects of non statistical science to statisticians beyond genetics, biology, medical science, economics, sociology, agricultural science, psychology, meteorology, and so on, where interdisciplinary collaborations are relatively well established.

The links for The Physicists and the book by Lehmann below were from Amazon.

Reminiscences of a Statistician: The Company I Kept by Erich Lehmann

The following excerpt from Reminiscence…, however, was more interesting how statistics were viewed to young Lehmann because I felt the same prior to learning statistics and watching how statistics were used in astronomical data analysis.

… I did not like it (statistics). It was lacking the element that had attracted me to mathematics as a boy: statistics did not possess the beauty that I have found in the integers and later in other parts of mathematics. Instead, ad hoc methods were used to solve problems that were messy and that where based on questionable assumptions that seemed quite arbitrary.

Aside, I have another post on his article , On the history and use of some standard statistical models.

I’d like to recommend another with a hope that someone finds its english translation (I have been searching but kept failed).

Der Teil und das Ganze by Werner Heisenberg.

YES, Heisenberg of the uncertainty principle! My understanding is that the notion of uncertainty is different among physicists, statisticians, and modern astronomers. I think it has evolved without communications.

Related to uncertainty, I also want to recommend again Professor Lindley’s insightful paper, discussed in another post, Statistics is the study of uncertainty.

Not many statisticians are exposed to (astro)physics and vice versa, probably the primary reason of wasting time on explaining λ (poisson rate parameter vs. wavelength), ν (nuisance parameter vs. frequency), π or φ (pdfs vs. particles), Ω (probability space vs. cosmological constant), Ho (null hypothesis vs. Hubble constant), to name a few. I hope this general reading recommendations is useful to narrow gaps and time wastes.

]]>
http://hea-www.harvard.edu/AstroStat/slog/2009/book-the-physicists/feed/ 0
On the history and use of some standard statistical models http://hea-www.harvard.edu/AstroStat/slog/2008/statistical-model-history/ http://hea-www.harvard.edu/AstroStat/slog/2008/statistical-model-history/#comments Fri, 27 Jun 2008 00:03:11 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=343 What if R. A. Fisher was hired by the Royal Observatory in spite that his interest was biology and agriculture, or W. S. Gosset[1] instead of brewery? An article by E.L. Lehmann made me think this what if. If so, astronomers could have handled errors better than now.

Every statistician, at least to my knowledge, knows E.L. Lehmann (his TPE and TSH are classic and Elements of Large Sample Theory was my textbook). Instead of reading daily astro-ph, I’m going through some collected papers from arxiv:stat and other references, in order to recover my hopefully existing research oriented mind (I like to share my stat or astrostat papers with you) and to continue slogging. The first one is an arxiv:math.ST paper by Lehmann.

His foremost knowledge and historic account on statistical models related to errors may benefit astronomers. Although I didn’t study history of astronomy and statistics, I’m very much aware of how astronomy innovated statistical thinking, particularly the area of large sample theory. Unfortunately, at the dawn of the 20th century, they went through an unwanted divorce. Papers from my [arXiv] series or a small portion of statistics papers citing astronomy, seem to pay high alimony without tax relief.

[math.ST:0805.2838] E.L.Lehmann
On the history and use of some standard statistical models

According to the author, the paper considers three assumptions: normality, independence, and the linear structure of the deterministic part. The particular reason for this paper into the slog is the following sentences:

The normal distribution became the acknowledged model for the distribution of errors of physical (particularly astronomical) measurements and was called the Law of Errors. It has a theoretical basis in the so called Law of Elementary Erros which assumed that an observational error is the sum of a large number of small independent errors and is therefore approximately normally distributed by the Central Limit Theorem.

A lot to be said but adding a quote referring Freedman that

“one problem noticeable to a statistician is that investigators do not pay attention to the stochastic assumptions behind the models. It does not seem possible to derive these assumptions from current theory, nor are they easily validated empirically on a case-by-case basis.”
The paper ends with the devastating conclusion:
“My opinion is that investigators need to think more about the underlying process, and look more closely at the data, without the distorting prism of convential (and largely irrelevant) stochastic models. Estimating nonexistent parameters cannot be very fruitful. And it must be equally a waste of time to test theories on the basis of statistical hypothesis that are rooted neither in prior theory nor in fact, even if the algorithms are recited in every statistics text without caveat.”

It is truly devastating.

A quote in the article referring the Preface of Snedecor’s book clearly tells the importance of collaborations.

“To the mathematical statistician must be delegated the task of developing the theory and devising the methods, accompanying these latter by adequate statements of the limitations of their use. …
None but the biologist can decide whether the conditions are fulfilled in his experiments.”

so does two sentences from the paper in the conclusion

A general text cannot provide the subject matter knowledge and the special features that are needed for successful modeling in specific cases. Experience with similar data is required, knowledge of theory and, as Freedman points out: shoe leather.

Other quotes in the article referring Scheffe,

“the effect of violation of the normality assumption is slightly on inferences about the mean but dangerous on inferences about variances.”

and Brownlee,

“applied statisticians have found empirically that usually there is no great need to fuss about the normality assumption”

and I confess that I’ve been fussing about astronomers’ gaussianity assumption; on the contrary, I advice my friends in other disciplines (for example, agriculture) treating their data with simpler analytic tools by assuming normality. To defend myself, I like to ask whether the independence assumption can be overlooked at the convenience of multiplying marginalized probabilities. I don’t think such concern/skepticism has not been addressed well enough compared to the normality assumption.

  1. Gosset’s pen name was Student, from which the name, Student-t in t-distribution or t-test was spawned.
]]>
http://hea-www.harvard.edu/AstroStat/slog/2008/statistical-model-history/feed/ 0