The AstroStat Slog » eyeballing http://hea-www.harvard.edu/AstroStat/slog Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 09 Sep 2011 17:05:33 +0000 en-US hourly 1 http://wordpress.org/?v=3.4 appealing eyes == powerful method http://hea-www.harvard.edu/AstroStat/slog/2008/appealing-eyes-powerful-method/ http://hea-www.harvard.edu/AstroStat/slog/2008/appealing-eyes-powerful-method/#comments Sat, 13 Sep 2008 03:30:57 +0000 hlee http://hea-www.harvard.edu/AstroStat/slog/?p=604 To claim results are powerful statistically, astronomers highly rely on eyeballing techniques (need apprenticeship to acquire skills but look subjective to me without such training). Some cases, I know actual statistical tests to support or to dissuade those claims. Hence, I believe astronomers are well aware of those statistical tests. I guess they are afraid that those statistics may reject their claims or are not powerful enough in numeric metrics. Instead, they spend efforts to make graphics more appealing.

One thing I want to point out is the phrase “statistically powerful method” (in the context that they preform statistical data analysis, claimed to be a powerful method) which often misguides me because I cannot find a powerful test statistic in their studies. Please, do not say eyeballing is a powerful test to me. A single outlier could change the result. I do understand the concept of “powerful test” is different as well as the fact that, in terms of good analysis, to make it visually appealing, astronomers provide very elaborated but non mathematical procedures to get rid of outliers, which by itself is very scientific.

I’m not selling this story to all publications in astronomy. There are plenty of papers which describe statistical data analysis in depth. However, there are more claiming that they performed powerful statistical data analysis and produced statistically superior results without statistical inference procedures. Prior to getting rid of unpleasant data points, making graphics appealing and saying an excellent results from statistics, I wish they give a second thought about outliers or statistical lexicons carefully. Also, instead of saying “statistically powerful method,” addressing as it is like “graphics show improved results,” “the result of data analysis is well confined in the predicted region and favors model A,” would make a statistician feel comfortable in reading astronomical journals. Otherwise, it would be better not to put “statistical” or “statistics” in the abstract.

By the way, there is a field called Exploratory Data Analysis, offering statistical eyeballing methods or diagnostic tools. (Papers and books by John Tukey, the inventor of Fast Fourier Transform, despite the time elapse , would be useful to people in any discipline requiring data eyeballing in a statistical fashion. He passed away at the turn of this century but I think he had foreseen the projected direction of statistical data analysis.)

]]>
http://hea-www.harvard.edu/AstroStat/slog/2008/appealing-eyes-powerful-method/feed/ 2