The AstroStat Slog

Astronomers have developed their ways of processing signals almost independent to but sometimes collaboratively with engineers, although the fundamental of signal processing is same: extracting information. Doubtlessly, these two parallel roads of astronomers’ and engineers’ have been pointing opposite directions: one toward the sky and the other to the earth. Nevertheless, without an intensive argument, we could say that somewhat statistics has played the medium of signal processing for both scientists and engineers. This particular issue of IEEE signal processing magazine may shed lights for astronomers interested in signal processing and statistics outside the astronomical society.

IEEE Signal Processing Magazine Jul. 2007 Vol 24 Issue 4: Bootstrap methods in signal processing

This link will show the table of contents and provide links to articles; however, the access to papers requires IEEE Xplore subscription via libraries or individual IEEE memberships). Here, I’d like to attempt to introduce some articles and tutorials.
Continue reading ‘Signal Processing and Bootstrap’ »

I have been observing some sorts of misconception about statistics and statistical nomenclature evolution in astronomy, which I believe, are attributed to the lack of references in the astronomical society. There are some textbooks designed for junior/senior science and engineering students, which are likely unknown to astronomers. Example-wise, these books are not suitable, to my knowledge. Although I never expect astronomers to learn standard graduate (mathematical) statistics textbooks, I do wish astronomers go beyond Numerical Recipes (W. H. Press, S. A. Teukolsky, W. T. Vetterling, & B. P. Flannery) and Error Data Reduction and Analysis for the Physical Sciences (P. R. Bevington & D. K. Robinson). Here are some good ones written by astronomers, engineers, and statisticians: Continue reading ‘Books – a boring title’ »

Only three papers this week. There were a few more with chi-square fitting and its error bars but excluded. Continue reading ‘[ArXiv] 4th week, Jan. 2008’ »

The High Energy Astrophysics Division of the American Astronomical Society will meet at Los Angeles on March 31 – April 3, and we have been allocated a slot for an AstroStatistics session. It will be a 60-minute lunch-time session, so we anticipate that the session will be dominated by poster haikus and panel discussions similar to the workshop we held during the New Orleans meeting in 2004.

The meeting website is at: http://www.confcon.com/head2008/.The abstract submission deadline is January 25, 2008 (now past, but late abstracts are not unheard of among astronomers).

If you are attending the meeting, and plan to present posters or talks that deal with astrostatistical methods or techniques, we welcome you to participate in this session. When you submit an abstract, be sure to indicate a category of “Other” and in the comments field state that it belongs with the AstroStatistics special session.If you have questions, please contact Aneta or me. There is also a page for this session on the astrostat google groups site.

Update (1/22): The abstract submission page currently says that only one abstract is allowed per person. We have been informed that this is incorrect, and that people can submit two abstracts, one for the special session and one as a regular contribution. Note that posters will be up only one day, and those associated with a special session will be put up the day of the session.

Update (1/26): A detailed program is not yet available, but here is a description of the session:

Astrostatistics: Methods and Techniques

This session will provide a forum for the discussion and presentation of statistical challenges in high energy astrophysics, highlighting the great deal of progress that has been made in methods and techniques over the past decade. The one hour session will cover the current and future directions in Astrostatistics, and will include a discussion of MCMC methods in the context of specific applications (such as propagating calibration errors, defining the significance of image features, etc.); a discussion of standardized methods for computing detection limits, upper limits, and confidence intervals for weak sources; and hypothesis testing and its limitations (including the significance testing of emission lines).

Update (2/19): We have been allocated the mid-day slot of March 31. The session will run from 12:30pm till 1:30pm2pm. The tentative program is as follows:

• Remarks on current and future trends in AstroStatistics, by Eric Feigelson
• Poster haiku
• F-Test theory and usage, by David van Dyk
• Discussion on MCMC techniques, led by Andy Ptak

Update (2/26): The final program is out, and the AstroStat session is scheduled for 12:30pm-2pm at the Museum/Bunker Hill Room.

Update (4/1): The talks and posters associated with the AstroStat special session are now online at
http://hea-www.harvard.edu/AstroStat/HEAD2008/. Additional comments and descriptions will be archived there.

One of the big problems that has come up in recent years is in how to represent the uncertainty in certain estimates. Astronomers usually present errors as +-stddev on the quantities of interest, but that presupposes that the errors are uncorrelated. But suppose you are estimating a multi-dimensional set of parameters that may have large correlations amongst themselves? One such case is that of Differential Emission Measures (DEM), where the “quantity of emission” from a plasma (loosely, how much stuff there is available to emit — it is the product of the volume and the densities of electrons and H) is estimated for different temperatures. See the plots at the PoA DEM tutorial for examples of how we are currently trying to visualize the error bars. Another example is the correlated systematic uncertainties in effective areas (Drake et al., 2005, Chandra Cal Workshop). This is not dissimilar to the problem of determining the significance of a “feature” in an image (Connors, A. & van Dyk, D.A., 2007, SCMA IV). Continue reading ‘Dance of the Errors’ »

Seven preprints were chosen this week and two mentioned model selection. Continue reading ‘[ArXiv] 3rd week, Jan. 2008’ »

It is notable that there’s an astronomy paper contains AIC, BIC, and Bayesian evidence in the title. The topic of the paper, unexceptionally, is cosmology like other astronomy papers discussed these (statistical) information criteria (I only found a couple of papers on model selection applied to astronomical data analysis without articulating CMB stuffs. Note that I exclude Bayes factor for the model selection purpose).

To find the paper or other interesting ones, click Continue reading ‘[ArXiv] 2nd week, Jan. 2007’ »

From Guaranteed Margins for LQG Regulartors J.C. Doyle (1978) IEEE Transactions on Automatic Control 23(4), pp. 756- 757

The abstract has one sentence: There are none and the first paragraph of this short paper explains the uniqueness of the abstract: Continue reading ‘[Quote] Abstract – There are none.’ »

Although it is a great read, Numerical Recipe^[1] is no more suitable as a statistical bible than Ptolemy is for astronomy.

Continue reading ‘[Quote] The “Bible”’ »

1. W.H.Press, S.A.Teukolsky, W.T. Vetterling, and B.P.Flannery, 2nd ed., 1992[↩]

It’s a rather short list, this week and I hope I can maintain this conciseness afterwards. Happy new year to everyone. Continue reading ‘[ArXiv] 1st week, Jan. 2008’ »

I found this website a while ago but haven’t checked until now. They are quite useful by its contents (even pages of the lecture notes are properly flipped for you while the lecture is given). Increasing popularity of machine learning among astronomers will find more use of such lectures. If you have time to learn machine learning and other related subjects, please visit http://videolectures.net/. Specifically classified links to interesting subjects are found by your click. Continue reading ‘On-line Machine Learning Lectures and Notes’ »