Archive for October 2008

#### A confession from a former “keV” junkie (2. Meet Ms. Electron)

- So, there is a state of matter other than solid, liquid and gas?
= Of course, are you thinking what I am thinking?
- ….
= Yes, it’s time for a jello-shot.
- ….

We cannot deny the arbitrary nature of units we use, but there is also a useful feature: a linkability to other arbitrary units.

The first step of data analysis or applications is reading the data sets into a tool of choice. Recent years, I’ve been using R (see also Learning R) for that regard but I’ve enjoyed freedoms for the same purpose from these languages and tools: BASIC, fortran77/90/95, C/C++, IDL, IRAF, AIPS, mongo/supermongo, MATLAB, Maple, Mathematica, SAS, SPSS, Gauss, ARC, Minitab, and recently Python and ciao which I just began to learn. Many of them I lost the fluency of how to use it. Quick learning tends to be flash memory. Some will need brain defragmentation and recovering time for extensive scientific work. A few I don’t like to use at all. No matter what, I’m not a computer geek. I’m not good at new gadgets, new softwares, nor welcome new and allegedly versatile computing systems. But one must be if he/she want to handle data. Until recently I believed R has such versatility in the aspect of reading in data. Yet, there is nothing without exceptions. Continue reading ‘read.table()’ »

#### missing data

The notions of missing data are overall different between two communities. I tend to think missing data carry as good amount of information as observed data. Astronomers…I’m not sure how they think but my impression so far is that a missing value in one attribute/variable from a object/observation/informant, all other attributes related to that object become useless because that object is not considered in scientific data analysis or model evaluation process. For example, it is hard to find any discussion about imputation in astronomical publication or statistical justification of missing data with respect to inference strategies. On the contrary, they talk about incompleteness within different variables. Putting this vague argument with a concrete example, consider a catalog of multiple magnitudes. To draw a color magnitude diagram, one needs both color and magnitude. If one attribute is missing, that star will not appear in the color magnitude diagram and any inference methods from that diagram will not include that star. Nonetheless, one will trying to understand how different proportions of stars are observed according to different colors and magnitudes. Continue reading ‘missing data’ »

#### Whew

Contact has been re-established with XMM-Newton. Continue reading ‘Whew’ »

#### GSL – GNU Scientific Library

I’ve talked about IMSL on my pyIMSL post, which is a commercial scientific library. There is a GNU version of IMSL, GSL. Finding GSL is the courtesy of Jiangang, who was the author of the poster that I most liked from the 212th AAS, (see My first AAS. V. measurement error and EM and his comment.) Continue reading ‘GSL – GNU Scientific Library’ »

#### “planetariums and other foolishness”

Last month, Senator McCain (R-AZ) wildly dissed on Chicago’s Adler Planetarium, characterizing a funding request on its behalf as “planetariums and other foolishness.” Continue reading ‘“planetariums and other foolishness”’ »

#### Killer App

The iPhone is an amazing device. I have heard that some people use it as a phone, too, but it really is an extraordinary portable computer. It is faster and more powerful than the Sparcstations I used as a grad student, and will fit into your pocket. And most importantly, you can fit an entire planetarium on it.

There are many good planetarium programs that you can access on laptops, but it is really not that much fun to lug them around on camping trips or even out on to the roof at night. But now, thanks to the iPhone (and the iPod Touch) there has been a great leap forward. Continue reading ‘Killer App’ »

#### The Big Picture

Our hometown rag (the Boston Globe) runs an occasional series of photo collections that highlight news stories called The Big Picture. This week, they take a look at the Sun: http://www.boston.com/bigpicture/2008/10/the_sun.html

The pictures come from space and ground observatories, from SoHO, TRACE, Hinode, STEREO, etc. Goes without saying, the images are stunning, and some are even animated. The real kicker is that images such as these are being acquired by the hundreds, every hour upon the hour, 24/7/365.25 . It is like sipping from a firehose. Nobody can sit there and look at them all, so who knows what we are missing out on. Can statistics help? Can we automate a statistically robust “interestingness” criterion to filter the data stream that humans can then follow up on?

#### Off the line

I do not like to be serious. papers…papers…papers. Off from papers for bridging two, allow me to talk about something relevant to the cultural difference between astronomers and statisticians. I hope this could generate a series of comments. Continue reading ‘Off the line’ »

#### [tutorial] multispectral imaging, a case study

Without signal processing courses, the following equation should be awfully familiar to astronomers of photometry and handling data:
$$c_k=\int_\Lambda l(\lambda) r(\lambda) f_k(\lambda) \alpha(\lambda) d\lambda +n_k$$
Terms are in order, camera response (c_k), light source (l), spectral radiance by l (r), filter (f), sensitivity (α), and noise (n_k), where Λ indicates the range of the spectrum in which the camera is sensitive.
Or simplified to $$c_k=\int_\Lambda \phi_k (\lambda) r(\lambda) d\lambda +n_k$$
where φ denotes the combined illuminant and the spectral sensitivity of the k-th channel, which goes by augmented spectral sensitivity. Well, we can skip spectral radiance r, though. Unfortunately, the sensitivity α has multiple layers, not a simple closed function of λ in astronomical photometry.
Or $$c_k=\Theta r +n$$
Inverting Θ and finding a reconstruction operator such that r=inv(Θ)c_k leads spectral reconstruction although Θ is, in general, not a square matrix. Otherwise, approach from indirect reconstruction. Continue reading ‘[tutorial] multispectral imaging, a case study’ »

#### When you register

I bet there are various scams. One of them is automatic user registration. This blog requires a registration for contributing free of approval comments unless one does not put many web links. Recently, there were frequent anonymous user registrations. What I mean by anonymous is that I don’t see their names or part of identities (for example, someone uses initials of their names in their email accounts or uses email accounts from their affiliations). This slog is open to anyone who is interested in AstroStatistics, although not many are currently active. Upon your request, this can be changed very simply and you immediately start writing your ideas about AstroStatistics. However, I must restrict those scams from now on. Please, provide me a small information about you if you do not want to be eliminated after your registration. As I mentioned, the information does not require your full name, nor email account of academic institution. When you register, use your email account that you use daily bases, not the ones that look like results from phishing.

#### [Book] The Grammar of Graphics

All of a sudden, partially owing to a thought provoking talk about visualization by Felice Frankel at IIC, I recollected a book, The Grammar of Graphics by Leland Wilkinson (2nd Ed. – I partially read the 1st ed. and felt little of use several years ago because there seemed no link for visualization of data from astronomy.) Continue reading ‘[Book] The Grammar of Graphics’ »

#### A Quote on Model

In order to understand a learning procedure statistically it is necessary to identify two important aspects: its structural model and its error model. The former is most important since it determines the function space of the approximator, thereby characterizing the class of functions or hypothesis that can be accurately approximated with it. The error model specifies the distribution of random departures of sampled data from the structural model.

#### survey and design of experiments

People of experience would say very differently and wisely against what I’m going to discuss now. This post only combines two small cross sections of each branch of two trees, astronomy and statistics. Continue reading ‘survey and design of experiments’ »