Archive for the ‘Astro’ Category.

May 21st, 2009| 05:55 pm | Posted by hlee

Among billion objects in our Galaxy, outside the Earth, our Sun drags most attention from astronomers. These astronomers go by solar physicists, who enjoy the most abundant data including 400 year long sunspot counts. Their joy is not only originated from the fascinating, active, and unpredictable characteristics of the Sun but also attributed to its influence on our daily lives. Related to the latter, sometimes studying the conditions on the Sun is called **space weather forecast.** Continue reading ‘space weather’ »

Tags:

classifier,

forecast,

logistic regression,

machine learning,

predictor,

response,

space weather,

Sun,

sunspot,

SVM,

test data,

training data,

weather Category:

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Imaging,

Jargon,

Stars,

Stat |

Comment
May 7th, 2009| 02:22 pm | Posted by hlee

Almost 100 years ago, A.S. Eddington stated in his book *Stellar Movements* (1914) that

…in calculating the mean error of a series of observations it is preferable to use the simple mean residual irrespective of sign rather than the mean square residual

Such eminent astronomer said already *least absolute deviation* over *chi-square*, if I match *simple mean residual* and *mean square residual* to relevant methodologies, in order. Continue reading ‘a century ago’ »

Tags:

chi-square minimization,

Eddington,

inference,

LAD,

Laplace,

mse,

PyMC,

Python,

R.A.Fisher,

utility function Category:

Astro,

Cross-Cultural,

Quotes,

Stat,

Uncertainty |

Comment
May 7th, 2009| 11:14 am | Posted by hlee

One of [ArXiv] papers from yesterday whose title might drag lots of attentions from astronomers. Furthermore, it’s a short paper.

[arxiv:math.CO:0905.0483] by Harmany, Marcia, and Willet.

Continue reading ‘[ArXiv] Sparse Poisson Intensity Reconstruction Algorithms’ »

Tags:

compressed sensing,

decomposition,

EM algorithm,

intensity,

MPLE,

multiscale,

penalty,

Poisson,

Poisson Intensity,

Sparcity,

wavelet Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

High-Energy,

Imaging,

Jargon |

Comment
Apr 22nd, 2009| 02:02 pm | Posted by hlee

I was reading Lehmann’s memoir on his friends and colleagues who influence a great deal on establishing his career. I’m happy to know that his meeting Landau, Courant, and Evans led him to be a statistician; otherwise, we, including astronomers, would have had very different textbooks and statistical thinking would have been different. On the other hand, I was surprised to know that he chose statistics over physics due to his experience from Cambridge (UK). I thought becoming a physicist is more preferred than becoming a statistician during the first half of the 20th century. At least I felt that way, probably it’s because more general science books in physics and physics related historic events were well exposed so that I became to think that physicists are more cooler than other type scientists. Continue reading ‘[Book] The Physicists’ »

Tags:

book,

Durrenmatt,

E. L. Lehmann,

Heisenberg,

physicists,

statistician,

uncertainty principle Category:

Cross-Cultural,

Misc,

Physics,

Quotes,

Stat,

Uncertainty |

Comment
Apr 2nd, 2009| 12:00 pm | Posted by hlee

I cannot remember when I first met **Chernoff face** but it hooked me up instantly. I always hoped for confronting multivariate data from astronomy applicable to this charming EDA method. Then, somewhat such eager faded, without realizing what’s happening. Tragically, this was mainly due to my absent mind. Continue reading ‘[MADS] Chernoff face’ »

Tags:

calibration,

Capella,

Chandra,

Chernoff face,

EDA,

line ratios,

MADS,

XAtlas Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Jargon,

Methods,

Misc,

News,

Quotes,

Spectral,

Stars,

X-ray |

2 Comments
Mar 25th, 2009| 10:13 am | Posted by chasc

From Christian Fendt comes this announcement:

——————————————————————

First Announcement and Call for Applications

——————————————————————

The “International Max Planck Research School for Astronomy & Cosmic Physics at the University of Heidelberg” (IMPRS-HD)

announces the

— 4th Heidelberg Summer School:

— Statistical Inferences from Astrophysical Data

— August 10-14, 2009

Continue reading ‘[Announce] Heidelberg Summer School’ »

Tags:

2009,

Announcement,

AstroStat,

August,

Heidelberg,

IMPRS,

IMPRS-HD,

inference,

Max Planck,

summer school Category:

Astro,

Misc,

News,

Stat |

Comment
Mar 24th, 2009| 03:18 pm | Posted by chasc

From Jogesh Babu comes this announcement:

Summer School in Statistics for Astronomers V

June 1-6, 2009

Penn State University

http://astrostatistics.psu.edu/su09/

Continue reading ‘[Announce] AstroStat Summer School at Penn State’ »

Mar 17th, 2009| 03:37 pm | Posted by hlee

I couldn’t believe my eyes when I saw 4754 degrees of freedom (d.f.) and chi-square test statistic 4859. I’ve often enough seen large degrees of freedom from journals in astronomy, several hundreds to a few thousands, but I never felt comfortable at these big numbers. Then with a great shock 4754 d.f. appeared. I must find out why I feel so bothered at these huge degrees of freedom. Continue reading ‘4754 d.f.’ »

Tags:

Binning,

chi-square,

chi-square minimization,

chi-square optimization,

chi-square statistic,

class,

degrees-of-freedom,

equiprobable,

goodness-of-fit test,

kernel density estimation Category:

Bad AstroStat,

Fitting,

High-Energy,

Methods,

Spectral,

X-ray |

2 Comments
Mar 6th, 2009| 03:42 pm | Posted by hlee

Ah ha~ Once I questioned, “what is systematic error?” (see [Q] systematic error.) Thanks to L. Lyons’ work discussed in [ArXiv] Particle Physics, I found this paper, titled **Systematic Errors** describing the concept and statistical inference related to **systematic errors** in the field of particle physics. It, gladly, shares lots of similarity with high energy astrophysics. Continue reading ‘systematic errors’ »

Tags:

coverage,

Heinrich,

likelihood,

Lyons,

nuisance parameter,

objective priors,

p-value,

particle physics,

statistical error,

subjective priors,

systematic error Category:

Algorithms,

arXiv,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

Jargon,

Misc,

News,

Physics,

Stat,

Uncertainty |

Comment
Feb 26th, 2009| 04:07 pm | Posted by hlee

I’ve been complaining about *how one can do machine learning on solar images without a training set?* (see my comment at the big picture). On the other hand, I’m also aware of challenges in astronomy that data (images) cannot be transformed freely and be fed into standard machine learning algorithms. Tailoring data pipelining, cleaning, and processing to currently existing vision algorithms may not be achievable. The hope of automatizing the detection/identification procedure of interesting features (e.g. flares and loops) and forecasting events on the surface of the Sun is only a dream. Even though the level of image data stream is that of tsunami, we might have to depend on human eyes to comb out interesting features on the Sun until the new paradigm of automatized feature identification algorithms based on a single image i.e. without a training set. The good news is that human eyes have done a superb job! Continue reading ‘An excerpt from …’ »

Tags:

brains,

computer vision,

human eyes,

Kendall,

machine learning,

shape theory,

Sun,

tsunami Category:

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Imaging,

Quotes |

Comment
Feb 20th, 2009| 07:48 pm | Posted by hlee

[stat.AP:0811.1663]

*Open Statistical Issues in Particle Physics* by **Louis Lyons**

My recollection of meeting Prof. L. Lyons was that he is very kind and listening. I was delighted to see his introductory article about particle physics and its statistical challenges from an [arxiv:stat] email subscription. Continue reading ‘[ArXiv] Particle Physics’ »

Tags:

chi-square,

chi-square minimization,

coverage,

hypothesis testing,

L.Lyons,

LHC,

LRT,

particle physics,

posterior distribution Category:

arXiv,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

High-Energy,

Methods,

Physics,

Stat |

Comment
Feb 10th, 2009| 04:37 pm | Posted by hlee

A continuation from my posting, titled circumspect frequentist.

Title: **Statistical Models: Theory and Practice** (click for the publisher’s website)

My one line review, rather a comment several months ago was

Bias in asymptotic standard errors is not a familiar topic for astronomers

and I don’t understand why I wrote it but I think I came up this comment owing to my pursuit of modeling measurement errors occurring in astronomical researches. Continue reading ‘A book by David Freedman’ »

Jan 20th, 2009| 01:59 pm | Posted by hlee

Someone emailed me for globular cluster data sets I used in a proceeding paper, which was about how to determine the multi-modality (multiple populations) based on well known and new information criteria without binning the luminosity functions. I spent quite time to understand the data sets with suspicious numbers of globular cluster populations. On the other hand, obtaining globular cluster data sets was easy because of available data archives such as VizieR. Most data sets in charts/tables, I acquire those data from VizieR. In order to understand science behind those data sets, I check ADS. Well, actually it happens the other way around: check scientific background first to assess whether there is room for statistics, then search for available data sets. Continue reading ‘accessing data, easier than before but…’ »

Tags:

archive,

ascii,

catalog,

CDA,

data analysis,

data mining,

database,

Gator,

globular cluster,

inference,

massive data,

multimodality,

multiple populations,

NED,

SDSS,

statistical inference,

statistician,

streaming data,

table,

tabulated,

visieR Category:

Algorithms,

Astro,

Cross-Cultural,

Data Processing,

Jargon,

Meta,

Nuggets,

Objects |

3 Comments
Jan 15th, 2009| 06:01 pm | Posted by hlee

I wonder what Fisher, Neyman, and Pearson would say if they see “Technique” after “Likelihood Ratio” instead of “Test.” A presenter’s saying “Likelihood Ratio Technique” for source identification, I couldn’t resist checking it out not to offend founding fathers of the likelihood principle in statistics since “Technique” sounded derogatory to be attached with “Likelihood” to my ears. I thank, above all, the speaker who kindly gave me the reference about this likelihood ratio technique. Continue reading ‘Likelihood Ratio Technique’ »

Tags:

Fisher,

likelihood principle,

likelihood ratio technique,

likelihood ratio test,

Neyman,

Pearson Category:

Algorithms,

arXiv,

Astro,

Bayesian,

Cross-Cultural,

Data Processing,

Fitting,

Frequentist,

Jargon,

Methods,

Objects,

Stat,

Uncertainty |

Comment
Jan 2nd, 2009| 11:24 pm | Posted by vlk

You would think that something like “measurement error” is a well-defined concept, and everyone knows what it means. Not so. I have so far counted at least 3 different interpretations of what it means.

Suppose you have measurements *X={X*_{i}, i=1..N} of a quantity whose true value is, say, *X*_{0}. One can then compute the mean and standard deviation of the measurements, *E(X)* and *σ*_{X}. One can also infer the value of a parameter *θ(X)*, derive the posterior probability density *p(θ|X)*, and obtain confidence intervals on it.

So here are the different interpretations:

- Measurement error is
*σ*_{X}, or the spread in the measurements. Astronomers tend to use the term in this manner.
- Measurement error is
*X*_{0}-E(X), or the “error made when you make the measurement”, essentially what is left over beyond mere statistical variations. This is how statisticians seem to use it, essentially the bias term. To quote David van Dyk

For us it is just English. If your measurement is different from the real value. So this is not the Poisson variability of the source for effects or ARF, RMF, etc. It would disappear if you had a perfect measuring device (e.g., telescope).

- Measurement error is the width of
*p(θ|X)*, i.e., the measurement error of the first type propagated through the analysis. Astronomers use this too to refer to measurement error.

Who am I to say which is right? But be aware of who you may be speaking with and be sure to clarify what you mean when you use the term!