Archive for the ‘Cross-Cultural’ Category.

Jun 2nd, 2009| 03:03 am | Posted by hlee

Even though I traced the astronomers’ casual usage of the **null hypothesis probability** in a fashion of reporting outputs from data analysis packages of their choice, there were still some curious cases of the **null hypothesis probability** that I couldn’t solve. They are quite mysterious to me. Sometimes too much creativity harms the original intention. Here are some examples. Continue reading ‘Curious Cases of the Null Hypothesis Probability’ »

Tags:

cases,

chi-sq,

curious,

degree of freedom,

dof,

F-test,

goodness-of-fit test,

Model Selection,

null hypothesis probability,

p-value,

reduced chi-sq Category:

arXiv,

Astro,

Cross-Cultural,

Fitting,

Methods,

Uncertainty |

3 Comments
Jun 1st, 2009| 09:51 pm | Posted by hlee

How would you assign orders to multivariate data? If you have your strategy to achieve this ordering task, I’d like to ask, “is your strategy **affine invariant**?” meaning that shift and rotation invariant. Continue reading ‘[MADS] data depth’ »

Tags:

break points,

data depth,

MADS,

mean,

median,

multivariate,

nonparametric,

order,

parasite,

quantile,

robust,

sort,

vertebrate Category:

Algorithms,

arXiv,

Cross-Cultural,

Jargon,

Stat |

Comment
May 21st, 2009| 05:55 pm | Posted by hlee

Among billion objects in our Galaxy, outside the Earth, our Sun drags most attention from astronomers. These astronomers go by solar physicists, who enjoy the most abundant data including 400 year long sunspot counts. Their joy is not only originated from the fascinating, active, and unpredictable characteristics of the Sun but also attributed to its influence on our daily lives. Related to the latter, sometimes studying the conditions on the Sun is called **space weather forecast.** Continue reading ‘space weather’ »

Tags:

classifier,

forecast,

logistic regression,

machine learning,

predictor,

response,

space weather,

Sun,

sunspot,

SVM,

test data,

training data,

weather Category:

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Imaging,

Jargon,

Stars,

Stat |

Comment
May 7th, 2009| 02:22 pm | Posted by hlee

Almost 100 years ago, A.S. Eddington stated in his book *Stellar Movements* (1914) that

…in calculating the mean error of a series of observations it is preferable to use the simple mean residual irrespective of sign rather than the mean square residual

Such eminent astronomer said already *least absolute deviation* over *chi-square*, if I match *simple mean residual* and *mean square residual* to relevant methodologies, in order. Continue reading ‘a century ago’ »

Tags:

chi-square minimization,

Eddington,

inference,

LAD,

Laplace,

mse,

PyMC,

Python,

R.A.Fisher,

utility function Category:

Astro,

Cross-Cultural,

Quotes,

Stat,

Uncertainty |

Comment
May 7th, 2009| 11:14 am | Posted by hlee

One of [ArXiv] papers from yesterday whose title might drag lots of attentions from astronomers. Furthermore, it’s a short paper.

[arxiv:math.CO:0905.0483] by Harmany, Marcia, and Willet.

Continue reading ‘[ArXiv] Sparse Poisson Intensity Reconstruction Algorithms’ »

Tags:

compressed sensing,

decomposition,

EM algorithm,

intensity,

MPLE,

multiscale,

penalty,

Poisson,

Poisson Intensity,

Sparcity,

wavelet Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

High-Energy,

Imaging,

Jargon |

Comment
Apr 27th, 2009| 05:26 pm | Posted by hlee

To my knowledge, **Richard Feynman** is an iconic figure among physicists and astrophysicists. Although I didn’t read every chapter of his lecture series, from other books like* QED*, *Surely You’re Joking, Mr. Feynman!*, *The Pleasure of Finding Things Out*, and some essays, I became and still am fond of him. The way how this famous physicist put things is straight and simple, blowing out the misconception that physics is full of mathematical equations.

Even though most of my memories about his writings are gone – how many people can beat the time and fading memories! – like other rudimentary astronomy and physics stuffs that I used to know, statistics brought up his name above the surface before it sinks completely to the abyss. Continue reading ‘Feynman and Statistics’ »

Apr 22nd, 2009| 02:02 pm | Posted by hlee

I was reading Lehmann’s memoir on his friends and colleagues who influence a great deal on establishing his career. I’m happy to know that his meeting Landau, Courant, and Evans led him to be a statistician; otherwise, we, including astronomers, would have had very different textbooks and statistical thinking would have been different. On the other hand, I was surprised to know that he chose statistics over physics due to his experience from Cambridge (UK). I thought becoming a physicist is more preferred than becoming a statistician during the first half of the 20th century. At least I felt that way, probably it’s because more general science books in physics and physics related historic events were well exposed so that I became to think that physicists are more cooler than other type scientists. Continue reading ‘[Book] The Physicists’ »

Tags:

book,

Durrenmatt,

E. L. Lehmann,

Heisenberg,

physicists,

statistician,

uncertainty principle Category:

Cross-Cultural,

Misc,

Physics,

Quotes,

Stat,

Uncertainty |

Comment
Apr 20th, 2009| 09:34 pm | Posted by hlee

I asked a couple of astronomers if they heard the term **plug-in estimator** and none of them gave me a positive answer. Continue reading ‘[MADS] plug-in estimator’ »

Tags:

biased,

breakdown point,

chi-square,

confidence interval,

coverage,

delta chi-square,

estimator,

LAD,

mean,

median,

plug-in,

rmse Category:

Bad AstroStat,

Cross-Cultural,

Data Processing,

Jargon,

Uncertainty |

2 Comments
Apr 16th, 2009| 07:32 am | Posted by vlk

http://www.tricki.org/

The wikipedia-like repository for mathematical “tricks” has now gone live. Their mission statement:

The main body of the Tricki will be a (large, if all goes according to plan) collection of articles about methods for solving mathematical problems. These will be everything from very general problem-solving tips such as, “If you can’t solve the problem, then try to invent an easier problem that sheds light on it,” to much more specific advice such as, “If you want to solve a linear differential equation, you can convert it into a polynomial equation by taking the Fourier transform.”

Tags:

Announcement,

math,

mathematics,

maths,

tricki,

tricks,

website,

Wikipedia Category:

Algorithms,

Cross-Cultural,

Jargon,

News,

Stat |

Comment
Apr 2nd, 2009| 12:00 pm | Posted by hlee

I cannot remember when I first met **Chernoff face** but it hooked me up instantly. I always hoped for confronting multivariate data from astronomy applicable to this charming EDA method. Then, somewhat such eager faded, without realizing what’s happening. Tragically, this was mainly due to my absent mind. Continue reading ‘[MADS] Chernoff face’ »

Tags:

calibration,

Capella,

Chandra,

Chernoff face,

EDA,

line ratios,

MADS,

XAtlas Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Jargon,

Methods,

Misc,

News,

Quotes,

Spectral,

Stars,

X-ray |

2 Comments
Mar 31st, 2009| 03:43 pm | Posted by hlee

Before using any adaptations of chi-square statistic, please spend a minute or two to ponder whether your strategy with chi-square belongs one of these categories.

**1**. Lack of independence among the single events or measures

**2**. Small theoretical frequencies

**3**. Neglect of frequencies of non-occurrence

**4**. Failure to equalize \sum O_i (the sum of the observed frequencies) and \sum M_i (the sum of the theoretical frequencies)

**5**. Indeterminate theoretical frequencies

**6**. Incorrect or questionable categorizing

**7**. Use of non-frequency data

**8**. Incorrect determination of the number of degrees of freedom

**9**. Incorrect computations (including a failure to weight by N when proportions instead of frequencies are used in the calculations)

From “**Chapter 10: On the Use and Misuse of Chi-square**” by K.L.Delucchi in *A Handbook for Data Analysis in the Behavioral Sciences* (1993). Delucchi acknowledged these nine principle sources of error to Lewis and Burke (1949), entitled “The Use and Misuse of the Chi-square” published in *Psychological Bulletin.* Continue reading ‘Use and Misuse of Chi-square’ »

Mar 18th, 2009| 08:30 pm | Posted by hlee

I was disappointed when video, audio, or handout files were not available from the research program “Statistical Theory and Methods for Complex High-Dimensional Data” held at Isaac Newton Institute for Mathematical Sciences during the first half of last year after checking the sites several times. Wow…They are now there~ Continue reading ‘Web Seminar’ »

Mar 11th, 2009| 01:04 pm | Posted by hlee

by **T. Cover and J. Thomas** website: http://www.elementsofinformationtheory.com/

Once, perhaps more, I mentioned this book in my post with the most celebrated paper by Shannon (see the posting). Some additional recommendation of the book has been made to answer offline inquiries. And this book always has been in my favorite book list that I like to use for teaching. So, I’m not shy with recommending this book to astronomers with modern objective perspectives and practicality. Before advancing for more praises, I must say that those admiring words do not imply that I understand every line and problem of the book. Like many fields, Information theory has grown fast since the monumental debut paper by Shannon (1948) like the speed of astronomers observation techniques. Without the contents of this book, most of which came after Shannon (1948), internet, wireless communication, compression, etc could not have been conceived. Since the notion of “**entropy**“, the core of **information theory**, is familiar to astronomers (physicists), the book would be received better among them than statisticians. This book should be read easier to astronomers than statisticians. Continue reading ‘[Book] Elements of Information Theory’ »

Tags:

bandwidth,

book,

Cover,

data mining,

education,

Entropy,

Information theory,

Kolmogorov complexity,

Shannon,

Thomas Category:

Algorithms,

arXiv,

Cross-Cultural,

Data Processing,

Jargon,

Quotes |

Comment
Mar 9th, 2009| 05:18 pm | Posted by hlee

It bears the name of its inventor, Prasanta Chandra Mahalanobis. As opposed to the Euclidean distance, a household name, the name of this distance is rarely used but many pseudonyms exist with variations adapted into broad scientific disciplines and applications. Therefore, under different names, I believe that the Mahalanobis distance is frequently applied in exploring and analyzing astronomical data. Continue reading ‘[MADS] Mahalanobis distance’ »

Mar 6th, 2009| 03:42 pm | Posted by hlee

Ah ha~ Once I questioned, “what is systematic error?” (see [Q] systematic error.) Thanks to L. Lyons’ work discussed in [ArXiv] Particle Physics, I found this paper, titled **Systematic Errors** describing the concept and statistical inference related to **systematic errors** in the field of particle physics. It, gladly, shares lots of similarity with high energy astrophysics. Continue reading ‘systematic errors’ »

Tags:

coverage,

Heinrich,

likelihood,

Lyons,

nuisance parameter,

objective priors,

p-value,

particle physics,

statistical error,

subjective priors,

systematic error Category:

Algorithms,

arXiv,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

Jargon,

Misc,

News,

Physics,

Stat,

Uncertainty |

Comment