Archive for the ‘arXiv’ Category.

Aug 12th, 2009| 06:03 pm | Posted by hlee

Statistical Resampling Methods are rather unfamiliar among astronomers. Bootstrapping can be an exception but I felt like it’s still unrepresented. Seeing an recent review paper on **cross validation** from [arXiv] which describes basic notions in theoretical statistics, I couldn’t resist mentioning it here. **Cross validation** has been used in various statistical fields such as classification, density estimation, model selection, regression, to name a few. Continue reading ‘[ArXiv] Cross Validation’ »

Tags:

ADS,

cross-validation,

machine learning,

Model Selection,

n-fold Category:

arXiv,

Astro,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

Jargon,

Methods,

Quotes,

Stat |

Comment
Jul 29th, 2009| 01:02 am | Posted by hlee

Speaking of XAtlas from my previous post I tried another visualization tool called **Parallel Coordinates** on these Capella observations and two stars with multiple observations (AL Lac and IM Peg). As discussed in [MADS] Chernoff face, full description of the catalog is found from XAtlas website. The reason for choosing these stars is that among low mass stars, next to Capella (I showed 16), IM PEG (HD 21648, 8 times), and AR Lac (although different phases, 6 times) are most frequently observed. I was curious about which variation, within (statistical variation) and between (Capella, IM Peg, AL Lac), is dominant. How would they look like from the parametric space of High Resolution Grating Spectroscopy from Chandra? Continue reading ‘[MADS] Parallel Coordinates’ »

Tags:

Classification,

clustering,

display,

EDA,

eye catcher,

GGobi,

Inselberg,

parallel coordinates,

visualization Category:

Algorithms,

arXiv,

Cross-Cultural,

Data Processing,

High-Energy,

Jargon,

Methods,

Spectral,

X-ray |

3 Comments
Jun 12th, 2009| 03:47 pm | Posted by hlee

**A Fast Thresholded Landweber Algorithm for Wavelet-Regularized Multidimensional Deconvolution**

Vonesch and Unser (2008)

IEEE Trans. Image Proc. vol. 17(4), pp. 539-549

Quoting the authors, I also like to say that __the recovery of the original image from the observed is an __**ill-posed** problem. They traced the efforts of wavelet regularization in deconvolution back to a few relatively recent publications by astronomers. Therefore, I guess the topic and algorithm of this paper could drag some attentions from astronomers. Continue reading ‘Wavelet-regularized image deconvolution’ »

Tags:

bound optimization,

deconvolution,

image processing,

impulse response,

MM algorithm,

PSF,

regularization,

restoration,

thresholding,

wavelet Category:

Algorithms,

arXiv,

Data Processing,

Fitting,

Imaging,

Jargon,

Methods,

Quotes,

Stat |

Comment
Jun 4th, 2009| 04:36 pm | Posted by hlee

Please, do not confuse **adaptive filter** (hereafter, **AF**) with **adaptive optics** (hereafter, **AO**). I have no expertise in both fields but have small experiences to tell you the difference. Simply put, **AF** is comparable to software as opposed to **AO** to hardware, which is for constructing telescopes in order to collect data with sharpness and to minimize time varying atmospheric blurring. When you search **adaptive filter** in ADS you’ll more likely come across with **adaptive optics** and **notch filter**. Continue reading ‘[MADS] Adaptive filter’ »

Jun 2nd, 2009| 03:03 am | Posted by hlee

Even though I traced the astronomers’ casual usage of the **null hypothesis probability** in a fashion of reporting outputs from data analysis packages of their choice, there were still some curious cases of the **null hypothesis probability** that I couldn’t solve. They are quite mysterious to me. Sometimes too much creativity harms the original intention. Here are some examples. Continue reading ‘Curious Cases of the Null Hypothesis Probability’ »

Tags:

cases,

chi-sq,

curious,

degree of freedom,

dof,

F-test,

goodness-of-fit test,

Model Selection,

null hypothesis probability,

p-value,

reduced chi-sq Category:

arXiv,

Astro,

Cross-Cultural,

Fitting,

Methods,

Uncertainty |

3 Comments
Jun 1st, 2009| 09:51 pm | Posted by hlee

How would you assign orders to multivariate data? If you have your strategy to achieve this ordering task, I’d like to ask, “is your strategy **affine invariant**?” meaning that shift and rotation invariant. Continue reading ‘[MADS] data depth’ »

Tags:

break points,

data depth,

MADS,

mean,

median,

multivariate,

nonparametric,

order,

parasite,

quantile,

robust,

sort,

vertebrate Category:

Algorithms,

arXiv,

Cross-Cultural,

Jargon,

Stat |

Comment
May 21st, 2009| 05:55 pm | Posted by hlee

Among billion objects in our Galaxy, outside the Earth, our Sun drags most attention from astronomers. These astronomers go by solar physicists, who enjoy the most abundant data including 400 year long sunspot counts. Their joy is not only originated from the fascinating, active, and unpredictable characteristics of the Sun but also attributed to its influence on our daily lives. Related to the latter, sometimes studying the conditions on the Sun is called **space weather forecast.** Continue reading ‘space weather’ »

Tags:

classifier,

forecast,

logistic regression,

machine learning,

predictor,

response,

space weather,

Sun,

sunspot,

SVM,

test data,

training data,

weather Category:

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Imaging,

Jargon,

Stars,

Stat |

Comment
May 7th, 2009| 11:14 am | Posted by hlee

One of [ArXiv] papers from yesterday whose title might drag lots of attentions from astronomers. Furthermore, it’s a short paper.

[arxiv:math.CO:0905.0483] by Harmany, Marcia, and Willet.

Continue reading ‘[ArXiv] Sparse Poisson Intensity Reconstruction Algorithms’ »

Tags:

compressed sensing,

decomposition,

EM algorithm,

intensity,

MPLE,

multiscale,

penalty,

Poisson,

Poisson Intensity,

Sparcity,

wavelet Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

High-Energy,

Imaging,

Jargon |

Comment
Apr 2nd, 2009| 12:00 pm | Posted by hlee

I cannot remember when I first met **Chernoff face** but it hooked me up instantly. I always hoped for confronting multivariate data from astronomy applicable to this charming EDA method. Then, somewhat such eager faded, without realizing what’s happening. Tragically, this was mainly due to my absent mind. Continue reading ‘[MADS] Chernoff face’ »

Tags:

calibration,

Capella,

Chandra,

Chernoff face,

EDA,

line ratios,

MADS,

XAtlas Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Jargon,

Methods,

Misc,

News,

Quotes,

Spectral,

Stars,

X-ray |

2 Comments
Mar 31st, 2009| 03:43 pm | Posted by hlee

Before using any adaptations of chi-square statistic, please spend a minute or two to ponder whether your strategy with chi-square belongs one of these categories.

**1**. Lack of independence among the single events or measures

**2**. Small theoretical frequencies

**3**. Neglect of frequencies of non-occurrence

**4**. Failure to equalize \sum O_i (the sum of the observed frequencies) and \sum M_i (the sum of the theoretical frequencies)

**5**. Indeterminate theoretical frequencies

**6**. Incorrect or questionable categorizing

**7**. Use of non-frequency data

**8**. Incorrect determination of the number of degrees of freedom

**9**. Incorrect computations (including a failure to weight by N when proportions instead of frequencies are used in the calculations)

From “**Chapter 10: On the Use and Misuse of Chi-square**” by K.L.Delucchi in *A Handbook for Data Analysis in the Behavioral Sciences* (1993). Delucchi acknowledged these nine principle sources of error to Lewis and Burke (1949), entitled “The Use and Misuse of the Chi-square” published in *Psychological Bulletin.* Continue reading ‘Use and Misuse of Chi-square’ »

Mar 18th, 2009| 08:30 pm | Posted by hlee

I was disappointed when video, audio, or handout files were not available from the research program “Statistical Theory and Methods for Complex High-Dimensional Data” held at Isaac Newton Institute for Mathematical Sciences during the first half of last year after checking the sites several times. Wow…They are now there~ Continue reading ‘Web Seminar’ »

Mar 11th, 2009| 01:04 pm | Posted by hlee

by **T. Cover and J. Thomas** website: http://www.elementsofinformationtheory.com/

Once, perhaps more, I mentioned this book in my post with the most celebrated paper by Shannon (see the posting). Some additional recommendation of the book has been made to answer offline inquiries. And this book always has been in my favorite book list that I like to use for teaching. So, I’m not shy with recommending this book to astronomers with modern objective perspectives and practicality. Before advancing for more praises, I must say that those admiring words do not imply that I understand every line and problem of the book. Like many fields, Information theory has grown fast since the monumental debut paper by Shannon (1948) like the speed of astronomers observation techniques. Without the contents of this book, most of which came after Shannon (1948), internet, wireless communication, compression, etc could not have been conceived. Since the notion of “**entropy**“, the core of **information theory**, is familiar to astronomers (physicists), the book would be received better among them than statisticians. This book should be read easier to astronomers than statisticians. Continue reading ‘[Book] Elements of Information Theory’ »

Tags:

bandwidth,

book,

Cover,

data mining,

education,

Entropy,

Information theory,

Kolmogorov complexity,

Shannon,

Thomas Category:

Algorithms,

arXiv,

Cross-Cultural,

Data Processing,

Jargon,

Quotes |

Comment
Mar 10th, 2009| 12:41 pm | Posted by vlk

The iPhone App Store has a couple of apps that make life significantly easier for those of us inundated and overwhelmed by the stream of daily arXiv preprints. These are ArXivReader.app and ArXiv.app, both providing a means to browse and search the arXiv preprint database ~~and both selling for 99c~~ with the first selling for 99c and the second free. Check them out! The former even lets you save papers for off-line reading.

For me at least, the hardest part of going through the arXiv emails every day was to pick out the interesting papers in the deluge of text. These apps do the right thing and segregate the categories and highlight the titles. Fitts’ Law in action — suddenly the daily ritual is orders of magnitude more pleasant!

Mar 9th, 2009| 05:18 pm | Posted by hlee

It bears the name of its inventor, Prasanta Chandra Mahalanobis. As opposed to the Euclidean distance, a household name, the name of this distance is rarely used but many pseudonyms exist with variations adapted into broad scientific disciplines and applications. Therefore, under different names, I believe that the Mahalanobis distance is frequently applied in exploring and analyzing astronomical data. Continue reading ‘[MADS] Mahalanobis distance’ »

Mar 6th, 2009| 03:42 pm | Posted by hlee

Ah ha~ Once I questioned, “what is systematic error?” (see [Q] systematic error.) Thanks to L. Lyons’ work discussed in [ArXiv] Particle Physics, I found this paper, titled **Systematic Errors** describing the concept and statistical inference related to **systematic errors** in the field of particle physics. It, gladly, shares lots of similarity with high energy astrophysics. Continue reading ‘systematic errors’ »

Tags:

coverage,

Heinrich,

likelihood,

Lyons,

nuisance parameter,

objective priors,

p-value,

particle physics,

statistical error,

subjective priors,

systematic error Category:

Algorithms,

arXiv,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

Jargon,

Misc,

News,

Physics,

Stat,

Uncertainty |

Comment