Oct 27th, 2008| 09:24 am | Posted by hlee

The notions of **missing data** are overall different between two communities. I tend to think missing data carry as good amount of information as observed data. Astronomers…I’m not sure how they think but my impression so far is that a missing value in one attribute/variable from a object/observation/informant, all other attributes related to that object become useless because that object is not considered in scientific data analysis or model evaluation process. For example, it is hard to find any discussion about **imputation** in astronomical publication or statistical justification of missing data with respect to inference strategies. On the contrary, they talk about **incompleteness** within different variables. Putting this vague argument with a concrete example, consider a catalog of multiple magnitudes. To draw a color magnitude diagram, one needs both color and magnitude. If one attribute is missing, that star will not appear in the color magnitude diagram and any inference methods from that diagram will not include that star. Nonetheless, one will trying to understand how different proportions of stars are observed according to different colors and magnitudes. Continue reading ‘missing data’ »

Tags:

bootstrap,

catalog,

Efron,

estimator,

ignorable,

imputation,

incompleteness,

Little,

MAR,

MCAR,

missing data,

nonparametric,

Rubin,

Schafer,

survey Category:

Astro,

Cross-Cultural,

Data Processing,

Stat |

2 Comments
Sep 10th, 2008| 10:46 pm | Posted by hlee

The following footnotes are from one of Prof. Babu’s slides but I do not recall which occasion he presented the content.

– In the XSPEC packages, the **parametric bootstrap** is command FAKEIT, which makes Monte Carlo simulation of specified spectral model.

– XSPEC does not provide a **nonparametric bootstrap** capability.

Continue reading ‘Parametric Bootstrap vs. Nonparametric Bootstrap’ »

May 31st, 2008| 11:59 pm | Posted by hlee

Eight astro-ph papers and two statistics paper are listed this week. One statistics paper discusses detecting filaments and the other talks about maximum likelihood estimation of satellite images (clouds). Continue reading ‘[ArXiv] 4th week, May 2008’ »

Tags:

AGN,

Bayes factor,

bootstrap,

confidence set,

cosmological constanct,

dark energy,

Exofit,

exoplanet,

filament,

jackknife,

KDE,

Model Selection,

time series,

Type Ia SNe,

unbiased,

wavelet Category:

arXiv,

Bayesian,

MCMC,

Stat |

Comment
May 19th, 2008| 10:42 am | Posted by hlee

There’s no particular opening remark this week. Only I have profound curiosity about jackknife tests in [astro-ph:0805.1994]. Including this paper, a few deserve separate discussions from a statistical point of view that shall be posted. Continue reading ‘[ArXiv] 2nd week, May 2008’ »

Tags:

bimodality,

bootstrap,

calibration uncertainty,

CF,

Classification,

CMB,

dip,

exoplanet,

Fisher matrix,

flare,

GL,

jackknife,

KS test,

marked point,

maximum likelihood,

MLE,

poisson point process,

spatial data,

XLF Category:

arXiv,

Frequentist,

Uncertainty,

X-ray |

Comment
Apr 20th, 2008| 09:05 pm | Posted by hlee

The dichotomy of outliers; detecting outliers to be discarded or to be investigated; statistics that is robust enough not to be influenced by outliers or sensitive enough to alert the anomaly in the data distribution. Although not related, one paper about outliers made me to dwell on what outliers are. This week topics are diverse. Continue reading ‘[ArXiv] 3rd week, Apr. 2008’ »

Tags:

background,

bootstrap,

calibration errors,

Cash statistics,

clusters,

CMB,

corona,

edge detection,

FFT,

gravitational lens,

maximum likelihood,

multiscale,

neural network,

outlier,

SDSS,

sunspot,

systematic errors,

topology,

WMAP,

XMM-Newton Category:

arXiv,

High-Energy,

MCMC |

Comment
Jan 30th, 2008| 02:33 am | Posted by hlee

Astronomers have developed their ways of processing signals almost independent to but sometimes collaboratively with engineers, although the fundamental of signal processing is same: extracting information. Doubtlessly, these two parallel roads of astronomers’ and engineers’ have been pointing opposite directions: one toward the sky and the other to the earth. Nevertheless, without an intensive argument, we could say that somewhat statistics has played the medium of signal processing for both scientists and engineers. This particular issue of IEEE signal processing magazine may shed lights for astronomers interested in signal processing and statistics outside the astronomical society.

IEEE Signal Processing Magazine Jul. 2007 Vol 24 Issue 4: Bootstrap methods in signal processing

This link will show the table of contents and provide links to articles; however, the access to papers requires IEEE Xplore subscription via libraries or individual IEEE memberships). Here, I’d like to attempt to introduce some articles and tutorials.

Continue reading ‘Signal Processing and Bootstrap’ »

Tags:

bootstrap,

compressive sensing,

confidence interval,

GLM,

IEEE,

jacknife,

machine learning,

multitaper estimate,

particle filter,

signal processing,

statistical inference,

Tutorial,

wavelet Category:

Algorithms,

arXiv,

Bayesian,

Cross-Cultural,

Fitting,

Frequentist,

MC,

MCMC,

Methods,

Misc,

Spectral,

Stat,

Uncertainty |

Comment
Dec 31st, 2007| 08:48 pm | Posted by hlee

** The Bootstrap and Modern Statistics ** Brad Efron (2000), JASA Vol. 95 (452), p. 1293-1296.

If the bootstrap is an automatic processor for frequentist inference, then MCMC is its Bayesian counterpart.

Continue reading ‘[Quote] Bootstrap and MCMC’ »

Nov 2nd, 2007| 05:59 pm | Posted by hlee

To be exact, the title of this posting should contain *5th week, Oct*, which seems to be the week of EGRET. In addition to astro-ph papers, although they are not directly related to astrostatistics, I include a few statistics papers which may be profitable for astronomical data analysis. Continue reading ‘[ArXiv] 1st week, Nov. 2007’ »

Tags:

bootstrap,

EGRET,

Fisher information,

Laplace transform,

maximum likelihood,

PCA,

PDF,

Poisson,

Ratio,

Uncertainty,

variance Category:

arXiv |

1 Comment
Aug 14th, 2007| 10:17 pm | Posted by hlee

During the International X-ray Summer School, as a project presentation, I tried to explain the inadequate practice of χ^2 statistics in astronomy. *If your best fit is biased (any misidentification of a model easily causes such bias), do not use χ^2 statistics to get 1σ error for the 68% chance of capturing the true parameter.*

Later, I decided to do further investigation on that subject and this paper came along: Astrostatistics: Goodness-of-Fit and All That! by Babu and Feigelson.

Continue reading ‘Astrostatistics: Goodness-of-Fit and All That!’ »

Tags:

Anderson-Darling,

Babu,

best-fit,

bias,

bootstrap,

chi-square,

Cramer-von Mises,

Feigelson,

Kolmogorov-Smirnoff,

Kullback-Leibler distance,

nonparametric,

parametric,

resampling Category:

Algorithms,

arXiv,

Astro,

Fitting,

High-Energy,

Methods,

Spectral,

Stat |

7 Comments