Archive for the ‘Methods’ Category.

Aug 25th, 2009| 09:19 pm | Posted by hlee

**Kriging** is the first thing that one learns from a spatial statistics course. If an astronomer sees its definition and application, almost every astronomer will say, “Oh, I know this! It is like the 2pt correlation function!!” At least this was my first impression when I first met **kriging.**

There are three distinctive subjects in spatial statistics: **geostatistics**, **lattice data analysis**, and **spatial point pattern analysis.** Because of the resemblance between the spatial distribution of observations in coordinates and the notion of spatially random points, spatial statistics in astronomy has leaned more toward the spatial point pattern analysis than the other subjects. In other fields from immunology to forestry to geology whose data are associated spatial coordinates of underlying geometric structures or whose data were sampled from lattices, observations depend on these spatial structures and scientists enjoy various applications from geostatistics and lattice data analysis. Particularly, **kriging** is the fundamental notion in **geostatistics** whose application is found many fields. Continue reading ‘[MADS] Kriging’ »

Tags:

BLUP,

book,

books,

CMB,

Cressie,

Diggle,

geostatistics,

hierarchical model,

kriging,

MADS,

point pattern analysis,

sparse,

spatial statistics,

Stein,

WMAP Category:

arXiv,

Astro,

Imaging,

Jargon,

Methods,

Stat |

Comment
Aug 12th, 2009| 06:03 pm | Posted by hlee

Statistical Resampling Methods are rather unfamiliar among astronomers. Bootstrapping can be an exception but I felt like it’s still unrepresented. Seeing an recent review paper on **cross validation** from [arXiv] which describes basic notions in theoretical statistics, I couldn’t resist mentioning it here. **Cross validation** has been used in various statistical fields such as classification, density estimation, model selection, regression, to name a few. Continue reading ‘[ArXiv] Cross Validation’ »

Tags:

ADS,

cross-validation,

machine learning,

Model Selection,

n-fold Category:

arXiv,

Astro,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

Jargon,

Methods,

Quotes,

Stat |

Comment
Jul 29th, 2009| 01:02 am | Posted by hlee

Speaking of XAtlas from my previous post I tried another visualization tool called **Parallel Coordinates** on these Capella observations and two stars with multiple observations (AL Lac and IM Peg). As discussed in [MADS] Chernoff face, full description of the catalog is found from XAtlas website. The reason for choosing these stars is that among low mass stars, next to Capella (I showed 16), IM PEG (HD 21648, 8 times), and AR Lac (although different phases, 6 times) are most frequently observed. I was curious about which variation, within (statistical variation) and between (Capella, IM Peg, AL Lac), is dominant. How would they look like from the parametric space of High Resolution Grating Spectroscopy from Chandra? Continue reading ‘[MADS] Parallel Coordinates’ »

Tags:

Classification,

clustering,

display,

EDA,

eye catcher,

GGobi,

Inselberg,

parallel coordinates,

visualization Category:

Algorithms,

arXiv,

Cross-Cultural,

Data Processing,

High-Energy,

Jargon,

Methods,

Spectral,

X-ray |

3 Comments
Jun 12th, 2009| 03:47 pm | Posted by hlee

**A Fast Thresholded Landweber Algorithm for Wavelet-Regularized Multidimensional Deconvolution**

Vonesch and Unser (2008)

IEEE Trans. Image Proc. vol. 17(4), pp. 539-549

Quoting the authors, I also like to say that __the recovery of the original image from the observed is an __**ill-posed** problem. They traced the efforts of wavelet regularization in deconvolution back to a few relatively recent publications by astronomers. Therefore, I guess the topic and algorithm of this paper could drag some attentions from astronomers. Continue reading ‘Wavelet-regularized image deconvolution’ »

Tags:

bound optimization,

deconvolution,

image processing,

impulse response,

MM algorithm,

PSF,

regularization,

restoration,

thresholding,

wavelet Category:

Algorithms,

arXiv,

Data Processing,

Fitting,

Imaging,

Jargon,

Methods,

Quotes,

Stat |

Comment
Jun 2nd, 2009| 03:03 am | Posted by hlee

Even though I traced the astronomers’ casual usage of the **null hypothesis probability** in a fashion of reporting outputs from data analysis packages of their choice, there were still some curious cases of the **null hypothesis probability** that I couldn’t solve. They are quite mysterious to me. Sometimes too much creativity harms the original intention. Here are some examples. Continue reading ‘Curious Cases of the Null Hypothesis Probability’ »

Tags:

cases,

chi-sq,

curious,

degree of freedom,

dof,

F-test,

goodness-of-fit test,

Model Selection,

null hypothesis probability,

p-value,

reduced chi-sq Category:

arXiv,

Astro,

Cross-Cultural,

Fitting,

Methods,

Uncertainty |

3 Comments
May 18th, 2009| 12:18 pm | Posted by hlee

My understandings of **“robustness”** from the education in statistics and from communicating with astronomers are hard to find a mutual interest. Can anyone help me to build a robust bridge to get over this abyss? Continue reading ‘Robust Statistics’ »

Tags:

break point,

Huber,

nonparametric,

robust,

Rousseeuw,

Tukey Category:

Bayesian,

Frequentist,

Jargon,

MCMC,

Methods,

Quotes,

Stat,

Uncertainty |

Comment
Apr 2nd, 2009| 12:00 pm | Posted by hlee

I cannot remember when I first met **Chernoff face** but it hooked me up instantly. I always hoped for confronting multivariate data from astronomy applicable to this charming EDA method. Then, somewhat such eager faded, without realizing what’s happening. Tragically, this was mainly due to my absent mind. Continue reading ‘[MADS] Chernoff face’ »

Tags:

calibration,

Capella,

Chandra,

Chernoff face,

EDA,

line ratios,

MADS,

XAtlas Category:

Algorithms,

arXiv,

Astro,

Cross-Cultural,

Data Processing,

Jargon,

Methods,

Misc,

News,

Quotes,

Spectral,

Stars,

X-ray |

2 Comments
Mar 17th, 2009| 03:37 pm | Posted by hlee

I couldn’t believe my eyes when I saw 4754 degrees of freedom (d.f.) and chi-square test statistic 4859. I’ve often enough seen large degrees of freedom from journals in astronomy, several hundreds to a few thousands, but I never felt comfortable at these big numbers. Then with a great shock 4754 d.f. appeared. I must find out why I feel so bothered at these huge degrees of freedom. Continue reading ‘4754 d.f.’ »

Tags:

Binning,

chi-square,

chi-square minimization,

chi-square optimization,

chi-square statistic,

class,

degrees-of-freedom,

equiprobable,

goodness-of-fit test,

kernel density estimation Category:

Bad AstroStat,

Fitting,

High-Energy,

Methods,

Spectral,

X-ray |

2 Comments
Feb 20th, 2009| 07:48 pm | Posted by hlee

[stat.AP:0811.1663]

*Open Statistical Issues in Particle Physics* by **Louis Lyons**

My recollection of meeting Prof. L. Lyons was that he is very kind and listening. I was delighted to see his introductory article about particle physics and its statistical challenges from an [arxiv:stat] email subscription. Continue reading ‘[ArXiv] Particle Physics’ »

Tags:

chi-square,

chi-square minimization,

coverage,

hypothesis testing,

L.Lyons,

LHC,

LRT,

particle physics,

posterior distribution Category:

arXiv,

Bayesian,

Cross-Cultural,

Data Processing,

Frequentist,

High-Energy,

Methods,

Physics,

Stat |

Comment
Feb 9th, 2009| 03:16 pm | Posted by hlee

There were (only) four articles from ADS whose abstracts contain the word **semiparametric** (none in titles). Therefore, **semiparametric** is not exactly [MADS] but almost [MADS]. One would like to say it is virtually [MADS] or quasi [MADS]. By introducing the term and providing rare examples in astronomy, I hope this scarce term **semiparametric** to be used adequately against its misguidance of astronomers to inappropriate usage for statistical inference with their data. Continue reading ‘[MADS] Semiparametric’ »

Jan 15th, 2009| 06:01 pm | Posted by hlee

I wonder what Fisher, Neyman, and Pearson would say if they see “Technique” after “Likelihood Ratio” instead of “Test.” A presenter’s saying “Likelihood Ratio Technique” for source identification, I couldn’t resist checking it out not to offend founding fathers of the likelihood principle in statistics since “Technique” sounded derogatory to be attached with “Likelihood” to my ears. I thank, above all, the speaker who kindly gave me the reference about this likelihood ratio technique. Continue reading ‘Likelihood Ratio Technique’ »

Tags:

Fisher,

likelihood principle,

likelihood ratio technique,

likelihood ratio test,

Neyman,

Pearson Category:

Algorithms,

arXiv,

Astro,

Bayesian,

Cross-Cultural,

Data Processing,

Fitting,

Frequentist,

Jargon,

Methods,

Objects,

Stat,

Uncertainty |

Comment
Nov 19th, 2008| 01:34 am | Posted by hlee

There will be a special session at the 213th AAS meeting on **meaning from surveys and population studies (SPS)**. Until then, it might be useful to pull out some interesting and relevant papers and questions/challenges as a preliminary to the meeting. I will not list astronomical catalogs and surveys only, which are literally countless these days but will bring out some if they change the way how science is performed with a description of the catalog (the best example would be SDSS, Sloan Digital Sky Survey, to my knowledge). Continue reading ‘[SPS] Testing Completeness’ »

Tags:

completeness,

incompleteness,

normal,

SPS,

test Category:

arXiv,

Astro,

Cross-Cultural,

Frequentist,

Methods,

News,

Quotes,

Stat,

Uncertainty |

Comment
Nov 17th, 2008| 01:39 pm | Posted by hlee

The full description is given http://cxc.harvard.edu/ciao3.4/ahelp/bayes.html about “bayes” under sherpa/ciao^{[1]}. Some sentences kept bothering me and here’s my account for the reason given outside of quotes. Continue reading ‘It bothers me.’ »

Tags:

bayes,

ciao,

ML,

Sherpa Category:

Algorithms,

Astro,

Cross-Cultural,

Data Processing,

Fitting,

High-Energy,

Jargon,

Languages,

Methods,

Spectral,

Uncertainty,

X-ray |

4 Comments
Nov 6th, 2008| 11:22 pm | Posted by hlee

Personally, it was a highly anticipated symposium at CfA because I was fascinated about the female computers’ (or astronomers’) contributions that occurred here about a century ago even though at that time women were not considered as scientists but mere assistants for tedious jobs. Continue reading ‘after “Thanks to Henrietta Leavitt”’ »

Oct 9th, 2008| 04:28 pm | Posted by hlee

Without signal processing courses, the following equation should be awfully familiar to astronomers of photometry and handling data:

$$c_k=\int_\Lambda l(\lambda) r(\lambda) f_k(\lambda) \alpha(\lambda) d\lambda +n_k$$

Terms are in order, camera response (c_k), light source (l), spectral radiance by l (r), filter (f), sensitivity (α), and noise (n_k), where Λ indicates the range of the spectrum in which the camera is sensitive.

Or simplified to $$c_k=\int_\Lambda \phi_k (\lambda) r(\lambda) d\lambda +n_k$$

where φ denotes the combined illuminant and the spectral sensitivity of the k-th channel, which goes by augmented spectral sensitivity. Well, we can skip spectral radiance r, though. Unfortunately, the sensitivity α has multiple layers, not a simple closed function of λ in astronomical photometry.

Or $$c_k=\Theta r +n$$

Inverting Θ and finding a reconstruction operator such that r=inv(Θ)c_k leads spectral reconstruction although Θ is, in general, not a square matrix. Otherwise, approach from indirect reconstruction. Continue reading ‘[tutorial] multispectral imaging, a case study’ »

Tags:

matrix,

Mona Lisa,

multispectral,

noise,

signal processing,

signal processing magazine,

Tutorial Category:

Algorithms,

arXiv,

Cross-Cultural,

Data Processing,

Fitting,

Imaging,

Methods,

Quotes,

Spectral,

Stat,

Uncertainty |

2 Comments