The AstroStat Slog » signal processing

Another exciting news (with no use)

hlee — Mon, 27 Jul 2009 12:37:45 +0000

I wish I could chase all rabbits. Another rabbit I missed came to a realization by a friend, who was sure that I already knew this “call for papers” notice for the special issue of the signal processing magazine (SPM). Although those due dates were mistaken (the white paper due was several months back), my friend thought it would be useful for me and my group just in case I didn’t know about it. Yes, I was very delighted such things were on going. No doubt that I was disappointed when the white paper due was long gone.

It was my fault that I didn’t chase this rabbit for a while. The slog has some SPM article review posts, as you recall. I mentioned a few times that statistics is about information retrieval (uncertainty included but not limited to). Because of this belief, I took courses in signal processing and information theory. I believed that research topics in signal processing is very strongly associated with astronomical data analysis. Nonetheless, fellow EE students saw me like an alien like students in computational physics class. Upon seeing this belated news, I became very delighted because it transfigured me from an alien to a human being.

According to the notice, the topics to be considered cover quite broad areas in astronomy, from which I was afraid that this SPM issue will be unprecedentedly thick. I can say so because the projects that I’m working on cover almost every item listed in the following from the notice.
Topics to be considered are:

Calibration (e.g. of large phased arrays in the presence of electronic and atmospheric disturbances)
Deconvolution, imaging and data analysis
Interference mitigation
Image restoration and reconstruction
Source separation; inverse problems
Data mining and machine learning techniques
Classification and feature identification
Bayesian techniques

For areas related to:

Radio telescopes, e.g. large arrays and focal plane arrays
Gamma-ray radio astronomy
Cosmological data, Cosmic Microwave Background (CMB) data
Optical and IR astronomy; adaptive optics in large telescopes
Digital image restoration in optical astronomy (including blind, non-blind, single frame, image sequence,and speckle methods)
Analysis of large astronomical databases
Stellar imaging and spectroscopy

I’d like to express my sincere gratitude for my friend’s very thoughtful gesture. If I knew it earlier, our group could have written many articles. I’m very curious who would have contributed to this January 2010 issue of SPM. I’m afraid that editors have been greatly challenged by the great volume of white papers submitted by scientists in the fields. Perhaps not, because the SPM is not well recognized by astronomers. I really want to see who have contributed and what topics have been covered in this SPM issue. I should not miss the second chase.

[MADS] HMM

hlee — Mon, 08 Dec 2008 03:23:11 +0000

MADS stands for “Missing in ADS.” Every astronomer, I believe, knows what ADS is. As we have [EotW] series and used to have [ArXiv] series, creating a new series for semi-periodic postings under the well known name ADS seems interesting.

I’m not sure about these days, but when I was studying astronomy a decade ago, ADS was Google in astronomy. Once switching to statistics, I was shocked at the fact that there was no composite search engine for statistical literature and databases. I showed ADS to fellow statistics students how good this is at that time and compared ADS with what are available in statistics: JSTOR only had 5 year and older materials. Citeseer was not born nor Project Euclid. Google scholar was not thinkable at all. I used to dig the library cd-roms to satisfy my craving for more information. Now those days are over thanks to Google and other scientific search engines. Yet, astronomers prefer ADS than any other database and search engines because of its comprehensiveness.

Let’s stop praising ADS here and focus on [MADS]. The key of [MADS] is to introduce something common and popular in other fields that does not appear in ADS. Believe it or not, sometimes I encounter missing elements, most likely jargon of other fields, from this giant and old (mature) data system. For example, HMM is one although more will come in the series. HMM stands for Hidden Markov Model. When you put “Hidden Markov Model” as keywords in your search among referred astronomical journals^[1], you’ll see no result within astronomical publications.

Then, what is Hidden Markov Model? I’d rather defer my answer to wiki:Hidden Markov Model, references therein, and image/signal processing text books (I learned the term from a undergraduate text book about a decade ago. So HMM must be a very common and well received methodology). Since astronomers handle images and signals so often, I thought HMM might be a useful tool for modeling and analyzing astronomical data some years back. Unfortunately, it hasn’t emerged yet.

Finding a MADS does not provide me an eureka moment. It only makes me wish that this MADS appears soon in ADS. One of you soon will be the first person who adopts HMM in your research and will be cited as a pioneer within the astronomy community.

Well, against all this hope, I might be forced to drop this post if someone finds out HMM is already described in published astronomy papers while he/she teaches me how to search ADS better in secret.

Otherwise, ADS search all arxiv papers, which include all computer science, math, statistics, physics, and more

[tutorial] multispectral imaging, a case study

hlee — Thu, 09 Oct 2008 20:28:21 +0000

Without signal processing courses, the following equation should be awfully familiar to astronomers of photometry and handling data:
$$c_k=\int_\Lambda l(\lambda) r(\lambda) f_k(\lambda) \alpha(\lambda) d\lambda +n_k$$
Terms are in order, camera response (c_k), light source (l), spectral radiance by l (r), filter (f), sensitivity (α), and noise (n_k), where Λ indicates the range of the spectrum in which the camera is sensitive.
Or simplified to $$c_k=\int_\Lambda \phi_k (\lambda) r(\lambda) d\lambda +n_k$$
where φ denotes the combined illuminant and the spectral sensitivity of the k-th channel, which goes by augmented spectral sensitivity. Well, we can skip spectral radiance r, though. Unfortunately, the sensitivity α has multiple layers, not a simple closed function of λ in astronomical photometry.
Or $$c_k=\Theta r +n$$
Inverting Θ and finding a reconstruction operator such that r=inv(Θ)c_k leads spectral reconstruction although Θ is, in general, not a square matrix. Otherwise, approach from indirect reconstruction.

Studying that Smile (subscription needed)
A tutorial on multispectral imaging of paintings using the Mona Lisa as a case study
by Ribes, Pillay, Schmitt, and Lahanier
IEEE Sig. Proc. Mag. Jul. 2008, pp.14-26
Conclusions: In this article, we have presented a tutorial description of the multispectral acquisition of images from a signal processing point of view.

From the section Camera Sensitivity: “From a signal processing point of view, the filters of a multispectral camera can be conceived as sampling functions, the other elements of φ being understood as a perturbation”.
From the section Understanding Noise Sources :”The noise is present in the spectral, temporal, and spatial dimensions of the image signal”. … (check out the equation and the individual term explanation) … “the quantization operator represent the analog-to-digital (A/D) conversion performed before stocking the signal in digital form. This conversion introduces the so-called quantization error, a theoretically predictable noise”. (This quantization error is well understood in astronomical photometry.)
Understanding the sampling function φ is common for imaging and photometry but strategies and modeling (including uncertainties by error models) are different. Figures 3, 7, 8 tell a lot about usefulness and connectivity of engineers’ spectral imaging and astronomers’ calibration.
Hessian matrix in regression suffers similar challenges corresponding to issues related to Θ which means spectral imaging can be converted into statistical problems and likewise astronomical photometry can be put into the shape of statistical research.
Discussion of Noise is personally most worthwhile.

I wonder if there’s literature in astronomy matching this tutorial from which we may expand and improve current astronomical photometry processes by adopting strategies developed by more populated signal/image processing engineers and statisticians. (Considering good textbooks on statistical signal processing, and many fundamental algorithms born thanks to them, I must include statisticians. Although not discussed in this tutorial, Hidden Markov Model (HMM) is often used in signal processing but from ADS, with such keywords, no astronomical publication is aware of HMM – please, confirm my finding that HMM is not used among astronomers because my search scheme is likely imperfect.)

Why Gaussianity?

hlee — Wed, 10 Sep 2008 14:15:03 +0000

Physicists believe that the Gaussian law has been proved in mathematics while mathematicians think that it was experimentally established in physics — Henri Poincare

Couldn’t help writing the quote from this article (subscription required).^[1]

Why Gaussianity? by Kim, K. and Shevlyakov, G. (2008) IEEE Signal Processing Magazine, Vol. 25(2), pp. 102-113

It’s been a while since my post, signal processing and bootstrap from IEEE signal processing magazine, described as tutorial style papers on signal processing research and applications. Because of its tutorial style, the magazine delivers most up to date information and applications to people in various disciplines (their citation rate is quite high among scientific fields where data are collected via digitization except astronomy. This statement is solely based on my experience and no proper test was carried out to test this hypothesis). This provoking title, perhaps, will drag attentions about advances in signal processing from astronomers in future.

A historical account on Gaussian distribution, which goes by normal distribution among statisticians is given: de Moivre, before Laplace, found the distribution; Laplace, before Gauss, derived the properties of this distribution. The paper illustrates the derivations by Gauss, Herschel (yes, astronomer), Maxwell (no need to mention his important contribution), and Landon along with these following properties:

the convolution of two Gaussian functions is another Gaussian function
the Fourier transform of a Gaussian function is another Gaussian function
the CLT
maximizing entropy
minimizing Fisher information

You will find pros and cons about Gaussianity in the concluding remark.

Wikiquote said it’s misattributed. And I don’t know French. My guess could be wrong in matching quotes based on french translations into english. Please, correct me.

[ArXiv] 2nd week, Mar. 2008

hlee — Fri, 14 Mar 2008 19:44:34 +0000

Warning! The list is long this week but diverse. Some are of CHASC’s obvious interest.

[astro-ph:0803.0997] V. Smolcic et.al.
A new method to separate star forming from AGN galaxies at intermediate redshift: The submillijansky radio population in the VLA-COSMOS survey
[astro-ph:0803.1048] T.A. Carroll and M. Kopf
Zeeman-Tomography of the Solar Photosphere — 3-Dimensional Surface Structures Retrieved from Hinode Observations
[astro-ph:0803.1066] M. Beasley et.al.
A 2dF spectroscopic study of globular clusters in NGC 5128: Probing the formation history of the nearest giant Elliptical
[astro-ph:0803.1098] Z. Lorenzo
A new luminosity function for galaxies as given by the mass-luminosity relationship
[astro-ph:0803.1199] D. Coe et.al.
LensPerfect: Gravitational Lens Massmap Reconstructions Yielding Exact Reproduction of All Multiple Images (could it be related to GREAT08 Challenge?)
[astro-ph:0803.1213] H.Y.Wang et.al.
Reconstructing the cosmic density field with the distribution of dark matter halos
[astro-ph:0803.1420] E. Lantz et.al.
Multi-imaging and Bayesian estimation for photon counting with EMCCD’s
[astro-ph:0803.1491] Wu, Rozo, & Wechsler
The Effect of Halo Assembly Bias on Self Calibration in Galaxy Cluster Surveys
[astro-ph:0803.1616] P. Mukherjee et.al.
Planck priors for dark energy surveys (some CHASCians would like to check!)
[astro-ph:0803.1738] P. Mukherjee and A. R. Liddle
Planck and reionization history: a model selection view
[astro-ph:0803.1814] J. Cardoso et.al.
Component separation with flexible models. Application to the separation of astrophysical emissions
[astro-ph:0803.1851] A. R. Marble et.al.
The Flux Auto- and Cross-Correlation of the Lyman-alpha Forest. I. Spectroscopy of QSO Pairs with Arcminute Separations and Similar Redshifts
[astro-ph:0803.1857] R. Marble et.al.
The Flux Auto- and Cross-Correlation of the Lyman-alpha Forest. II. Modelling Anisotropies with Cosmological Hydrodynamic Simulations

Signal Processing and Bootstrap

hlee — Wed, 30 Jan 2008 06:33:25 +0000

Astronomers have developed their ways of processing signals almost independent to but sometimes collaboratively with engineers, although the fundamental of signal processing is same: extracting information. Doubtlessly, these two parallel roads of astronomers’ and engineers’ have been pointing opposite directions: one toward the sky and the other to the earth. Nevertheless, without an intensive argument, we could say that somewhat statistics has played the medium of signal processing for both scientists and engineers. This particular issue of IEEE signal processing magazine may shed lights for astronomers interested in signal processing and statistics outside the astronomical society.

IEEE Signal Processing Magazine Jul. 2007 Vol 24 Issue 4: Bootstrap methods in signal processing

This link will show the table of contents and provide links to articles; however, the access to papers requires IEEE Xplore subscription via libraries or individual IEEE memberships). Here, I’d like to attempt to introduce some articles and tutorials.

Special topic on bootstrap:
The guest editors (A.M. Zoubir & D.R. Iskander)^[1] open the issue by providing the rationale, the occasional invalid Gaussian noise assumption, and the consequential complex modeling in their editorial opening, Bootstrap Methods in Signal Processing. A practical approach has been Monte Carlo simulations but the cost of repeating experiments is problematic. The suggested alternative is the bootstrap, which provides tools for designing detectors for various signals subject to noise or interference from unknown distributions. It is said that the bootstrap is a computer-intensive tool for answering inferential questions and this issue serves as tutorials that introduce this computationally intensive statistical method to the signal processing community.

The first tutorial is written by those two guest editors: Bootstrap Methods and Applications, which begins with the list of bootstrap methods and emphasizes its resilience. It discusses the number of bootstrap samples to compensate a simulation (Monte Carlo) error to a statistical error and the sampling methods for dependent data with real examples. The flowchart from Fig. 9 provides the guideline for how to use the bootstrap methods as a summary.

The title of the second tutorial is Jackknifing Multitaper Spectrum Estimates (D.J. Thomson), which introduces the jackknife, multitaper estimates of spectra, and applying the former to the latter with real data sets. The author added the reason for his preference of jackknife to bootstrap and discussed the underline assumptions on resampling methods.

Instead of listing all articles from the special issue, a few astrostatistically notable articles are chosen:

Bootstrap-Inspired Techniques in Computational Intelligence (R. Polikar) explains the bootstrap for estimating errors, algorithms of bagging, boosting, and AdaBoost, and other bootstrap inspired techniques in ensemble systems with a discussion of missing.
Bootstrap for Empirical Multifractal Analysis (H. Wendt, P. Abry & S. Jaffard) explains block bootstrap methods for dependent data, bootstrap confidence limits, bootstrap hypothesis testing in addition to multifractal analysis. Due to the personal lack of familiarity in wavelet leaders, instead of paraphrasing, the article’s conclusion is intentionally replaced with quoting sentences:

First, besides being mathematically well-grounded with respect to multifractal analysis, wavelet leaders exhibit significantly enhanced statistical performance compared to wavelet coefficients. … Second, bootstrap procedures provide practitioners with satisfactory confidence limits and hypothesis test p-values for multifractal parameters. Third, the computationally cheap percentile method achieves already excellent performance for both confidence limits and tests.
Wild Bootstrap Test (J. Franke & S. Halim) discusses the residual-based nonparametric tests and the wild bootstrap for regression models, applicable to signal/image analysis. Their test checks the differences between two irregular signals/images.
Nonparametric Estimates of Biological Transducer Functions (D.H.Foster & K.Zychaluk) I like the part where they discuss generalized linear model (GLM) that is useful to expend the techniques of model fitting/model estimation in astronomy beyond gaussian and least square. They also mentioned that the bootstrap is simpler for getting confidence intervals.
Bootstrap Particle Filtering (J.V.Candy) It is a very pleasant reading for Bayesian signal processing and particle filter. It overviews MCMC and state space model, and explains resampling as a remedy to overcome the shortcomings of importance sampling in signal processing.
Compressive sensing. (R.G.Baranuik)

A lecture note presents a new method to capture and represent compressible signals at a rate significantly below the Nyquist rate. This method employs nonadaptive linear projections that preserve the structure of the signal;

I do wish this brief summary assists you selecting a few interesting articles.

They wrote a book, the bootstrap and its application in signal processing.