The AstroStat Slog » Timing Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 09 Sep 2011 17:05:33 +0000 en-US hourly 1 [AAS-HEAD 2011] Time Series in High Energy Astrophysics Fri, 09 Sep 2011 17:05:33 +0000 vlk We organized a Special Session on Time Series in High Energy Astrophysics: Techniques Applicable to Multi-Dimensional Analysis on Sep 7, 2011, at the AAS-HEAD conference at Newport, RI. The talks presented at the session are archived at

A tremendous amount of information is contained within the temporal variations of various measurable quantities, such as the energy distributions of the incident photons, the overall intensity of the source, and the spatial coherence of the variations. While the detection and interpretation of periodic variations is well studied, the same cannot be said for non-periodic behavior in a multi-dimensional domain. Methods to deal with such problems are still primitive, and any attempts at sophisticated analyses are carried out on a case-by-case basis. Some of the issues we seek to focus on are methods to deal with are:
* Stochastic variability
* Chaotic Quasi-periodic variability
* Irregular data gaps/unevenly sampled data
* Multi-dimensional analysis
* Transient classification

Our goal is to present some basic questions that require sophisticated temporal analysis in order for progress to be made. We plan to bring together astronomers and statisticians who are working in many different subfields so that an exchange of ideas can occur to motivate the development of sophisticated and generally applicable algorithms to astronomical time series data. We will review the problems and issues with current methodology from an algorithmic and statistical perspective and then look for improvements or for new methods and techniques.

]]> 0
CAS 2010 Sat, 14 Aug 2010 18:50:05 +0000 chasc The schedule for the mini-Workshop on Computational AstroStatistics is set:

]]> 0
[MADS] ARCH Fri, 04 Sep 2009 18:30:27 +0000 hlee ARCH (autoregressive conditional heteroscedasticity) is a statistical model that considers the variance of the current error term to be a function of the variances of the previous time periods’ error terms. I heard that this model made Prof. Engle a Nobel prize recipient.

Heteroscedesticity in regression problems in astronomy has been discussed with various kind of observations since astronomers are interested in correlations and causality between variables in the data set. Perhaps, the heteroscedasticity in volatilty of finance does not have any commonality with astronomical times series data, which could be the primary reason this celebrated model does not appear in ADS. However, there are various times series models branched from ARCH that consider heteroscedasticity in errors and I hope some can be useful to astronomers for analyzing times series data with inhomogeneous errors.

[Note] All links lead to

]]> 0
Bipartisanship Wed, 10 Dec 2008 17:41:58 +0000 hlee We have seen the word “bipartisan” often during the election and during the on-going recession period. Sometimes, I think that the bipartisanship is not driven by politicians but it’s driven by media, commentator, and interpreters.

me: Why Bayesian methods?
astronomers: Because Bayesian is robust. Because frequentist method is not robust.

By intention, I made the conversation short. Obviously, I didn’t ask all astronomers the same question and therefore, this conversation does not reflect the opinion of all astronomers. Nevertheless, this summarizes what I felt at CfA.

I was educated in frequentist school which I didn’t realize before I come to CfA. Although I didn’t take their courses, there were a few Bayesian professors (I took two but it’s nothing to do with this bipartisanship. Contents were just foundations of statistics). However, I found that getting ideas and learning brilliant algorithms by Bayesians were equally joyful as learning mature statistical theories from frequentists.

How come astronomers possess the idea that Bayesian statistics is robust and frequentist is not? Do they think that the celebrated Gaussian distribution and almighty chi-square methods compose the whole frequentist world? (Please, note that F-test, LRT, K-S test, PCA take little fraction of astronomers’ statistics other than chi-square methods according to astronomical publications, let alone Bayesian methods but no statistics can compete with chi-square methods in astronomy.) Is this why they think frequentist methods are not robust? The longer history is, the more flaws one finds so that no one expect chi-square stuffs are universal panacea. Applying the chi-square looks like growing numbers of epicycles. From the history, finding shortcomings makes us move forward, evolve, invent, change paradigms, etc., instead of saying that chi-square (frequentist) methods are not robust. I don’t think we spent time to learn chi-square stuffs from class. There are too many robust statistics that frequentists have developed. Text books have “robust statistics” in their titles are most likely written by frequentists. Did astronomers check text books and journals before saying frequentists methods are not robust? I’m curious how this bipartisanship, especially that one party is favored and the other is despised but blindly utilized in data analysis, has developed (Probably I should feel relieved about no statistics dictatorship in the astronomical society and exuberant about the efforts of balancing between two parties from a small number of scientists).

Although I think more likely in a frequentist way, I don’t object Bayesian. It’s nothing different from learning mother tongues and cultures. Often times I feel excited how Bayesian get over some troubles that frequentists couldn’t.. If I exaggerate, finding what frequentists achieved but Bayesians haven’t yet or the other way around is similar to the event that by changing the paradigm from the geocentric universe to the heliocentric one could explain the motions of planets with simplicity instead of adding more numbers of epicycles and complicating the description of motions. I equally cherish results from both statistical cultures. Satisfying the simplicity and the fundamental laws including probability theories, is the most important in pursuing proper applications of statistics, not the bipartisanship.

My next post will be about “Robust Statistics” to rectify the notion of robustness that I acquired from CfA. I’d like to hear your, astronomer and statistician alike, thoughts on robustness associated with your statistical culture of choice. I only can write about robustness based what I read and was taught. This also can be biased. Perhaps, other statisticians advocate the astronomer’s notion that Bayesian is robust and frequentist is not. Not much communications with statisticians makes me difficult to obtain the general consensus. Equally likely, I don’t know every astronomer’s thoughts on robustness. Nonetheless, I felt the notion of robustness is different between statisticians and astronomers and this could generate some discussions.

I may sound like Joe Liberman, overall. But remember that tossing him one party to the other back and forth was done by media explicitly. People can be opinionated but I’m sure he pursued his best interests regardless of parties.

]]> 2
“Thanks to Henrietta Leavitt” Thu, 06 Nov 2008 10:00:17 +0000 vlk [9/30/2008]

The CfA is celebrating the 100th anniversary of the discovery of the Cepheid period-luminosity relation on Nov 6, 2008. See for details.

[Update 10/03] For a nice introduction to the story of Henrietta Swan Leavitt, listen to this Perimeter Institute talk by George Johnson:

[Update 11/06] The full program is now available. The symposium begins at Noon today.

]]> 1
~ Avalanche(a,b) Sun, 14 Oct 2007 01:14:21 +0000 vlk arXiv:0710.1021), by Melatos, Peralta, & Wyithe]]> Avalanches are a common process, occuring anywhere that a system can store stress temporarily without “snapping”. It can happen on sand dunes and solar flares as easily as on the snow bound Alps.

Melatos, Peralta, & Wyithe (arXiv:0710.1021) have a nice summary of avalanche processes in the context of pulsar glitches. Their primary purpose is to show that the glitches are indeed consistent with an avalanche, and along the way they give a highly readable description of what an avalanche is and what it entails. Briefly, avalanches result in event parameters that are distributed in scale invariant fashion (read: power laws) with exponential waiting time distributions (i.e., Poisson).

Hence the title of this post: the “Avalanche distribution” (indulge me! I’m using stats notation to bury complications!) can be thought to have two parameters, both describing the indices of power-law distributions that control the event sizes, a, and the event durations, b, and where the event separations are distributed as an exponential decay. Is there a canned statistical distribution that describes all this already? (In our work modeling stellar flares, we assumed that b=0 and found that a>2 a<-2, which has all sorts of nice consequences for coronal heating processes.)

]]> 0
Betraying your heritage Thu, 20 Sep 2007 16:26:07 +0000 vlk [arXiv:0709.3093v1] Short Timescale Coronal Variability in Capella (Kashyap & Posson-Brown)

We recently submitted that paper to AJ, and rather ironically, I did the analysis during the same time frame as this discussion was going on, about how astronomers cannot rely on repeating observations. Ironic because the result reported there hinges on the existence of small, but persistent signal that is found in repeated observations of the same source. Doubly ironic in fact, in that just as we were backing and forthing about cultural differences I seemed to have gone and done something completely contrary to my heritage!

btw, this paper is interesting because Capella is a strong X-ray source, and “everybody believes” that such sources should exhibit some variability, so finding such shouldn’t be a big deal, and yet Capella itself has been remarkably stable and had all this while defied the characterization and even the detection of such variability. Even now, the estimated magnitude of the variability fraction is rather small. It’s a good thing that we had some 22 counts/sec over 205 kiloseconds to play with.

]]> 0
Change Point Problem Wed, 08 Aug 2007 21:14:49 +0000 hlee X-ray summer school is on going. Numerous interesting topics were presented but not much about statistics (Only advice so far, “use implemented statistics in x-ray data reduction/analysis tools” and “it’s just a tool”). Nevertheless, I happened to talk two students extensively on their research topics, finding features from light curves. One was very empirical from comparing gamma ray burst trigger time to 24kHz observations and the other was statistical and algorithmic by using Bayesian Block. Sadly, I could not give them answers but the latter one dragged my attention.

Recently I went to JSM 2007 and tried to attend talks about (bayesian) change point problems, which frequently appears in time series models, often found in economics. With ARCH (autoregressive conditional heteroskedecity) or GARCH (generalized ARCH) and by adding a parameter indicates a change point, I thought bayesian modeling could handle astronomical light curves.

Developing algorithms based on statistical theories, writing algorithms down in a heuristics way, making the code public, and finding/processing proper datum examples from huge astronomical data archives should come simultaneously, and this multiple steps make proposing new statistics to astronomical society difficult. I’m glad to know that there are individuals who are devoting themselves to make these steps happened. Unfortunately, they are loners.

]]> 4
[ArXiv] Complete Catalog of GRBs from BeppoSAX, July 13, 2007 Mon, 16 Jul 2007 16:15:52 +0000 hlee From arxiv/astro-ph:0707.1900v1
The complete catalogue of gamma-ray bursts observed by the Wide Field Cameras on board BeppoSAX by Vetere,

This paper intend to publicize the largest data set of Gamma Ray Burst (GRB) X-ray afterglows (right curves after the event), which is available from It is claimed to be a complete on-line catalog of GRB observed by two wide-Field Cameras on board BeppoSAX (Click for its Wiki) in the period of 1996-2002. It is comprised with 77 bursts and 56 GRBs with Xray light curves, covering the energy range 40-700keV. A brief introduction to the instrument, data reduction, and catalog description is given.

]]> 1
Quote of the Week, July 12, 2007 Thu, 12 Jul 2007 19:37:21 +0000 aconnors Ingrid Daubechies on the impact of her 1980's wavelets:
"... a way of seeing it. A way of paying attention." CLICK FOR MORE
Ingrid Daubechies, color gif from her websiteThis is from the very interesting Ingrid Daubechies interview by Dorian Devins,
, National Academy of Sciences, U.S.A., 2004. It is from part 6, where Ingrid Daubechies speaks of her early mathematics paper on wavelets. She tries to put the impact into context:

I really explained in the paper where things came from. Because, well, the mathematicians wouldn’t have known. I mean, to them this would have been a question that really came out of nowhere. So, I had to explain it …

I was very happy with [the paper]; I had no inkling that it would take off like that… [Of course] the wavelets themselves are used. I mean, more than even that. I explained in the paper how I came to that. I explained both [a] mathematicians way of looking at it and then to some extent the applications way of looking at it. And I think engineers who read that had been emphasizing a lot the use of Fourier transforms. And I had been looking at the spatial domain. It generated a different way of considering this type of construction. I think, that was the major impact. Because then other constructions were made as well. But I looked at it differently. A change of paradigm. Well, paradigm, I never know what that means. A change of … a way of seeing it. A way of paying attention.

]]> 0