Comments on: survey and design of experiments http://hea-www.harvard.edu/AstroStat/slog/2008/survey-and-design-of-experiments/ Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 01 Jun 2012 18:47:52 +0000 hourly 1 http://wordpress.org/?v=3.4 By: vlk http://hea-www.harvard.edu/AstroStat/slog/2008/survey-and-design-of-experiments/comment-page-1/#comment-781 vlk Fri, 03 Oct 2008 18:46:40 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=894#comment-781 You are right, astronomy surveys are probably more concerned with measurement than analysis. The analysis, other than the primary goal for which the survey would have been set up, is usually left as an exercise for grad students downstream. There are also "accidental surveys", which are not planned at all beforehand, but simply make use of a large body of existing observations to serendipitously catalog what is out there. Some examples are the Einstein Slew Survey, the Chandra Multiwavelength Project, X-Atlas, etc. You are right, astronomy surveys are probably more concerned with measurement than analysis. The analysis, other than the primary goal for which the survey would have been set up, is usually left as an exercise for grad students downstream.

There are also “accidental surveys”, which are not planned at all beforehand, but simply make use of a large body of existing observations to serendipitously catalog what is out there. Some examples are the Einstein Slew Survey, the Chandra Multiwavelength Project, X-Atlas, etc.

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/survey-and-design-of-experiments/comment-page-1/#comment-777 hlee Thu, 02 Oct 2008 18:25:54 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=894#comment-777 I was trying to expose a subtle difference purely based on my listening a few talks about LSST. They emphasize PB/day (?) data collection. Somewhat it'll be processed to the level computer memory can handle but no one mentioned how this massive data will be handled. I never got how much degree of reduction through source detection or compression. Furthermore, since data are collected through years on daily bases, some sort of streaming data analysis schemes must be laid out but no one mentioned it. I'm not saying there is missing in Astro survey but there is a difference in surveys. Statisticians are more interested in how to analyze data and survey reflect this objective and astronomers are interested in what to extract for science. How to comes after collecting data but generally use chi sq after data reduction (throwing outliers - even after one throws data points away, still one has thousands of data points and chi square does work). I was trying to expose a subtle difference purely based on my listening a few talks about LSST. They emphasize PB/day (?) data collection. Somewhat it’ll be processed to the level computer memory can handle but no one mentioned how this massive data will be handled. I never got how much degree of reduction through source detection or compression. Furthermore, since data are collected through years on daily bases, some sort of streaming data analysis schemes must be laid out but no one mentioned it. I’m not saying there is missing in Astro survey but there is a difference in surveys. Statisticians are more interested in how to analyze data and survey reflect this objective and astronomers are interested in what to extract for science. How to comes after collecting data but generally use chi sq after data reduction (throwing outliers – even after one throws data points away, still one has thousands of data points and chi square does work).

]]>
By: vlk http://hea-www.harvard.edu/AstroStat/slog/2008/survey-and-design-of-experiments/comment-page-1/#comment-776 vlk Thu, 02 Oct 2008 00:22:47 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=894#comment-776 I would say it is unfair to compare a census with an astro survey. To begin with, a census has very limited objectives, and it is easy to design the questions to gather exactly the kind of data that are needed. And, they have been running it every decade for two centuries, and have had plenty of time to perfect their procedure. Not to mention, the census takers do not have to worry about the Malmquist bias. Even so, consider a typical astro survey nowadays, say the SDSS -- you get number counts of stars, fluxes in five passbands, measurement errors on each of them, and a host of well-defined secondary characteristics such as source position, extent, variability, data quality, etc. So, let me pose this question instead -- what is missing? What should be done in an Astro survey that is not being done now? I would say it is unfair to compare a census with an astro survey. To begin with, a census has very limited objectives, and it is easy to design the questions to gather exactly the kind of data that are needed. And, they have been running it every decade for two centuries, and have had plenty of time to perfect their procedure. Not to mention, the census takers do not have to worry about the Malmquist bias. Even so, consider a typical astro survey nowadays, say the SDSS — you get number counts of stars, fluxes in five passbands, measurement errors on each of them, and a host of well-defined secondary characteristics such as source position, extent, variability, data quality, etc.

So, let me pose this question instead — what is missing? What should be done in an Astro survey that is not being done now?

]]>