Comments on: The GREAT08 Challenge http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/ Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 01 Jun 2012 18:47:52 +0000 hourly 1 http://wordpress.org/?v=3.4 By: Douglas Applegate http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-815 Douglas Applegate Tue, 28 Oct 2008 15:47:38 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-815 Hi All- The Great08 challenge has just gone live! Over the last few months, the Great08 team has ensured the usability and tractability of the challenge. One noteworthy change is the inclusion of training sets for cosmological lensing. Code is also available to help ease data-reading issues. Check out the updated rules, and download data at www.great08challenge.info. Good luck! Doug Applegate Hi All-
The Great08 challenge has just gone live! Over the last few months, the Great08 team has ensured the usability and tractability of the challenge. One noteworthy change is the inclusion of training sets for cosmological lensing. Code is also available to help ease data-reading issues. Check out the updated rules, and download data at http://www.great08challenge.info.

Good luck!
Doug Applegate

]]>
By: Douglas Applegate http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-180 Douglas Applegate Sun, 30 Mar 2008 06:16:19 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-180 In past simulations, unsheared images were included. However, we don't include unsheared images in GREAT08 because every real extragalactic image has cosmological lensing. The size of the effect may be negligable for most work, and its definitely less strong than shearing in images such as the galaxy cluster Abell 2218, but we can still measure it. The lack of an unsheared training set is probably the biggest obstacle in this challenge. In real life, we'll have high resolution images of 'nearby' galaxies in addition to the very deep observations. Though still sheared, these hi-res galaxies can provide a 'training-set' for the deep observations. We'll include a set of hi-res simulated images that can be used to extract shape information about galaxies in addition to the lo-res simulated images. The core of the challenge is to develop a way to describe shapes of galaxies and understand how your system responds to shearing and convolution with gaussian-like kernels to make statements about the amount of shearing. Other image processing problems like measuring the point spread function (ie the convolution kernel), detecting sources, etc, are all removed. Hope that answers your question! Cheers, Doug Applegate In past simulations, unsheared images were included. However, we don’t include unsheared images in GREAT08 because every real extragalactic image has cosmological lensing. The size of the effect may be negligable for most work, and its definitely less strong than shearing in images such as the galaxy cluster Abell 2218, but we can still measure it.

The lack of an unsheared training set is probably the biggest obstacle in this challenge. In real life, we’ll have high resolution images of ‘nearby’ galaxies in addition to the very deep observations. Though still sheared, these hi-res galaxies can provide a ‘training-set’ for the deep observations. We’ll include a set of hi-res simulated images that can be used to extract shape information about galaxies in addition to the lo-res simulated images.

The core of the challenge is to develop a way to describe shapes of galaxies and understand how your system responds to shearing and convolution with gaussian-like kernels to make statements about the amount of shearing. Other image processing problems like measuring the point spread function (ie the convolution kernel), detecting sources, etc, are all removed.

Hope that answers your question!

Cheers,
Doug Applegate

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-175 hlee Tue, 25 Mar 2008 02:31:39 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-175 Thanks for your account for the GREAT08 challenge. The challenge sounds more complicated than I expected because there are layers like image processing and data processing before getting to the statistical inference problem. I began to understand that the photometry is very complicated and it depends on the templates (psfs or profiles for a extended source). It seems like that lensing becomes another layer of the convolving process or in image analysis. I should spend more time to learn details. Yet, there's a naive question that <i>why not a simulated image of no lensing (no shear) or partial lensing (only a proportion of the sky has lensing material)?</i> I bet it's something to do with the design of the challenge, though. Thanks for your account for the GREAT08 challenge. The challenge sounds more complicated than I expected because there are layers like image processing and data processing before getting to the statistical inference problem.

I began to understand that the photometry is very complicated and it depends on the templates (psfs or profiles for a extended source). It seems like that lensing becomes another layer of the convolving process or in image analysis.

I should spend more time to learn details. Yet, there’s a naive question that why not a simulated image of no lensing (no shear) or partial lensing (only a proportion of the sky has lensing material)? I bet it’s something to do with the design of the challenge, though.

]]>
By: Douglas Applegate http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-174 Douglas Applegate Tue, 18 Mar 2008 20:30:44 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-174 ou are definitely right about all the points you raise. The central challenge in gravitational lensing (and in the GREAT08 challenge) is that we have no idea what the unlensed source image looks like! To clear any confusion, let me briefly define some terms. 'Strong' lensing is when we see multiple distinct images of the same galaxy. These are the large arcs seen on Hubble images like http://hubblesite.org/hubble_discoveries/10th/photos/graphics/slide36high.jpg . We can use caustic theory and the location of the arcs to reconstruct the mass distribution of the lens. In GREAT08, we are interested in 'cosmological,', or 'weak' lensing. In this regime, the shapes are systematically distorted (sheared), but only at a very small level. Magnitude information is mostly ignored. The usual game is to define a shape measurement that averages to 0 over an ensemble of unlensed galaxies. A lensing signal would be a shift in the average away from 0. Each galaxy becomes an extremely noisy point estimation of the shear field caused by foreground material. The challenge is to develop an inference technique that accurately estimates the shear for an ensemble of galaxies. Some methods only rely on the averaging to 0 property, while some new methods are exploring hierarchical Bayesian methods to infer the prior distribution of shapes from the data. None recover the shear at a high enough accuracy yet. The main obstacle is removing optical effects of the telescope (deconvolve the image), which mimics the shearing signal. As you pointed out, we do need to determine if a galaxy is lensed. When discussing lensing by a large compact mass, such as a galaxy cluster, we are only interested in galaxies behind the source. We use other information, such as galaxy redshifts, to make this determination. In cosmological lensing, the filaments of mass throughout the universe lenses all galaxies, regardless of distance. However, the size of the effect is considerably smaller than with the large compact mass. In the GREAT08 simulations, all galaxies in an image are lensed. I'm happy to answer any lensing questions. Also, http://www.cosmocoffee.info has relevant discussion boards where GREAT08, lensing, and cosmological questions can be addressed. Cheers, Doug Applegate ou are definitely right about all the points you raise. The central challenge in gravitational lensing (and in the GREAT08 challenge) is that we have no idea what the unlensed source image looks like!

To clear any confusion, let me briefly define some terms. ‘Strong’ lensing is when we see multiple distinct images of the same galaxy. These are the large arcs seen on Hubble images like http://hubblesite.org/hubble_discoveries/10th/photos/graphics/slide36high.jpg . We can use caustic theory and the location of the arcs to reconstruct the mass distribution of the lens.

In GREAT08, we are interested in ‘cosmological,’, or ‘weak’ lensing. In this regime, the shapes are systematically distorted (sheared), but only at a very small level. Magnitude information is mostly ignored. The usual game is to define a shape measurement that averages to 0 over an ensemble of unlensed galaxies. A lensing signal would be a shift in the average away from 0. Each galaxy becomes an extremely noisy point estimation of the shear field caused by foreground material.

The challenge is to develop an inference technique that accurately estimates the shear for an ensemble of galaxies. Some methods only rely on the averaging to 0 property, while some new methods are exploring hierarchical Bayesian methods to infer the prior distribution of shapes from the data. None recover the shear at a high enough accuracy yet. The main obstacle is removing optical effects of the telescope (deconvolve the image), which mimics the shearing signal.

As you pointed out, we do need to determine if a galaxy is lensed. When discussing lensing by a large compact mass, such as a galaxy cluster, we are only interested in galaxies behind the source. We use other information, such as galaxy redshifts, to make this determination. In cosmological lensing, the filaments of mass throughout the universe lenses all galaxies, regardless of distance. However, the size of the effect is considerably smaller than with the large compact mass. In the GREAT08 simulations, all galaxies in an image are lensed.

I’m happy to answer any lensing questions. Also, http://www.cosmocoffee.info has relevant discussion boards where GREAT08, lensing, and cosmological questions can be addressed.

Cheers,
Doug Applegate

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-173 hlee Sat, 15 Mar 2008 05:50:07 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-173 Before I begin, I must confess that I never studied multiple hypothesis testing nor gravitational lensing. All my lessons were from a few talks and abstracts on both subjects. Another confession is that I didn't read the paper :(. Yet, I roughly thought that in the foreground, if lensing material exists, the lights from background galaxies are blurred and brightened but this lensing material is not strong enough to make distinct separable images (forgive my lack of vocabulary in lensing, I only remember one jargon, caustics). According to lensing theory, foreground material brightens/blurs/separates images of background galaxies. The problem is, how we know the observed galaxy is E4 or elongated E3 due to lensing. How we know the smaller magnitude of the galaxy is due to lensing or due to bias in distance measurement? Observing one single galaxy with or without the lensing effect to discover/test the existence of foreground material will be not easy but if there are many galaxies with or without lensing effects, I thought statistical multiple hypothesis testing would help to discover/test the existence of lensing material and its distribution. We welcome any discussion or sharing information both in statistics and astronomy, (probably, a list of contestants for this challenge and their approaches). Unless I go astray, I will watch for tutorials or relevant reviews on the topic for the slog postings. Before I begin, I must confess that I never studied multiple hypothesis testing nor gravitational lensing. All my lessons were from a few talks and abstracts on both subjects. Another confession is that I didn’t read the paper :( . Yet, I roughly thought that in the foreground, if lensing material exists, the lights from background galaxies are blurred and brightened but this lensing material is not strong enough to make distinct separable images (forgive my lack of vocabulary in lensing, I only remember one jargon, caustics). According to lensing theory, foreground material brightens/blurs/separates images of background galaxies. The problem is, how we know the observed galaxy is E4 or elongated E3 due to lensing. How we know the smaller magnitude of the galaxy is due to lensing or due to bias in distance measurement? Observing one single galaxy with or without the lensing effect to discover/test the existence of foreground material will be not easy but if there are many galaxies with or without lensing effects, I thought statistical multiple hypothesis testing would help to discover/test the existence of lensing material and its distribution.

We welcome any discussion or sharing information both in statistics and astronomy, (probably, a list of contestants for this challenge and their approaches). Unless I go astray, I will watch for tutorials or relevant reviews on the topic for the slog postings.

]]>
By: Douglas Applegate http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-172 Douglas Applegate Fri, 14 Mar 2008 21:49:36 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-172 Thanks for posting about GREAT08! I'm not sure I follow what you mean by a multiple testing problem. Could you elaborate? Cheers, Doug Applegate Thanks for posting about GREAT08!

I’m not sure I follow what you mean by a multiple testing problem. Could you elaborate?

Cheers,
Doug Applegate

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/comment-page-1/#comment-171 hlee Fri, 29 Feb 2008 19:10:37 +0000 http://hea-www.harvard.edu/AstroStat/slog/2008/the-great08-challenge/#comment-171 It sounds like a multiple testing problem. It sounds like a multiple testing problem.

]]>