Comments on: [ArXiv] Pareto Distribution http://hea-www.harvard.edu/AstroStat/slog/2008/arxiv-pareto-distribution/ Weaving together Astronomy+Statistics+Computer Science+Engineering+Intrumentation, far beyond the growing borders Fri, 01 Jun 2012 18:47:52 +0000 hourly 1 http://wordpress.org/?v=3.4 By: vlk http://hea-www.harvard.edu/AstroStat/slog/2008/arxiv-pareto-distribution/comment-page-1/#comment-206 vlk Sat, 19 Apr 2008 19:58:35 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=265#comment-206 A little digression away from Astronomy.. even in Economics the Pareto can be a diagnostic tool. If there is no scale in the system, then that system will show a self-similar or power-law distribution. This happens, for example, in feudal societies where there is no governmental authority to redistribute wealth via targeted taxation. (I recall seeing a paper recently which plotted the wealth and income in medieval Hungary, and that was a power-law far as the eye could see. Ask if you want to see that paper and I will try and hunt it down.) In modern societies, it seems the lower income groups are no longer power-law distributed, though the upper income groups still are. (If you google for Pareto or power-law and income distributions you will find a whole lot of work that has been done on this.) A little digression away from Astronomy.. even in Economics the Pareto can be a diagnostic tool. If there is no scale in the system, then that system will show a self-similar or power-law distribution. This happens, for example, in feudal societies where there is no governmental authority to redistribute wealth via targeted taxation. (I recall seeing a paper recently which plotted the wealth and income in medieval Hungary, and that was a power-law far as the eye could see. Ask if you want to see that paper and I will try and hunt it down.) In modern societies, it seems the lower income groups are no longer power-law distributed, though the upper income groups still are. (If you google for Pareto or power-law and income distributions you will find a whole lot of work that has been done on this.)

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/arxiv-pareto-distribution/comment-page-1/#comment-190 hlee Tue, 15 Apr 2008 00:33:33 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=265#comment-190 A few questions I wish to address. My vague memory tells that this Pareto distribution appears in economics to explain the 80% wealth is distributed among 20% people. The broken power law implies this wealth distribution is different before and after the breaking point, whereas the truncated power law includes an upper limit into fitting process but the wealth allocation is still same. My questions are how to test whether the distribution is a power law, a broken power law, or a truncated power law, how to test one of these power laws is a good fit, and how to determine the breaking point is significant regardless of the righteousness of the broken power law. The former question also asked for model selection or variable selection issues, in which so far physics has played a dominant role to choose among three may be more for multiple broken points but I wonder if statistics can contribute further. I really do appreciate your telling different nomenclatures so that I can look for those names with care in future. A few questions I wish to address. My vague memory tells that this Pareto distribution appears in economics to explain the 80% wealth is distributed among 20% people. The broken power law implies this wealth distribution is different before and after the breaking point, whereas the truncated power law includes an upper limit into fitting process but the wealth allocation is still same. My questions are how to test whether the distribution is a power law, a broken power law, or a truncated power law, how to test one of these power laws is a good fit, and how to determine the breaking point is significant regardless of the righteousness of the broken power law. The former question also asked for model selection or variable selection issues, in which so far physics has played a dominant role to choose among three may be more for multiple broken points but I wonder if statistics can contribute further.

I really do appreciate your telling different nomenclatures so that I can look for those names with care in future.

]]>
By: TomLoredo http://hea-www.harvard.edu/AstroStat/slog/2008/arxiv-pareto-distribution/comment-page-1/#comment-189 TomLoredo Fri, 11 Apr 2008 19:56:21 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=265#comment-189 Nice post, Hyunsook. The Pareto distribution (and its relatives) appears all over in astronomy, but almost never with the name so familiar to statisticians. Instead, you see it as: power-law distribution, truncated power-law, Salpeter initial mass function, Schecter function (really a kind of gamma dist'n, but in a regime where it looks like a truncated power law). Roughly speaking, a power law is an indication of some kind of scale-free behavior. The slope can be an important indicator of the underlying physics, and astronomers often don't do a great job in estimating it (esp. when there is measurement error). But truncation or "breaks" (change in the Pareto power law index) are often the most exciting thing, as they indicate the presence of important physical scales in the underlying process. So astronomers do often find themselves trying to infer breaks and truncation. For truncation in particular, the statistics is messy (I had to look into it for inferring the edge of the Kuiper belt in a paper some years ago). The likelihood is "non-regular" (i.e., the usual asymptotics doesn't hold), and there is still research being done in statistics on how to do things like calculate sound confidence or credible regions. Nice post, Hyunsook. The Pareto distribution (and its relatives) appears all over in astronomy, but almost never with the name so familiar to statisticians. Instead, you see it as: power-law distribution, truncated power-law, Salpeter initial mass function, Schecter function (really a kind of gamma dist’n, but in a regime where it looks like a truncated power law). Roughly speaking, a power law is an indication of some kind of scale-free behavior. The slope can be an important indicator of the underlying physics, and astronomers often don’t do a great job in estimating it (esp. when there is measurement error). But truncation or “breaks” (change in the Pareto power law index) are often the most exciting thing, as they indicate the presence of important physical scales in the underlying process. So astronomers do often find themselves trying to infer breaks and truncation. For truncation in particular, the statistics is messy (I had to look into it for inferring the edge of the Kuiper belt in a paper some years ago). The likelihood is “non-regular” (i.e., the usual asymptotics doesn’t hold), and there is still research being done in statistics on how to do things like calculate sound confidence or credible regions.

]]>
By: hlee http://hea-www.harvard.edu/AstroStat/slog/2008/arxiv-pareto-distribution/comment-page-1/#comment-182 hlee Mon, 07 Apr 2008 22:51:50 +0000 http://hea-www.harvard.edu/AstroStat/slog/?p=265#comment-182 A review of arxiv:physics.data-an:0706.1062 on power laws from the slog: <a href="http://hea-www.harvard.edu/AstroStat/slog/2007/astroph-07061062/" rel="nofollow">Everything you wanted to know about power-laws but were afraid to ask</a> A review of arxiv:physics.data-an:0706.1062 on power laws from the slog:
Everything you wanted to know about power-laws but were afraid to ask

]]>