some python modules

I was told to stay away from python and I’ve obeyed the order sincerely. However, I collected the following stuffs several months back at the instance of hearing about import inference and I hate to see them getting obsolete. At that time, collecting these modules and getting through them could help me complete the first step toward the quest Learning Python (the first posting of this slog).

There are quite many websites dedicated to python as you already know. Some of them talk only to astronomers. A tiny fraction of those websites are for statisticians but I haven’t met any statistician preferring only python. We take the gist of various languages. So, I’ll leave a general website aggregation, such as AstroPy (I think this website is extremely useful for astronomers), to enrich your bookmark under the “python” tab regardless of your profession. Instead, I’ll discuss some python libraries and modules that can be useful for those exercising astrostatistics and make their work easier. I must say that by intention I omitted a few modules because I was not sure their publicity and copyright sensitivity. If you have modules that can be introduced publicly, let me know. I’ll be happy to add them. If my description is improper and want them to be taken off, also let me know.

Over the past few years, python became the most common and versatile script language for both communities, and therefore, I believe, it would accelerate many collaborations. Much of my time is spent to find out how to read, maneuver, and handle raw data/image. Most of tactics for astronomers are quite unfamiliar, sometimes insensible to me (see my read.table() and data analysis system and its documentation). Somehow, one script language, thanks to its open and free intention to all communities, is promising by narrowing the gap for prosperous and efficient collaborations, Python

The first posting on this slog was about Python. I thought that kicking off with a computer language relatively new and open to many communities could motivate me and others for more interdisciplinary works with diversity. After a few years, unfortunately, I didn’t achieve that goal. Yet, I still think that these libraries and modules, introduced below, to be useful for your transition from some programming languages, or for writing your own but pro bono wrapper for better communication with the others.

I’ll take numpy, scipy, and RPy for granted. For the plotting purpose, matplotlib seems most common.

Reading astronomical data (click links to download libraries, modules, and tutorials)

  • First, start with Using Python for Interactive Data Analysis (in pdf) Quite useful manual, particularly for IDL users. It compares pros and cons of Python and IDL.
  • IDLsave Simply, without IDL, a .save file becomes legible. This is a brilliant small module.
  • PyRAF (I was really frustrated with IRAF and spent many sleepless nights. Apart from data reduction, I don’t remember much of statistics from IRAF except simple statistics for Gaussian populations. I guess PyRAF does better job). And there’s PyFITS for handling fits format data.
  • APLpy (the Astronomical Plotting Library in Python) is a Python module aimed at producing publication-quality plots of astronomical imaging data in FITS format (this introduction is copied from the APLpy site).

Statistics, Mathematics, or data science
Due to RPy, introducing smaller modules seems not much worthy but quite many modules and library for statistics are available, not relying on R.

  • MDP (Modular toolkit for Data Processing)
    Multivariate data analysis methods like PCA, ICA, FA, etc. become very popular in the astronomical society.
  • pywavelets (Not only FT, various transformation methodologies are often used and wavelet transformation ranks top).
  • PyIMSL (see my post, PyIMSL)
  • PyMC I introduced this module in a century ago. It may be lack of versatility or robustness due to parametric distribution objects but I liked the tutorial very much from which one can expand and devise their own working MCMC algorithm.
  • PyBUGS (I introduced this python wrapper in BUGS but the link to PyBUGS is not working anymore. I hope it revives.)
  • SAGE (Software for Algebra and Geometry Experimentation) is a free open-source mathematics software system licensed under the GPL (Link to the online tutorial).
  • python_statlib descriptive statistics for the python programming language.
  • PYSTAT Nice website but the product is not available yet. Be aware! It is not PhyStat!!!

Module for AstroStatistics
import inference (Unfortunately, the links to examples and tutorial are not available currently)

Without clear objectives, it is not easy to pick up a new language. If you are used to work with one from alphabet soup, you most likely adhere to your choice. Changing alphabets or transferring language names only happens when your instructor specifically ask you to use their preferring languages and when analysis {modules, libraries, tools} are only available within that preferred language. Somehow, thanks to the object oriented style, python makes transition and communication easier than other languages. Furthermore, script languages are more intuitive and better interpretable.

  1. robert:

    Thanks for the useful links! I’m in the process of switching from IDL to python and some of these modules are new to me. Your opening sentence makes me a bit worried though! I was really torn between R and python, but eventually considering also Rpy I went with python. What are your main arguments against python? I’m really curious, whether I should reconsider…

    Also – I was following this blog from the beginning, but roughly a year ago it seemingly died. I was suprised to find many new posts here today when I actually visited the website. Could there be a problem with your RSS feed? Neither my google reader nor NetNewsWire see posts after 11th Dec 2008.

    Thanks and keep up the nice blog!

    11-18-2009, 4:38 am
  2. hlee:

    I appreciate your kind words and my apologies for the inconvenience by RSS feed failure. From time to time, I look for a solution but most of description assumes a certain directory structure for the wordpress blogging package. This slog not being located under my home directory makes a fix difficult. Even if live feed fails from time to time, it generally has worked out after correcting the rss feed url.

    To answer your question, as mentioned, some motivation helps learning a new language. It was not python that I was banned from but a data analysis package that uses python. Not being allowed to use that package, I lost a reason to use python, while most of my stuffs can be done with R and a bunch of others (IDL,c, Fortran,c-shell script, pearl, Matlab, Mathematica, SAS, Minitab, SPSS, and few more). The ability of communicating and embracing other languages is the great merit of python, I think.

    I’m happy to know that you find these modules and libraries are informative. Thanks!

    11-19-2009, 4:10 pm
Leave a comment