This ChangeLog covers both the Funtools library and the suite of
applications. It will be updated as we continue to develop and improve
Funtools. The up-to-date version can be found here.
Beta release 1.0.b28 (06/26/01)
- In funcnts, removed extra new-lines from the primary table,
inadvertently added in cases where zero-area rows are skipped.
- Added code to support columns in binary tables that do not have names.
- Changed the ds9 radial plot so that the radius (x value) for each
plotted point is taken to be the middle of the annulus, (Rin+Rout)/2,
instead of just the inner annulus, Rin.
Beta release 1.0.b27 (06/21/01)
- Added missing new-lines in funcnts primary tables that did not
have -r switch.
- Filtering with dynamic shared objects (gcc only) now links the
region code into the shared object, instead of relying on finding the
region code in global space.
- A few minor changes to column headers in funcnts.
Beta release 1.0.b26 (05/21/01)
- Fixed funcnts to work properly when the background region
overlaps the source and therefore explicitly excludes the source.
It was simply ignoring the source regions in such a case.
Added a second DS9 init file, funcnts2.ds9, which contains funcnts
and radial profile routines that work from the image stored in DS9's
memory, rather than the original FITS file. This is useful in cases
where no original FITS file exists (e.g., a temp file was created
by a program and sent to DS9).
- Fixed funcnts so that region specified with a variable number of
arguments must come last (as is the case with panda and the n=
accelerator). Thus, a region specification such as "pie 504 512 10 20
30 & circle 504 512 10" is now properly an error. Using "circle 504 512 10
& pie 504 512 10 20 30" instead ensures that the circle is applied to
- Fixed funcnts output of radii/angles when boolean expressions
such as "pie & annulus" are specified.
- Enhanced funcnts so that -r outputs valid radii for circular regions.
Beta release 1.0.b25 (4/19/01)
- Added support to funtable to generate a FITS binary table
from an image. By default, a 3-column table is generated, where the
columns are "X", "Y", and "VALUE". For each pixel in the image, a
single row (event) is generated with the "X" and "Y" columns assigned
the dim1 and dim2 values of the image pixel, respectively and the
"VALUE" column assigned the value of the pixel. If the -i
("individual" rows) switch is specified, then only the "X" and "Y"
columns are generated. However the number of rows (events) are written
out for each image pixel is equal to the value of that image pixel(or
1, whichever is larger).
- Added -z switch to funcnts to display shapes in primary table
that have no area. This is useful when automatic processing assumes
that there are a set number of rows.
- Reworked how the "$region" column is handled. A "$region" column
can now be added to a table already having a "region" column. If a
"region" column already exists, the new column will be "region1". If
that exists, we try "region2", etc.
- Added the header parameters EXTNAME="IMAGE" and EXTVER=1 when an
image extension of created automatically.
- When funcnts is run with -r, the rad1, rad2, ang1, and ang2 columns
are filled in with "NA" if the shape is not annulus, pie, or panda (instead
of being left blank).
- Changed funcnts to display arc-sec/pixel instead of degrees/pixel in the
- Fixed bug in FunColumnSelect() that prevented a new table from
being created that consists only of new columns.
- Fixed processing of blank values in FunImageRowGet() (was a SEGV).
- Fixed fundisp processing of images (SEGV upon completion, trying to
free space only used by tables).
- Fixed bug in funcalc which left executable in /tmp space if -n
(no execute) was specified.
- Fixed various bugs processing raw event files, especially on
little-endian machines and reading these files via stdin.
- Changed rad1, rad2 columns to radius1, radius2 in funcnts.
- Purified dynamically loaded filter code (since purify now can work
Beta release 1.0.b24 (03/26/01)
- First release of funcalc, the Funtools table calculator.
funcalc is a calculator program that allows arbitrary
expressions to be constructed, compiled, and executed on columns in a
Funtools table (FITS binary table or raw event file). It works by
integrating user-supplied expression(s) into a template C program,
then compiling and executing the program. funcalc expressions
are valid C statements, although some important simplifications (such
as automatic declaration of variables) are supported.
Within a funcalc expression, reference is made to a
column of the current row using the C syntax
cur->[colname], e.g. cur->x, cur->pha, etc. Local scalar
variables can either be defined using C syntax at very the beginning
of the expression, or else they can be defined automatically by
funcalc (to be of type double). Thus, for example, a swap of
columns x and y in a table can be performed using either of the following
equivalent funcalc expressions:
temp = cur->x;
cur->x = cur->y;
cur->y = temp;
temp = cur->x;
cur->x = cur->y;
cur->y = temp;
When this expression is executed using a command such as:
funcalc -f swap.expr itest.ev otest.ev
the resulting file will have values of the x and y columns swapped.
Many other features are available in funcalc to make table
manipulation easy. See the Funtools program.html documentation.
- First release of the funtools.ds9 set-up file for adding
Funtools analysis programs to the DS9 Analysis menu. This set-up file
is installed in the same bin directory where Funtools programs are
installed and can be loaded into DS9 from the Load Analysis
Commands ... option of the Analysis menu. Alternatively,
you can tell DS9 to load this file each time it starts by adding the
file to the Edit->Preferences->Analysis
Menu->Analysis File menu option.
- Added support for non-integral binning of binary tables. The bincols
specifier on the command line now can take the form:
where the tlmin, tlmax, and binsiz specifiers determine the image binning
dim = (tlmax - tlmin)/binsiz (floating point data)
dim = (tlmax - tlmin)/binsiz + 1 (integer data)
These tlmin, tlmax, and binsiz specifiers can be omitted if TLMIN,
TLMAX, and TDBIN header parameters (respectively) are present in the
FITS binary table header for the column in question. Note that if
only one parameter is specified, it is assumed to be tlmax, and tlmin
defaults to 1. If two parameters are specified, they are assumed to be
tlmin and tlmax.
- Added "mask=transparent" support to the plist argument in
FunTableRowGet(). If this string is passed in the call's plist
argument, then all events are passed back to the user. This is useful
when FunColumnSelect() specifies "$region" as a column in order to
return the regionid value for each event. In such a case, events
found in a region have regionid >0, events passing the filter but not
in a region have regionid == -1, events not passing the filter have
- Added FUN_WCS0 to the FUN_WCS option available to FunInfoGet().
The original FUN_WCS option returns WCS Library handle (for use with
Doug Mink's wcssubs library) suitable for use with images, regardless
of whether the data are images or tables. For this structure, the WCS
reference point (CRPIX) has been converted to image coordinates if the
underlying file is a table (and therefore in physical coordinates). The
new FUN_WCS0 structure has not had its WCS reference point converted
to image coordinates. It therefore is useful when passing processing
physical coordinates from a table.
- Added -G switch to funcnts to print out floating point values
with maximum %.14g precision.
- Added -l switch to fundisp, which displays the pixels of an image as
a list with X, Y, VAL columns.
- Added support for images to funhist. The program will create
a histogram of the values found in each pixel, or it can perform
a projection over either axis.
- All Funtools programs now accept "-" to mean "stdin" or "stdout",
when the io mode is "r" or "w", respectively.
- Changed behavior of the prec (precision) argument in FunParamPutd()
(and the underlying fitsy routine ft_cardfmt()) so that if the double
value being put is less than 0.1 or greater than or equal to
10**(20-2-prec), then %20.[prec]e format is used, otherwise
%20.[prec]f format is used.
- Fixed behavior of "merge=replace" in FunColumnSelect() so that if
tlmin/tlmax values are not specified in the replacing column, but are
specified in the replaced column, then the original tlmin/tlmax values
are used in the replacing column.
- Improved funcnts error-handling when no valid region is specified.
- Fixed region parsing of '#' comment character so that comments
are terminated by new-lines, but not ';'. This is more intuitive behavior.
- Fixed region parser so that a region (without parens), followed by
a column expression (e.g., "circle 5 5 1,pha==4") is processed correctly.
- Fixed funcnts so that the timecorr parameter specified by -t
[timecorr] can be in lower case.
- Fixed panda region shapes when processing a blocked image. The
number of pies and number of annuli (args 5 and 8) were incorrectly
being divided by the block factor (i.e., they were being treated as
- Fixed bug in funcnts that resulted in slightly smaller
integerized pixel boundaries being used when filtering events. This
does not affect ordinary event filtering(in fundisp, funtable, etc.).
In funcnts (which filters binary table events using image-style pixel
filtering), this bug could result in fewer photons being counted than
is the case when the equivalent image is used.
- Fixed funcnts to work with raw event files on little-endian machines.
- Fixed funhist so that it will read data from a pipe.
- Fixed region parser (and funcnts) so that an include file ending
with a comment stops the comment at the end of the include file.
- Clarified the meaning of the "," operator (should it be "or" or
"and") between a region and a non-region expression in a filter: if
the second operand in the expression contains a region, the operator
is "or", otherwise it is "and".
- Fixed bug in funmerge, which was not handling integerization of
negatively-valued physical pixels properly (not actually used in any
Beta release 1.0.b23 (02/16/01)
- Fixed funcnts to report area correctly in arc-seconds.
- Fixed funcnts to report radii correctly when summing.
- Fixed bug in fundisp: a SEGV when trying to display an ASCII column
from a binary table.
Beta release 1.0.b22 (02/15/01)
- Funcnts now will automatically output appropriate columns from
the primary table in units of arc-seconds instead of pixels if WCS info
is present. Use -p to force the output to be in pixels.
- Added qualitative exposure correction to funcnts by means of the switch
"-e source_exp[;background_exp]". For each region, the average exposure is
calculated and net counts (and background) are divided by the average
exposure. See programs.html for more info.
- Added qualitative time correction to funcnts by means of the switch
"-t source_time[;background_time]". The net counts (and background) are
divided by this time. See programs.html for more info.
- Improved funcnts output. For example, column units are displayed
(since surf_bri units now can be cnts/pix**2, cnts/arcsec**2, etc.)
- Changed funcnts.gnuplot and funhist.gnuplot scripts to funcnts.plot
and funhist.plot, respectively. The new scripts take an argument
such as "gnuplot" or "ds9" and output data appropriate for each
target. Also enhanced funcnts.plot so that it senses the axis units
- Fixed bug in funcnts when handling regions whose centers are out of the
image. Processing often resulted in BUS ERROR or zero counts, and it took
forever to reach those results
- Fixed bug in FunImagePut() when outputting float data on
little-endian machines (PCs/Dec Alpha) -- an erroneous error was
signaled trying to convert from native to IEEE before writing.
- Fixed bug in fundisp when displaying the mask of a double/float
image using the mask=all option.
- Fixed bug in fitsy/headimage/ft_imageloadhead(fits), in which the
default value for LTV[1,2] was incorrectly set to 1.0, not 0.0. This
means that region physical coordinates applied to FITS images and
arrays that did not have LTM/LTV keywords were 1 pixel off.
- Fixed obscure bug in region circle processing when a block factor
is specified in the section command but the circle has radius less
than the block.
- Fixed bug in FunImageGet which was returning the full image in
cases where a region was specified but no image pixels were in the
region. An empty image is now returned.
Beta release 1.0.b21 (02/01/01)
Beta release 1.0.b20 (11/29/00)
Beta release 1.0.b19 (11/21/00)
- Fixed bug in filter code that caused a SEGV on Solaris machines
when the first specified spatial region is an exclude region. Our
user-supplied qsort/compare algorithm was confusing the Solaris qsort()
routine, causing it to SEGV by trying to process a record prior to the
beginning of the passed array of records.
Beta release 1.0.b18 (11/13/00)
- Fixed bug in handling bitpix=-32 (single float) images.
- Fixed gio gskip routine to better handle skip of 0 bytes.
Beta release 1.0.b17 (11/10/00)
- Fixed working of $REGION keyword in funtable.
- Removed FunFlush() from end of funtable (it was redundant).
- Fixed bug in gopen() handling of "pipe:".
- Fixed bug in gseek() where pipes, streams, and sockets were
not skipping bytes properly (that can do skips, but cannot do other
sorts of seek).
Beta release 1.0.b16 (10/23/00)
- Added -s switch to funcnts to support display of summed results
(as well as individual results for each region), i.e. each row in the
summed bkgd-subtracted table contains the sum of counts and areas from
- Added -f [format] switch to fundisp to allow control over the
display format for each data type.
- Fixed bug in which regions could be incorrectly re-ordered. This
was a problem with annular ellipses and rectangles created by ds9 and
then used in funcnts (the only program where the order of the regions is
Beta release 1.0.b15 (10/11/00)
- Changed the names of routines FunEventsGet() and FunEventsPut()
to FunTableRowGet() and FunTableRowPut(), respectively. The old names
are still valid (using #define in funtools.h), so no code change is
- Changed funevents program name to funtable, in line with API changes.
- Renamed FunInfoGet/Put() parameter FUN_EVSIZE to FUN_ROWSIZE, in
order to reflect change from use of "events" to use of "row" in
funtools binary table support. The old name is still supported as
- Completed first version of funmerge program to merge FITS binary tables.
- Fixed bug in line region shape that was causing a SEGV with event data.
- Fixed minor compiler warnings using "gcc-Wall".
- Added the ability for FunOpen() to open a list of event files and
read events from this list synchronously or asynchronously. This
facility is part of an experimental set of parallel processing
techniques that are being added to funtools. Documentation will be
forthcoming when we know which techniques have value!
Beta release 1.0.b14 (9/22/00)
- Added first version of funmerge program to merge FITS binary tables.
- Changed the output format of funcnts so that the main results
(background-subtracted table) are displayed first. This means that the
result columns always start at line 4 in the file (after a 1-line
comment and a 2-line header) and end at the first blank line. The
fixed format makes it easier for programs such as sed to extract
results for further processing. For example:
csh> cat fun.sed
csh> funcnts snr.ev[pha==1] "annulus 512 512 0 200 n=8" | sed -f fun.sed
1 49.000 7.000 0.000 0.000 1941 0.025 0.004
2 91.000 9.539 0.000 0.000 5884 0.015 0.002
3 129.000 11.358 0.000 0.000 9820 0.013 0.001
4 159.000 12.610 0.000 0.000 13752 0.012 0.001
5 176.000 13.266 0.000 0.000 17652 0.010 0.001
6 183.000 13.528 0.000 0.000 21612 0.008 0.001
7 137.000 11.705 0.000 0.000 25528 0.005 0.000
8 198.000 14.071 0.000 0.000 29420 0.007 0.000
- Fixed bug in row# processing in which all range was ignored if
lo range value was 1 (e.g., row#=1:7).
- Fixed bug in event header processing in which the multiple
instances of keywords HISTORY, COMMENT, and CONTINUE were not all
being copied from the old to the new header (e.g. in funevents).
- Fixed processing of ARRAY() and EVENTS() specifiers in FunOpen().
- Fixed 'make clean' directive so that it also cleans funtools subdirs.
Beta release 1.0.b12 (9/5/00)
- Improved the performance of panda regions so that the funcnts
"wall time" now is proportional to the size of the panda region, not
the size of the image. (The latter is the case with the pie shape; use
of panda is recommended over pie.) This means that it is possible to
run funcnts on an ACIS file at zoom 1 (8192x8192) in seconds rather
than (tens of) minutes.
- Added support for INET sockets to gio and hence, to funtools.
This means that you can read/write from/to sockets on other machines,
creating a distributed pipeline. For example:
on m1: funevents foo.ev m2:1428
on m2: funevents :1428 m3:1428
on m3: funevents :1428 ...
etc. Tests indicate that this is faster than pipes on a single
machine, once the CPU is saturated on that machine. (But note that it
is not faster until the CPU is saturated, due to the efficiency of
Unix pipes and the I/O wait time on non-saturated CPUs.) This new
facility implements the parallel processing technique called "process
decomposition" for pipelines, in which a pipeline process is run on
several machines at once, with different machines handling separate
parts of the pipeline process.
NB: socket support requires that the libraries:
be added to the Solaris link line.
- Added support for the row#=lo:hi keyword to process specific rows
in a FITS binary table. For example:
funevents "test.ev[row#=3:8]" stdout ...
funevents "test.ev[row#=(3,8)]" stdout ...
will only process rows 3 to 8 (inclusive) of the test.ev file. Along
with image section specification, use of the row#= keyword implements
the parallel processing technique called "data decomposition", in
which several copies of a single program operate on different parts of
a single data file.
- Added guard code to image region processing to catch illegal event
- Fixed bug when writing FITS image extensions in which dim1, dim2,
and bitpix were being output as 0.
- reversed the y row order of displayed images in fundisp, so that pixel
(1,1) is in the lower left corner, as is the case for ds9 image display.
Beta release 1.0.b11 (8/10/00)
- Fixed annulus accelerators and panda regions -- again. Old
problems (from the original implementation) were uncovered related to
the use of these shapes in boolean expressions. Documented an old
restriction that panda and accelerators must be put last in a boolean
expression and added code to signal an error if they are not placed last.
- The behavior of the point shape was changed so that multiple x,y
pairs in a single shape specifier now are assigned different region
ids. This makes the behavior of points and annuli consistent with one
Beta release 1.0.b10 (8/08/00)
- Fixed annulus accelerators and panda regions. These were
broken when dynamic loading was implemented.
- Fixed a bug in the event filter body code (i.e., the basis for
the slave filter program). Reading the data sometimes incorrectly
calculated the number of events being passed -- which only showed up
occasionally on the Alpha!
Beta release 1.0.b9 (8/03/00)
- Removed compilation of some extraneous routines from wcs library.
Also renamed wcssubs directory to wcs.
- Added calls to hlength() before wcsinit(). This is necessary in
ds9 (and is a safeguard in other programs) because once hlength() is
called before any invocation of wcsinit(), it must always be used.
- Fixed bug in filter code in which CTYPE1 and CTYPE2 param values
were not being passed to wcsinit() as valid FITS strings: the single
quotes were missing.
- Fixed a bug in fitsy in which the card buffer was not being
null-terminated properly when a "card insert" call reallocated space
for more cards.
- Added protective code so that one cannot set FILTER_PTYPE to
"dynamic" if dynamic filter objects are not available.
- Ported to Debian Linux, which (believe it or not) required
removal of extraneous strdup() and strstr() declarations in the code
(apparently these are macros in that version of Debian gcc, so you
cannot declare them).
Beta release 1.0.b8 (8/01/00)
- Added new filter ptype ("contained"), which builds a separate
process by compiling both the main routine and the region code. This
is different from the "process" ptype, which compiles the main
routine, but links in pre-compiled region code (in order to make the
program build more quickly). It is needed by ds9 so that the latter
does not have to keep track of the compiled region code module.
Beta release 1.0.b7 (7/25/00)
Beta release 1.0.b6 (7/15/00)
- Ran Purify with each high level program and each funtest
- Changed behavior of merge=update option of
FunColumnSelect() so that the update of the user column value only
takes place if the user mode has "w" in it. Previously, merge=update
overrode the mode flag and always updated the user value. Note that
all calls to FunColumnSelect with merge=update must change "r" to "rw"
in order to have that user column merged. (The merge=replace already
was checking the mode flag -- the fact that they were doing different
things is a bug.)
- Added ability to FunOpen extensions by HDU name. (I
thought I had already done this!)
- Fixed bug that was adding a NULL table at the end of
most binary table files (i.e., those that were not copying the rest of
the input file).
- Re-ported to Alpha. The problem found most often was the
casting of pointers to ints when doing pointer calculations, i.e.:
char *s, *t;
n = ((int)s - (int)t);
which is invalid on the 64-bit Alpha. Instead use:
char *s, *t;
n = (s - t);
Also broadened the check for use of dlopen in configure to match Alpha's
library configurations (On Alpha, dlopen is in libc).
- Changed FunColumnActivate() so that funtools will
process columns in the sorted order specified by that routine. Thus:
fundisp foo.ev "time y x"
will display columns in that order.
Sorting does not take place if the activate list contains only exclude
columns (since there is nothing to sort). Also, you can turn off
sorting altogether (mimicking the old behavior) by calling
FunColumnActivate() with a "sort=false" in the plist argument:
FunColumnActivate(fun, "y x", "sort=false");
or by adding "sort=false" to the activate string itself:
# by default, its sorted
fundisp $E "time y x"
TIME Y X
---------------- ------- -------
6.8500 -7 -7
6.8600 -7 -7
# turn off sorting
./fundisp $E "time y x sort=false"
X Y TIME
------- ------- ----------------
-7 -7 6.8500
-7 -7 6.8600
-7 -7 6.8700
Beta release 1.0.b5 (7/8/00)
- Changed all FunParamSet calls to FunParamPut, to make the
naming consistent with FunImagePut, FunEventsPut, etc.
- Fixed bugs preventing tlmin/tlmax from being changed by the
user in binary tables. Also tlmin/tlmax are now written out using a
data type that matches the data type of the respective column.
- Extended filter syntax to allow "," as separator between
filename and filters (as well as brackets), i.e.:
foo.fits,events # event extension
foo.fits,pha==1 # filter on default extension
foo.fits,1 # first extension
Note that all but simple expressions will need to be quoted because
of the shell:
foo.fits,pha==1&&pi==2 # & tells shell to run in bkgd
foo.fits,pha==1||pi==2 # similar problems with pipes
foo.fits,circle(1,2,3) # parens are grabbed by shell
- Fixed configure so that --with-lib is no longer necessary
to generate a single funtools.a library. Removed this argument from
saoconfig. Note that:
now works properly again, so saoconfig should not be used.
- Changed FunFlush() mode argument (single characters) to a
plist argument (keyword arguments). In particular,
(or FunFlush(fun, "copy=remainder"); )
This syntax extension allows FunFlush to support the copy of the
extension associated with the reference handle, which allows one to
copy any extension from an input file to an output file:
/* open a new input extension */
ifun = FunOpen(...);
/* make this new extension the output reference extension */
FunInfoPut(ofun, FUN_IFUN, &ifun, 0);
/* copy the current reference extension to output */
- Fixed bugs in region parser that caused pure floating
point positions (i.e., numbers w/o format characters) always to be
interpreted as pixels. Also fixed galactic and ecliptic conversions.
Index to the Funtools Help Pages
Index to the Funtools Help Pages
Last updated: November 17, 2005