filename: oper.man
Date: 26 Feb 95
orig: deh
tlm: 21 Nov 96


Operator's Manual for Visual Inspection -ROSAT RESULTS ARCHIVE

1.0 General
2.0 FLAGS
3.0 Operator's Instructions
4.0 SAO site specific notes

1.0 General

During the summer of 1994, the ROSAT consortium (MPE, Leicester/UK, GSFC, and SAO) agreed that a significant effort was warranted to produce a reliable source catalog from the SASS processing of pointed observations. At the same time, it was felt that the source lists produced by SASS should be available in the public archive.

Since SASS processing occasionally produced unreliable results, and since it was known that there were certain problems that SASS did not attempt to solve (i.e. aspect differences for obis separated by 6 months), it was decided that the SASS source tables from the reprocessing (when the switch to RDF files occurred) would form the basis of the catalogue, and, that quality control (QC) flags would be attached to key source and field parameters so that when the SASS results passed into the public domain, they would already have these QC flags.

Since there was a certain urgency felt, it was decided to deal with the straightforward fields first since they should present the least amount of problems. Therefore the 'RRA Project' came into being by mutual consent and discussion and has, as its primary goal, the production of a high quality source catalog. Phase Zero of the project was prototyping software to generate the QC flags. This essentially took the fall of 1994. Phase 1 is meant to mean the generation of the easy part of the catalog, and is scheduled to start in summer 1995. Phase 2 will overlap the ongoing parts of phase 1, and is meant to deal with the more vexing problems.

In a sense, the RRA project is the construction of a pipeline (involving human participation as one of its elements) which takes the SASS as its input, and produces entries in a database, which, in the first instance, will be the well known 'browse' system.

2.0 Flags

Early in the project, it was decided to have true flags, not status indicators. This means that a flag is either true or false, it cannot have other values. Furthermore, we decided that all flags would be 'false' if everything was all right. Although this may seem counter-intuitive at first, the committee had the sense that the key questions could be phrased: "Is the position produced by SASS suspect?"

For obvious reasons, there are two sorts of flags: those which pertain to individual sources, and those which describe conditions of the whole field.

NB:    *** The operator is referred to other documents for details:**********

flags.sum               is a summary of all flags. It should be viewed on a wide screen, and printed                                 landscape with smallish type.

flagset.hri              The detailed specification of HRI flags, and other matters. ***************************************************************************

2.1 Rationale for flags

The QC flags will serve many functions. They will serve as guides for views of the database. The most common use of the catalog will be via electronic means since this source list, unlike the Einstein 2E catalog, will not be complete until some undetermined date in the future. Normally, users will not want to retrieve sources that have been identified as 'false sources' (source 'f' flag set to 'true').

The flags will also alert users to certain situations about the environment since most users will not have access to the image.

2.2 Automatic vs. human decision

Many of the flags can be set directly by considering the values of SASS output parameters (e.g. if the s/n is below some threshold value). Some flags may be dependent on several other flags, and some are set exclusively by visual inspection (VI).

Therefore, there are three logical steps in the QC process. In step 1 we do everything which can be programmed. This includes setting the automatic flags on the basis of SASS output, converting the RDF fits files to system files, and generating any required new files (e.g. a large smoothed image of the central region of the HRI field).

In step 2, the VI operator examines the field and sets any required field and source flags. These in turn, may set other flags (logical step 3). In practice, step 3 operations are meshed to step 2 procedures and thus do not appear as a separate process.

2.3 Extraneous matters

The field flag M is set whenever the operator finds that an obvious source has been missed by the detection algorithm. It was decided that the VI operator should identify the source(s) by clicking on the source, thereby recording the RA, DEC, and a few other rough source parameters. It was felt that the additional time required for this process was a worthwhile investment, and that the tables of missed sources would not, in the first instance, be a part of the RRA, but be a 'common property' of the consortium. In phase 2, it may be possible to get some "SASS equivalent" parameters by giving the rough positions to the ML algorithm.

Another set of quantities produced by the RRA project will be the statistics of flags set, and these numbers will be kept.

3.0 Operator's Instructions

We assume that the preparatory tasks ('step 1') will be done by cron jobs. Here we attach a suggested procedure, and eventually some logistical information for our local site (SAO). Thus the other sites can replace sections 3.4 and 3.5 if they differ substantially. The user interface has been designed to provide rapid display and easy input of decisions. Please bring to our attention suggested improvements.

The MIDAS/EXSAS system for the PSPC is described in the documentation by Bohler, and the operator should consult that for the mechanics of operation. Likewise, the IDL/HRI system is described by Corcoran's 'hvi2users_guide.txt'. Here we discuss the decision making process. 3.1 Suggested Procedure The field is shown with overlayed source circles (squares) for the PSPC (HRI). In the former case, the circle approximates the size of the PRF; in the latter, the size of the detect cell. It is often the case that one may fiddle the color look-up table and cutoff levels at the beginning of a session (deh sets the LUT to black and white in his IDL startup file). Here we draw your attention to the primary and secondary flags you should evaluate, and list the automatic ones for reference. Picture examples for each condition appear in the HRI SOURCE FLAG DESCRIPTION (//heasarc.gsfc.nasa.gov/rosat/data/qsrc/www/hrisflags.html). We recommend first examing the field, setting any of the four field flags listed in 3.2. In practice, the D (deferred) and A (bad aspect) will be very rare, and hence need close to zero operator time. This leaves E (strong, extended emission which will adversely affect the background) and M (missed sources). Finding missed sources should be done carefully at this stage so that later source flagging is easier. It should not take long if the overlays for low s/n sources make them clearly visible. After setting E and M, we scan the sources to note which might require flag setting and further examination. This can best be done with the zoom feature (IDL/HRI) or the 'Quick Mode' (MIDAS/PSPC). For the HRI, one can use both windows...the full field for the outer regions, and the smoothed blowup with 2" pixels for the central region. After finishing with the field flags one can click on the sources needing flagging. For the IDL system, it is possible to have the zoom feature operational in one display and click on (thereby 'selecting') particular sources in the other window. Only 5 field flags need be considered by the operator. In practice, f, p, and w should be very quick and obvious (see section 3.3, below). However, please note the difference between 'c' which means the algorithm was confused - there are more than one source; and 'm' which means there have been multiple detections of only one source. 3.2 FIELD FLAGS

AUTOMATIC

B                  bkg_susp background map suspect
F                  fals_det one or more sources were given the f(alse) flag
S                  slist_mis empty or missing source list

OPERATOR DECISION

A                  asp_susp aspect solution may be bad
D                  deferred field deferred
E                  ext_emis extended emission in field
M                 src_mis one or more sources missed by detect

A                  The common forms of aspect error are either registration so bad that every source appears twice, with a fixed offset of any size; or a poor registration of obis which leads to smearing of the image by 5" to 15" (typically). The former case is extremely rare, and in rev2 processing, we expect that the latter case will also be rare. In the HRI, if a known point source is elliptical, or if all sources near the field center have the same ellipticity, bad aspect is probably the cause. However, we do not intend to flag bad aspect which results in egg-shaped HRI sources from residual wobble problems. This is normally less than 5" and would be difficult to see in any case without more detailed techniques. A spawns i,p,v for all sources.

D                 Deferred is for fields that contain problems which we cannot or do not wish to deal with at this stage. Examples are clusters of galaxies with hundreds of (mostly) spurious detections; fields for which SASS aborted the detect routine; etc. D spawns n for all sources.

E                 By extended emission in this context, we mean reasonably strong, and reasonably extended emission which will probably adversely affect the background maps and may compromise source parameters. Examples are clusters of galaxies and SNR. E spawns B.

M                Whenever you find a reasonably convincing source which was missed (no overlaid circle or box) by the detect algorithm, set the M flag. When you do this, you will be asked to click the mouse on the source(s) so that we may have a record of the positions etc.

3.3 SOURCE FLAGS

AUTOMATIC

a               assoc_nn associated with another source
d               det_struc close to an edge, hotspot, or rib
e               extended source is significantly larger than PRF
i                inten_susp intensity is suspect
l                ltc_susp light curve is suspect (PSPC only)
n              not_checked source not checked by visual inspection
p              pos_susp position may be wrong
s              spec_susp spectrum suspect (PSPC only)
u              un_unique source already detected w. smaller cell (HRI only)
v              varflg_susp variability flag suspect

OPERATOR DECISION PRIMARY

c              confused close sources confounded detect algorithm; or complex emission
f               false_det detection believed to be wrong
m             mult_det multiple detections for a single source
p              pos_susp position may be wrong
w             within_ext source is within extended emission

SECONDARY

e             extended source is significantly larger than PRF
i              inten_susp intensity is suspect

c             The detect algorithm has been confounded by the presence of more than one source. The classical case is where detect identifies a position between two close sources of roughly equal intensity. Other examples are when the sources are unequal or there are more than two. In the worst case, the c flag would be invoked together with the f flag, and then there would be missed sources (M field flag). For less extreme cases, the detection is marked with a 'c' only, which will tell the end user that there are more than one source here. 'c' spawns i, p, and v.

f             If, for any reason, you believe the detection has no basis in fact, set the 'f' flag. Examples are spurious detections in extended emission (like SNR) and detections in the wings of the PRF of a very strong source. By applying a s/n threshold, we have tried to avoid the problem of humans deciding on the reality of weak sources. 'f' spawns F.

m          Multiple detection flag gets assigned to all detections of a single source. All detections except for that which best describes the location of the real source (aka "mother") also get the 'f' flag. Since (as an operational convenience) the IDL software automatically sets f to true whenever m is set, this means that the real source must get the flase flag set back to false.

p           Position may be in error. While there are some automatic conditions which set this flag, the operator should set 'p' to true if the overlaid square or circle is obviously not centered on the source.

w           If a discrete source is located within an extended emission region, assign the 'w' flag. Here we are thinking about an AGN within a cluster of galaxies; a pulsar within a SNR; etc. 'w' spawns i.

The SECONDARY flags of interest to the operator will not normally be needed or used, but can be set if, for example, the algorithm for an obviously extended source did not identify the source as extended.

4.0 SAO site specific notes

4.1 MIDAS start

Midas asks for a session number. It seems that even though the prompt suggests that a two digit letter combination is as good as two numbers, when letters are entered, there is some problem with writing to files in the midwork directory.

4.2 Environment

Place the following in your .cshrc.user file

source /soft/midas/exsas/EXSAS.csh
alias vih "/proj/rra/scripts_rra/init/vih"
alias vip "/proj/rra/scripts_rra/init/vip"
alias done_rra "/proj/rra/scripts/done_rra.sh"
alias rra "cd /proj/rra/data"
set path = ($path /proj/rra/scripts_rra)

To run idl standalone, add these lines to your .login file. Then just type "idl".

setenv IDL_VI /proj/rra/hrra/idl_vi
setenv IDL_PATH +"$IDL_VI":+/soft/idl/lib
setenv IDL_STARTUP $IDL_VI/startup --------------------------------------------------------------------

4.3 Selecting Fields to Check

Move to the data directory (e.g. with alias 'rra') and edit the file 'rra_signup', putting your initials after those seqs you plan to VI. Exit the editor. When you finish the session, go back to signup sheet; remove your initials and replace with 'done' for those you finished. If you did not get to some, remove your initials.

4.4 procedure HRI

                                 1. vih
                                 2. visually inspect-exit and save sequence
                                 3. revisit signup sheet

John will do            3. rlogin to linde; su rosat; cd to subdir 'scripts'
this hence-            4. done_rra dirname#1 dirname#2 dirname#3 ...
forth         NB: if you do a bunch of dirnames, you can do steps 3 and 4 altogether

4.5 procedure PSPC

                                1. cd to the data directory where the files are eg, use the alias 'rra' and                                     then cd to the particular subdir containing the data
                                2. vip
                                3. gui/rra
                                4. visually inspect-exit
                                5. type "archive/rra" No longer needed.....
                                6. rlogin to linde and su rosat
                                7. cd /proj/rra/scripts
                                8. done_rra dirname dirname dirname ...

4.6 HRI DISPERSION TEST

From time to time, all checkers (worldwide) will do a few sequences to see how well we conform/agree.

1) sign the log:                       dt/deh                    for 'dispersion test'
2) vih
3) choose seq
4) click on the RESET button, with option 'all'
5) perform VI and save results
6) create an ascii file of your results from within idl. For example,

                     > idl
                     idl>cd,'/proj/rra/data/rh150004n00_vv10000'
                     idl> prthflags,'rh150004n00_qsrc.fits'

7) send the files <seq>_qsrc.txt and <seq>missed.txt (if it exists) to               corcoran@heasrc.gsfc.nasa.gov.
8) in the seq data dir,

                   chmod 777 * (to allow the next user to write files)
9) edit the signup dt/done     deh