filename: oper.man
Date:	26 Feb 95
orig:	deh
tlm:	29 Jun 95

ROSAT RESULTS ARCHIVE OPERATOR MANUAL

General
Flags
Operator's Instructions

1.0 General

During the summer of 1994, the ROSAT consortium (MPE, Leicester/UK, GSFC, and SAO) agreed that a significant effort was warranted to produce a reliable source catalog from the SASS processing of pointed observations. At the same time, it was felt that the source lists produced by SASS should be available in the public archive.

Since SASS processing occasionally produced unreliable results, and since it was known that there were certain problems that SASS did not attempt to solve (i.e. aspect differences for obis separated by 6 months), it was decided that the SASS source tables from the reprocessing (when the switch to RDF files occurred) would form the basis of the catalogue, and, that quality control (QC) flags would be attached to key source and field parameters so that when the SASS results passed into the public domain, they would already have these QC flags.

Since there was a certain urgency felt, it was decided to deal with the straightforward fields first since they should present the least amount of problems. Therefore the 'RRA Project' came into being by mutual consent and discussion and has, as its primary goal, the production of a high quality source catalog. Phase Zero of the project was prototyping software to generate the QC flags. This essentially took the fall of 1994. Phase 1 is meant to mean the generation of the easy part of the catalog, and is scheduled to start in summer 1995. Phase 2 will overlap the ongoing parts of phase 1, and is meant to deal with the more vexing problems.

In a sense, the RRA project is the construction of a pipeline (involving human participation as one of its elements) which takes the SASS as its input, and produces entries in a database, which, in the first instance, will be the well known 'browse' system.

2.0 Flags

Early in the project, it was decided to have true flags, not status indicators. This means that a flag is either true or false, it cannot have other values. Furthermore, we decided that all flags would be 'false' if everything was all right. Although this may seem counter-intuitive at first, the committee had the sense that the key questions could be phrased: "Is the position produced by SASS suspect?"

For obvious reasons, there are two sorts of flags: those which pertain to individual sources, and those which describe conditions of the whole field.


NB: ***	The operator is referred to other documents for details:**********

flags.sum	is a summary of all flags.  It should be viewed on a wide
		screen, and printed landscape with smallish type.

flagset.hri	The detailed specification of HRI flags, and other matters.
***************************************************************************

2.1 Rationale for flags

The QC flags will serve many functions. They will serve as guides for views of the database. The most common use of the catalog will be via electronic means since this source list, unlike the Einstein 2E catalog, will not be complete until some undetermined date in the future. Normally, users will not want to retrieve sources that have been identified as 'false sources' (source 'f' flag set to 'true'). The flags will also alert users to certain situations about the environment since most users will not have access to the image.

2.2 Automatic vs. human decision

Many of the flags can be set directly by considering the values of SASS output parameters (e.g. if the s/n is below some threshold value). Some flags may be dependent on several other flags, and some are set exclusively by visual inspection (VI).

Therefore, there are three logical steps in the QC process. In step 1 we do everything which can be programmed. This includes setting the automatic flags on the basis of SASS output, converting the RDF fits files to system files, and generating any required new files (e.g. a large smoothed image of the central region of the HRI field).

In step 2, the VI operator examines the field and sets any required field and source flags. These in turn, may set other flags (logical step 3). In practice, step 3 operations are meshed to step 2 procedures and thus do not appear as a separate process.

2.3 Extraneous matters

The field flag M is set whenever the operator finds that an obvious source has been missed by the detection algorithm. It was decided that the VI operator should identify the source(s) by clicking on the source, thereby recording the RA, DEC, and a few other rough source parameters. It was felt that the additional time required for this process was a worthwhile investment, and that the tables of missed sources would not, in the first instance, be a part of the RRA, but be a 'common property' of the consortium. In phase 2, it may be possible to get some "SASS equivalent" parameters by giving the rough positions to the ML algorithm. Another set of quantities produced by the RRA project will be the statistics of flags set, and these numbers will be kept.

3.0 Operator's Instructions

We assume that the preparatory tasks ('step 1') will be done by cron jobs. Here we attach a suggested procedure, and eventually some logistical information for our local site (SAO). Thus the other sites can replace sections 3.4 and 3.5 if they differ substantially. The user interface has been designed to provide rapid display and easy input of decisions. Please bring to our attention suggested improvements.

The MIDAS/EXSAS system for the PSPC is described in the 'document_rra.ps' by Boller, and the operator should consult that for the mechanics of operation. Likewise, the IDL/HRI system is described by Corcoran's 'hvi2users_guide.txt'. Here we discuss the decision making process.

3.1 Suggested Procedure

The field is shown with overlayed source circles (squares) for the PSPC (HRI). In the former case, the circle approximates the size of the PRF; in the latter, the size of the detect cell. It is often the case that one may fiddle the color look-up table and cutoff levels at the beginning of a session (deh sets the LUT to black and white in his IDL startup file).

Here we draw your attention to the primary and secondary flags you should evaluate, and list the automatic ones for reference. Picture examples for each condition appear below.

We recommend first examining the field, setting any of the four field flags listed in 3.2. In practice, the D (deferred) and A (bad aspect) will be very rare, and hence need close to zero operator time. This leaves E (strong, extended emission which will adversely affect the background) and M (missed sources). Finding missed sources should be done carefully at this stage so that later source flagging is easier. It should not take long if the overlays for low s/n sources make them clearly visible.

After setting E and M, we scan the sources to note which might require flag setting and further examination. This can best be done with the zoom feature (IDL/HRI) or the 'Quick Mode' (MIDAS/PSPC). For the HRI, one can use both windows...the full field for the outer regions, and the smoothed blowup with 2" pixels for the central region.

After finishing with the field flags one can click on the sources needing flagging. For the IDL system, it is possible to have the zoom feature operational in one display and click on (thereby 'selecting') particular sources in the other window. Only 5 field flags need be considered by the operator. In practice, f, p, and w should be very quick and obvious (see section 3.3, below). However, please note the difference between 'c' which means the algorithm was confused - there are more than one source; and 'm' which means there have been multiple detections of only one source.

3.2 FIELD FLAGS


AUTOMATIC

B	bkg_susp	background map suspect
F	fals_det	one or more sources were given the f(alse) flag
S	slist_mis	empty or missing source list

OPERATOR DECISION

A	asp_susp	aspect solution may be bad
D	deferred	field deferred
E	ext_emis	extended emission in field
M	src_mis		one or more sources missed by detect

A (ASPECT SUSPECT): The common forms of aspect error are either registration so bad that every source appears twice, with a fixed offset of any size; or a poor registration of obis which leads to smearing of the image by 5" to 15" (typically). The former case is extremely rare, and in rev2 processing, we expect that the latter case will also be rare. In the HRI, if a known point source is elliptical, or if all sources near the field center have the same ellipticity, bad aspect is probably the cause. However, we do not intend to flag bad aspect which results in egg-shaped HRI sources from residual wobble problems. This is normally less than 5" and would be difficult to see in any case without more detailed techniques.; Subordinate Flags:A spawns i,p,v for all sources.
D (DEFERRED): Deferred is for fields that contain problems which we cannot or do not wish to deal with at this stage. Examples are clusters of galaxies with hundreds of (mostly) spurious detections; fields for which SASS aborted the detect routine; etc.; Subordinate Flags:D spawns n for all sources.
E (EXTENDED EMISSION IN FIELD): By extended emission in this context, we mean reasonably strong, and reasonably extended emission which will probably adversely affect the background maps and may compromise source parameters. Examples are clusters of galaxies and SNR.; Subordinate Flags:E spawns B.
M (MISSED SOURCES PRESENT): Whenever you find a reasonably convincing source which was missed (no overlaid circle or box) by the detect algorithm, set the M flag. When you do this, you will be asked to click the mouse on the source(s) so that we may have a record of the positions etc.; Subordinate Flags: NONE

3.3 SOURCE FLAGS


AUTOMATIC

a	assoc_nn	associated with another source
d	det_struc	close to an edge, hotspot, or rib
e	extended	source is significantly larger than PRF
i	inten_susp	intensity is suspect
l	ltc_susp	light curve is suspect (PSPC only)
n	not_checked	source not checked by visual inspection
p	pos_susp	position may be wrong
s	spec_susp	spectrum suspect (PSPC only)
u	un_unique	source already detected w. smaller cell (HRI only)
v	varflg_susp	variability flag suspect 

OPERATOR DECISION

PRIMARY
c	confused	close sources confounded detect algorithm
f	false_det	detection believed to be wrong
m	mult_det	multiple detections for a single source
p	pos_susp	position may be wrong
w	within_ext	source is within extended emission

SECONDARY

e	extended	source is significantly larger than PRF
i	inten_susp	intensity is suspect

c (confused): The detect algorithm has been confounded by the presence of more than one source. The classical case is where detect identifies a position between two close sources of roughly equal intensity. Other examples are when the sources are unequal or there are more than two. In the worst case, the c flag would be invoked together with the f flag, and then there would be missed sources (M field flag). For less extreme cases, the detection is marked with a 'c' only, which will tell the end user that there are more than one source here.; Subordinate Flags: 'c'spawns i, p, and v.; HRI Example of source with 'c' flag = 'T'
f (false): If, for any reason, you believe the detection has no basis in fact, set the 'f' flag. Examples are spurious detections in extended emission (like SNR) and detections in the wings of the PRF of a very strong source. By applying a s/n threshold, we have tried to avoid the problem of humans deciding on the reality of weak sources.; Subordinate Flags: 'f' (source flag) spawns F (field flag).; HRI Example of source with 'f' flag = 'T'; Source 69 was detected by the processing system but is not real.
m (multiple detection): Multiple detection flag gets assigned to all detections of a single source.; Subordinate Flags: All detections except for that which best describes the location of the real source (aka "mother") also get the 'f' flag.; HRI Example of source with 'm' flag = 'T'; Source 1 (LMC X-1) is very bright. The wings of the psf were detected as spurious sources and have their 'm' flags set to 'T'
p (position suspect): Position may be in error. While there are some automatic conditions which set this flag, the operator should set 'p' to true if the overlaid square or circle is obviously not centered on the source.; Subordinate Flags: i; HRI Example of source with 'p' flag = 'T'
w (within extended emission): If a discrete source is located within an extended emission region, assign the 'w' flag. Here we are thinking about an AGN within a cluster of galaxies; a pulsar within a SNR; etc.; Subordinate Flags: 'w' spawns i.; HRI Example of source with 'w' flag = 'T'
d (detector structure): Source near a detector structure. For the HRI this means that the source is near the edge of the detector or near a hotspot. For the PSPC this means the source is near a rib or near the edge of the detector.; Subordinate Flags: 'd' spawn i, l, p, v; HRI Example of source with 'd' flag = 'T'
e (extended source): Source appears non-point-like. For the HRI, all sources with SIZ_COR > 1.2 have the extended flag set to 'T'; Subordinate Flags: 'e' spawns i, l, p, v; HRI Example of source with 'e' flag = 'T'
n (not checked): The source has not been checked by the V&V team. For the HRI, sources with S/N < 4 are ignored.; Subordinate Flags: NONE; HRI Example of source with 'n' flag = 'T'
a (associated source): A source which has a nearby (but resolved) neighbor has the associated flag set to True. For the HRI the ASSOCIATED flag is set to True if either of 2 cases hold: 1) the distance between that source and any neighbor is less than the sum of twice the 50% point spread function radius for the source and the neighbor; or 2) if SIZCOR > 1.2 for the neighbor and the distance between the source and the center of the extended circle for the neighbor is less than the radius of the extended circle plus twice the 50% radius of the psf of the source.; Subordinate Flags: i, l, v; HRI Example of source with 'a' flag = 'T'.; Source 55 is one of a number of sources with the 'a' flag = 'T' in this M31 field.
u (source is not unique) (HRI ONLY): The HRI processing software runs a source detection using detect cells of various sizes. The UN_UNIQUE flag is set to True if the source was matched with another source with a smaller detect cell size.; Subordinate Flags: NONE; HRI Example of source with u flag = 'T'.; Source 53 is the outermost detect cell associated with the bright source at the center of the 4 detect cells.
i (intensity suspect): The derived source count rate may be in error.; Subordinate Flags: NONE
l (lightcurve suspect): The derived source lightcurve may be in error. Note that for the HRI the source lightcurve is NOT output to the user.; Subordinate Flags: NONE
v (variability flag is suspect): The derived source variability flag may be in error.; Subordinate Flags: NONE

The SECONDARY flags of interest to the operator will not normally be needed or used, but can be set if, for example, the algorithm for an obviously extended source did not identify the source as extended.