12. An OM Data Processing and Analysis Primer

As with EPIC and RGS datasets, many files are associated with an OM dataset. The INDEX.HTM file, and links therein, are viewable with a web browser and will help you navigate the dataset. The different types of files are discussed in Chapter 3.2; however, since the OM is somewhat different from the other instruments on-board XMM-Newton, we will discuss them in more detail in later sections.

The OM can operate in Imaging, Fast, and Grism mode. Each of these modes has dedicated commands to reprocess the data: omichain, omfchain, and omgchain. These are metatasks that each call several procedures that are used to prepare the data for processing, make and apply flatfield images, and detect sources. The tasks omichain and omfchain also calculate the instrumental magnitudes of sources, find the position of the sources (in equatorial coordinates), and produce a sky image; omgchain produces a spectrum. If you run these chains, it is helpful to inspect the sas_log file to get a detailed list of the performed tasks. These chains rely on filters specified by the user; if no arguments are given, they run on all the files present in the ODF directory. Due to the long file names and the large number of input parameters, users are urged to simply use the chains and not run the chains' individual tasks one at a time.

Most OM data are obtained in Imaging mode. If they were obtained in the Fast mode, there will be an additional event list file corresponding to the Fast window (*FAE.FIT). Reprocessing of data taken in Fast mode using the command line and SAS GUI is discussed in §12.3. Reprocessing OM Grism data is discussed in §12.4.

As always, it is strongly recommended that you keep all reprocessed data in its own directory! SAS places output files in whichever directory it is in when a task is called. Throughout this Guide, it is assumed that the Pipeline Processed data are in the PPS directory, the ODF data (with upper case file names, and uncompressed) are in the directory ODF, and the analysis is taking place in the PROC directory. It is also assumed that the data were prepared for processing (see §5). For example data, we will use observations of the Lockman Hole (Obs ID 0123700101; the same as for the EPIC walk-through) for Image mode, Mkn 421 (Obs ID 0411081601) for Fast mode, and BPM 16274 (Obs ID 0125320801) for Grism mode, though any dataset with the appropriate mode will suffice.

12.1 OM Artifacts and General Information

Before proceeding with the pipeline, it is appropriate to discuss the artifacts that often affect OM images. These can affect the accuracy of a measurement by, for example, increasing the background level. Some of these can be seen in Fig. 12.1.

$\bullet$

Stray light - background celestial light is reflected by the OM detector housing onto the center on the OM field of view, producing a circular area of high background. This can also produce looping structures and long streaks.

$\bullet$

Modulo 8 noise - In the raw images, a modulo 8 pattern arises from imperfections in the event centroiding algorithm in the OM electronics. This is removed during image processing.

$\bullet$

Smoke rings - light from bright sources is reflected from the entrance window back on the detector, producing faint rings located radially away from the center of the field of view.

$\bullet$

Out-of-time events - sources with count rates of several tens of counts/sec show a strip of events along the readout direction, corresponding to photons that arrived while the detector was being read out.

Further, artifacts also can contaminate grism data. Due to this mode's complexity, users are urged to be very careful when working with grism data, and should refer to the SOC's website on this topic.

Users should also keep in mind some differences between OM data and X-ray data. Unlike EPIC and RGS, there are no good time intervals (GTIs) in OM data; an entire exposure is either kept or rejected. Also, OM exposures only provide direct energy information when in grism mode, and the flat field response of the detector is assumed to be unity.

For detailed descriptions of PP data nomenclature, file contents, and which tasks can be used to view them, see Tables 3.2 and 3.3.

If you simply want a quick look at your data, sky images and source lists are in *SIMAGE*.FTZ and *SWSRLI*.FTZ, respectively. Further, there are low resolution sky images for each filter; they follow the nomenclature:

PjjjjjjkkkkOMX000RSIMAGbb000.QQQ

jjjjjj - Proposal number
kkkk - Observation ID
b - Filter keyword: B, V, U, M (UVM2), L (UVW1) and S (UVW2)
QQQ - File type (e.g., PNG, FTZ)

So for example, P0123700101OMX000RSIMAGV000.FTZ is the final low resolution sky image in the V filter. To see what files have been summed to make the final image, search for the keyword XPROC0 in the FITS header. For our example image, this would be

XPROC0  = 'ommosaic imagesets=''product/P0123700101OMS004SIMAGE1000.FIT produc&'
CONTINUE  't/P0123700101OMS415SIMAGE1000.FIT product/P0123700101OMS416SIMAGE10&'
CONTINUE  '00.FIT product/P0123700101OMS417SIMAGE1000.FIT product/P0123700101O&'
CONTINUE  'MS418SIMAGE1000.FIT'' mosaicedset=product/P0123700101OMX000RSIMAGV0&'
CONTINUE  '00.FIT exposuremap=no exposure=1000 # (ommosaic-1.11.7) [xmmsas_200&'
CONTINUE  '61026_1802-6.6.0]'

The source list file (*SWSRLI*.FTZ) also contains useful information for the user; the column names are listed in Table 12.1.

Table 12.1: Some of the important columns in the source list file.

Column name	Contents
SRCNUM	Source number
RA	RA of the detected source
DEC	Dec of the detected source
POSERR	Positional uncertainty
RATE	extracted count rate
RATE_ERR	error estimate on the count rate
SIGNIFICANCE	Significance of the detection (in $\sigma$)
MAG	Brightness of the source in magnitude
MAGERR	uncertainty on the magnitude

12.2 Imaging Mode

12.2.1 Rerunning the Pipeline

Please note that calling any of the repipelining tasks will initiate processing on all OM data of that particular mode; currently, only omichain will accept parameters to limit processing to a specific filter or exposure.

To rerun the pipeline on all exposures and filters from the command line, type

omichain

Alternatively, to run the pipeline from the SAS GUI,

1)

Call omichain.

2)

In the task pop-up window, click ``Run''.

This produces numerous files, including images and regions for each exposure and each filter. If we are interested in the sources detected in the mosaicked V band image, we could run omichain with the appropriate flags from the command line typing:

omichain filters=V processmosaicedimages=yes omdetectnsigma=2.0
$$ omdetectminsignificance=3.0

where

filters - list of filters to be processed
processmosaicedimages - process the mosaicked sky images?
omdetectnsigma - number of $\sigma$ above background required for a pixel to be considered part of a source
omdetectminsignificance - minimum significance of a source to be included in the source list file

Alternatively, we can do it with the SAS GUI:

1)

Call omichain.

2)

In the pop-up window, in the ``0'' tab, next to ``filters'', enter V; next to ``omdetectnsigma'', enter 2.0; next to ``omdetectminsignificance'', enter 3.0. In the ``1'' tab, next to ``processmosaicedimages'', enter yes.

3)

Click ``Run''.

The output files can be used immediately for analysis, though users are strongly urged to examine the output for consistancy first (see §12.2.2). The chains apply all necessary corrections, so no further processing or filtering needs to be done. Please note that the chains do not produce output files with exactly the same names as those in the PPS directory (they also produce some files which are not included in the PPS directory at all.)

12.2.2 Verifying the Output

While the output from the chains is ready for analysis, OM does have some peculiarities, as discussed in §12.1. While these usually have only an aesthetic effect, they can also affect source brightness measurements, since they increase the background. In light of this, users are strongly encouraged to verify the consistency of the data prior to analysis. There are a few ways to do this. Users can examine the combined source list with fv, which will let them see if interesting sources have been detected in all the filters where they are visible. Users can also overlay the image source list on to the sky image with implot by downloading the relevant files to the user's local machine. This can also be done with ds9 or gaia by using slconv to change source lists into region files and downloading the relevant files to your local machine. The task slconv allows users to set the regions radii in arcseconds to a constant value or scale them to header keywords, such as RATE. By default, ds9 region files have suffixes of .reg; gaia region files have suffixes of .gaia. In the example below, we make a region file from the source list for the mosaicked, V-band sky image.

To make a ds9 region file from a source list from the command line, type:

slconv srclisttab=P0123700101OMS000RSISWSV.FIT radiusexpression=5
$$ outfileprefix=Vband_mosaic outputstyle=ds9

where

srclisttab - source list file name
radiusexpression - constant or expression (possibly involving keywords) used
$$ to determine the radii of the plotted circles
outputstyle - output format; either ds9 or gaia
outfileprefix - prefix of output file name

The mosaicked, V-band sky image, P0123700101OMS000RSIMAGV.FIT, with the region file from slconv overlayed, is shown in Fig. 12.1. There clearly are spurious detections; these can be removed by hand, or by rerunning omichain with a different background-level threshold or source significance.

Figure 12.1: A mosaicked, V-band sky image and corresponding region file viewed with ds9. Spurious detections are clearly present and can be removed either by hand or by rerunning omichain with the appropriate flags. Some of the artifacts mentioned in §12.1 are also visible.

12.3 Fast Mode

12.3.1 Rerunning the Pipeline

The repipelining task for OM data taken in fast mode is omfchain. It produces images of the detected sources, extracts events related to the sources and the background, and extracts the corresponding light curves. At present, unlike omichain, omfchain does not allow for keywords to specify filters or exposures; calling this task will process all fast mode data.

To run the pipeline on fast mode data on the command line, type

omfchain

Alternatively, omfchain can be run from the GUI:

1)

Call omfchain.

2)

In the task pop-up window, click ``Run''.

There are two types of output files: those that start with f are intermediate images or time series files; those that start with p are products.

To demonstrate some of these output files, we have rerun the pipeline on the example dataset. The processed image in sky-coordinates from one exposure, P0411081601OMS006SIMAGE1000.FIT, is shown in Fig. 12.2 (left). The background-subtracted light curve produced automatically by the task, F0411081601OMS006TIMESR1000.PS, is shown in Fig. 12.2 (right).

The background light curve in Fig. 12.2 (right) is constant because omfchain runs with the parameter bkgfromimage=yes by default, so that the background light curve is found by using the imaging-mode data, instead of the fast-mode window. This is preferable for even only moderately bright sources (count rate $>$ 0.6 ct/s), as the fast-mode window is small and any background measurement that uses it will likely be contaminated with source photons. This is less of a concern if your source is faint, in which case the background can by found from data in the fast-mode window by typing:

omfchain bkgfromimage=no

12.3.2 Verifying the Output

A good first step in checking the output is to examine the light curve plot for both the source and background, making sure they are reasonable: no isolated, unusually high (or low) values, and no frequent drop-outs. Users should also check the image with fv or ds9 in the Fast mode window to see if the source is near an edge. If it is, it's a good idea to examine the light curves from diffent exposures to verify that they are consistent from exposure to exposure (while keeping in mind any intrinsic source variability). If the image is blurred or unusual in any way, users should check the tracking history file to verify the tracking was reliable.

Figure 12.2: Left: The processed Fast mode sky image. Right: the light curve produced automatically by omfchain.

12.4 Grism Analysis

12.4.1 Rerunning the Pipeline

The repipelining task for OM data taken in grism mode is omgchain. It produces images of the detected sources and background, extracts source spectra and region files, and makes source lists and postscript and PDF plots. At present, unlike omichain, omgchain does not allow for keywords to specify filters or exposures; calling this task will process all grism mode data.

To run the pipeline on fast mode data on the command line, type

omgchain

Alternatively, omgchain can be run from the GUI:

1)

Call omgchain.

2)

In the task pop-up window, click ``Run''.

There are two types of output files: those that start with g are intermediate or auxiliary files and source lists; those that start with p are products.

To demonstrate some of these output files, we have rerun the pipeline on the example dataset. The processed image, rotated to align with the columns of the image (p0125320801OMS005RIMAGE0000.FIT), is shown in Fig. 12.3 (left). Two region files are overlayed: p0125320801OMS005REGION0001.ASC, which corresponds to the sources detected in this rotated image (green), and p0125320801OMS005SPCREG0001.ASC, which corresponds to the sources in the spectra list file (red) and indicates the locations of the zero and first orders. The task omgchain automatically extracted the spectrum of the red region (p0125320801OMS005SPECTR0000.FIT); this is shown in Fig. 12.3 (right).

Figure 12.3: Left: The repipelined, rotated image with regions overlayed. Right: the spectrum extracted from the source (red region).

12.4.2 Verifying the Output

The correct correlation of zero and first orders is crucial for grism analysis. Users should inspect the rotated image with fv or ds9 and verify the identification of the orders by overlaying the *SPCREG* region file, as shown in Fig. 12.3 (left); the *SPECLI* file also contains this informtaion. If users are interested in all source detections, the region file can also be overlayed and the full source list examined. Users should also examine the spectra plots automatically produced by omgchain, for both the source and background, making sure they are reasonable. For improved source detection, the parameter nsigma can be changed.