HEADAS help file

NAME

aepipeline - Suzaku XIS/HXD reprocessing tool

SYNOPSIS


aepipeline indir=<Input Directory> outdir=<Output Directory> steminputs=<Stem for input files> entry_stage=<Entry stage> exit_stage=<Exit stage> instrument=<Instrument>

DESCRIPTION

This task duplicates most of the Suzaku pipeline (not trend data). It allows the user to run all or part of the pipeline processing and to vary the calibration files and filtering (screening) criteria used. A number of other pipeline processing parameters can also be changed.

Pipeline Stages

The pipeline is divided into 2 stages:

Calibration
Data screening

Each stage may be run singly or in combination with the preceeding stages. This is controlled by the entry_stage and exit_stage parameters.

HXD Stage 1 consists of the following steps:

Generate a telemetry un-saturated GTI (hxdgtigen)
Run hxdtime
Run hxdpi
Run hxdgrade

XIS Stage 1 consists of the following steps:

Generate a telemetry un-saturated GTI (xisgtigen)
Run xistime
Run xiscoord
Run xisputpixelquality
Run xispi

The data screening (Stage 2) is identical to that in the production Suzaku pipeline, when default parameters are used. For details on the default screening applied to the XIS and HXD events (respectively), see:

To change the screening criteria from the default, use the xis[0-3]_expr, hxd_{pse,gso,pin}_expr, xis[0-3]_grade and xis[0-3]_psum_grade parameters.

Instruments

aepipeline allows the processing of data on an instrument by instrument basis. This is controlled by the instrument parameter. Possible values are:

ALL - Process all data for all modules of all instruments
HXD - Process HXD data (both GSO and PIN) only
GSO - Process HXD data GSO only
PIN - Process HXD data PIN only
XIS - Process all XIS data from all 4 XIS instrument modules - XIS0/XIS1/XIS2/XIS3
XIS0 - Process XIS0 data only
XIS1 - Process XIS1 data only
XIS2 - Process XIS2 data only
XIS3 - Process XIS3 data only

More than one value may be specified at one time.
Combination of GSO and PIN is allowed, is equivalent to HXD.
Combinations of two or more of XIS0, XIS1, XIS2, XIS3 are allowed.
Combination of XIS0, XIS1, XIS2, XIS3 is equivalent to XIS.

INPUT

The input to aepipeline is specified using (at minimum) the indir parameter. This should be specified as the top level sequence directory, e.g.:

aepipeline indir=/path/to/100039010 ...

Paths to specific housekeeping and satellite data files can be specified using the attitude, housekeeping, extended_housekeeping, makefilter, orbit and timfile parameters. But this should NOT be necessary if indir is specified correctly.

NOTE: Errors can occur if a subdirectory of a sequence directory is used for the indir parameter or if the output is stored within the sequence directory.

OUTPUT

Report

An output report is produced with the name "<steminputs>_aepipeline_report.txt". This lists all processing done in any particular run of aepipeline. The production of this report is controlled by the report parameter. (Default = "yes".) The level of detail of this report is controled by the chatter parameter. (Default = "2", range 0 - 5.)

NOTE: Setting chatter to "3" or above can produce very large output reports.

Filenames, etc.

The number of output files depends on both pipeline processing stage(s) and the instrument(s) selected. All output files are written to the directory specified by the outdir parameter. The archive directory structure is NOT reproduced (i.e. all output files will be in a single directory).

The names of files produced are the same as those found in the HEASARC archive. However the usual "aeXXXXXXXXX" prefix, where "XXXXXXXXX" is the sequence number, can be replaced by a character string set by the stemoutputs parameter. This defaults to the value set by the steminputs parameter.

PARAMETERS

indir [string]

Directory containing the input data. This should be the full, or relative path to the top level sequence directory, e.g.:

/path/to/805062010/

NOTE: Problems can arise if a sub-directory of the top level sequence directory is used.

outdir [string]

Output directory used for all output data, as well as the report file. If clobber is not set, and this directory already exists, the task will fail. This should NEVER be a sub-directory of the input directory (the top level sequence directory).

CAUTION: If clobber is set, and this directory already exists, then this task will overwrite files as needed in this directory.

steminputs [string]

Stem for FITS input files, e.g. ae100039010. This string is used as the base filename for finding input files. For example, if steminputs=ae100039010, then to find the attitude file, aepipeline matches the regular expression /ae100039010\.att(\..+)?$/ against all files found in the indir directory.

When doing this matching, aepipeline will prefer files that match the original archive directory structure over files with the same name that do not. For example, if the files:

/path/to/805062010/hxd/event_uf/ae805062010hxd_0_wel_uf.evt

/path/to/805062010/hxd/event_uf_2/ae805062010hxd_0_wel_uf.evt

both exist, aepipeline will use the first file, and not the second. If two files of the same name are found and neither one resides in an archive directory structure, aepipeline will exit with an error.

NOTE: The above file finding issues can be avoided if the downloaded archive data set is left "pristine", by specifying an output directory OUTSIDE of the input directory tree. This is strictly enforced as of V1.1.0 of aepipeline.

entry_stage = 1 [1|2]

Entry stage, 1 or 2.

Stage 1: Re-calibrate unfiltered event files.

Stage 2: Start from existing unfiltered event files.

exit_stage = 2 [1|2]

Exit stage, 1 or 2.

Stage 1: Produces calibrated unfiltered event files.

Stage 2: Produces screened event files.

instrument = ALL [string]

Comma delimited list of which instruments to process. List can include the following values:

ALL - Process all data for all modules of all instruments
HXD - Process HXD data (both GSO and PIN) only
GSO - Process HXD data GSO only
PIN - Process HXD data PIN only
XIS - Process all XIS data from all 4 XIS instrument modules - XIS0/1/2/3
XIS0 - Process XIS0 data only
XIS1 - Process XIS1 data only
XIS2 - Process XIS2 data only
XIS3 - Process XIS3 data only

More than one value may be specified at one time:

Combination of GSO and PIN is allowed, is equivalent to HXD.
Combinations of two or more of XIS0, XIS1, XIS2, XIS3 are allowed.
Combination of XIS0, XIS1, XIS2, XIS3 is equivalent to XIS.

(stemoutputs = DEFAULT) [string]

Base (stem) output name used for creating output files. If set to "DEFAULT", then steminputs will be used.

(clobber = no) [bool]

Clobber existing output files?

(chatter = 2) [int]

Chattiness level.

0 - Essentially nothing will be produced.

5 - Debugging mode with large quantities of output.

NOTE: Setting chatter=5 can generate a huge report file (see report parameter) and a very large amount of terminal output. It is not recommended for general use.

(history = yes) [bool]

Add HISTORY keywords to output FITS files?

(report = yes) [bool]

Produce separate report file for run?

(remove_temp_files = yes) [bool]

Remove temporary files?

NOTE: If set to 'no' and entry_stage is set to 1, roughly 4 to 5 times the disk space will be required for output.

(verify_input = no) [bool]

Run ftverify on input files?

(seed = 7) [int]

Seed for random number generators.

(xisconflist = $ENV{LHEA_DATA}/aexisconf.list) [filename]

XIS discriminator threshold letter code lookup table. This list determines how screened XIS event files are named.

(leapfile = CALDB;$ENV{LHEA_DATA}/leapsec.fits) [filename]

Leap second calibration file.

(attitude = DEFAULT) [filename]

Attitude file. A value of "DEFAULT" will use the "<steminputs>.att" file found in the indir. A value of "EULER" will use fixed Euler angles for the attitude of the satellite (see ea1, ea2, ea3 parameters).

(housekeeping = DEFAULT) [filename]

Housekeeping file. A value of "DEFAULT" will use the "<steminputs>.hk" file found in the indir.

(extended_housekeeping = DEFAULT) [filename]

Extended housekeeping file. A value of "DEFAULT" will use the "<steminputs>.ehk" file found in the indir.

(makefilter = DEFAULT) [filename]

Makefilter file. A value of "DEFAULT" will use the "<steminputs>.mkf" file found in the indir.

(orbit = DEFAULT) [filename]

Orbit file. A value of "DEFAULT" will use the "<steminputs>.orb" file found in the indir.

(timfile = DEFAULT) [filename]

Packets Timing file. A value of "DEFAULT" will use the "<steminputs>.tim" file found in the indir.

(hxd_start = 77604208E+08) [real]

HXD CALDB start time, in seconds. You should not need to change this parameter.

(hxdpi_old = no) [bool]

Use hxdpi_old version instead of hxdpi?

(hxd_gsoght = CALDB) [filename]

HXD GSO gain history table - ONLY used for hxdpi_old.

(hxd_gsogpt = CALDB) [filename]

HXD GSO gain parameter table - ONLY used for hxdpi (current version).

(hxd_gsolin = CALDB) [filename]

HXD GSO linearity table.

(hxd_gsopsd = CALDB) [filename]

PSD selection file.

(hxd_pinghf = CALDB) [filename]

HXD PIN gain history file.

(hxd_pinlin = CALDB) [filename]

HXD PIN linearity table.

(hxd_pinthr = CALDB) [filename]

HXD PIN threshold file.

(WPU = "0123") [string]

WPU board IDs.

(xis_start = 77120000E+08) [real]

XIS CALDB start time, in seconds. You should not need to change this parameter.

(xis0_badcolum = CALDB) [filename]

XIS0 bad column file.

(xis0_calmask = CALDB) [filename]

XIS0 calmask file.

(xis0_makepi = CALDB) [filename]

XIS0 trail file.

(xis0_teldef = CALDB) [filename]

XIS0 telescope definition file.

(xis1_badcolum = CALDB) [filename]

XIS1 bad column file.

(xis1_calmask = CALDB) [filename]

XIS1 calmask file.

(xis1_makepi = CALDB) [filename]

XIS1 trail file.

(xis1_teldef = CALDB) [filename]

XIS1 telescope definition file.

(xis2_badcolum = CALDB) [filename]

XIS2 bad column file.

(xis2_calmask = CALDB) [filename]

XIS2 calmask file.

(xis2_makepi = CALDB) [filename]

XIS2 trail file.

(xis2_teldef = CALDB) [filename]

XIS2 telescope definition file.

(xis3_badcolum = CALDB) [filename]

XIS3 bad column file.

(xis3_calmask = CALDB) [filename]

XIS3 calmask file.

(xis3_makepi = CALDB) [filename]

XIS3 trail file.

(xis3_teldef = CALDB) [filename]

XIS3 telescope definition file.

(anl_profile = yes) [bool]

Enable ANL profiling.

(anl_verbose = -1) [int]

ANL verbosity setting. (-1 for full, 0 for minimal)

(event_freq = 100000) [int]

Event row counting frequency in Suzaku tools.

(num_event = -1) [int]

Number of events to process (-1:all, 0:exit).

(time_convert_mode = 4) [int]

HxdTime2aetime mode.: 1, 2, 3, 4.

1: use internal function in interpolation (TI to aetime).

2: use telemetry aetime, and simply add sub_aetime.

3: use telemetry aetime, and interpolate event aetime

4: use astetool, aste_ti2time() with tim FITS file.

(use_pwh_mode = no) [bool]

Use HXD_WEL_PWH column in HK FITS or not.

(bstgti = no) [bool]

Tool Specific Parameter - XISTIME.

XISTIME - Generate GTI for the XIS burst option without approximation?

(pointing = KEY) [KEY|USER]

Tool Specific Parameter - XISCOORD.

XISCOORD - Pointing type, KEY/USER.

(ea1 = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - 1st XYZ-Euler angle (deg), used when attitude=EULER.

(ea2 = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - 2nd XYZ-Euler angle (deg), used when attitude=EULER.

(ea3 = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - 3rd XYZ-Euler angle (deg), used when attitude=EULER.

(ref_alpha = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - R.A. of the sky reference position, used when pointing=USER.

(ref_delta = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - DEC. of the sky reference position, used when pointing=USER.

(ref_roll = 0.0) [real]

Tool Specific Parameter - XISCOORD.

XISCOORD - Roll angle of the sky reference, used when pointing=USER.

(aberration = no) [bool]

Tool Specific Parameter - XISCOORD.

XISCOORD - Correct aberration.

(enable_scipixq = yes) [bool]

Tool Specific Parameter - XISPUTPIXELQUALITY.

XISPUTPIXELQUALITY - Flag to enable SCI pixel quality bits.

(enable_trcor = yes) [bool]