seextrct -- creates a light curve and/or spectrum from XTE Science Event (SE) data from the PCA and HEXTE instruments.
seextrct infile lcbinarray gtiarray maxmiss gtiorfile gtiandfile
outroot extenpha extenlc phasefile bitfile timecol columns multiple
binsz mfracexp tnull timezero printmode lcmode spmode mlcinten
mspinten timemin timemax timeint gticols chmin chmax chint chbin ephem
period phaseint obsdate obstime sensecase chkit clobber negative
bailout mode
This task creates a single light curve and/or spectrum from one or more Science Event files, allowing options for filtering by time using FITS and/or ASCII Good Time Interval (GTI) information, by detector and layer (where available), by channel, and by phase. It takes as its input a standard Level 0 XTE/SE FITS file, or list of such files, and creates as output standard Level 2 spectral and/or light curve files in OGIP standard formats.
Internally, the task bins a requested portion of a vector column. Selection is based on (optional) GTI files or other applied filters. The binning can be specified for the entire file, or for a subset of rows that fulfill the filtering criteria..
The binning has five modes. If the bin mode ('lcmode' or 'spmode') = 'SUM' or 'EVENT_SUM', the output is the total counts -- this is the appropriate default for energy spectra. If the bin mode = 'RATE' or 'EVENT_RATE', the output is the total of the value of the counts divided by the time binsize -- this is the appropriate default for light curves. If the bin mode = 'MEAN' then the output is the mean value of the binned quantity during each time interval -- for housekeeping information or other telemetry values, this would be an appropriate default.
In general this parameter should be left at its default value. However, it is supplied to allow the expert user to tune the extractor based upon their individual requirements. The larger the size of the light bin array, the fewer cycles are necessary to process large files. However, there may be a runtime cost if your system lacks the RAM for the size of array you have defined. Leave this value alone unless you have the requisite computing knowledge.
If used correctly this parameter can speed up processing considerably; e.g. for XTE PCA data, MAXMISS=200 is an effective choice. However, use of this parameter assumes that (i) your data is time-ordered in ascending order and (ii) no input files have overlapping time ranges. XTE data generally fulfill these requirements.
If more than one file is supplied, the files will be OR'ed together but are specific for each input file.
If this parameter is set to APPLY - the default - then the GTI information found in the 2nd extension is used, however, if the input files contain more than 1 GTI, and the GTI information is contained in e.g. the 3rd extension (the 2nd GTI in the file), then the user can use the APPLYnnn option. If this parameter is set to e.g. APPLY003, the code will automatically attempt to apply any GTI information found in the 3rd extension of the input files.
Note that in general only the GTI's included with each file being input should be used; hence the default value of "APPLY".
Multiple GTI files can be specified at the GTIANDFILE parameter by entering their names in an ASCII file, and specifying this ASCII file in response to the prompt ("@gtiandfiles"). Multiple files will be sorted and ANDed together. Up to 100 GTIANDFILEs can be accepted. The good time intervals within each file are assumed to be in ascending time order.
If time intervals are supplied using the GTIANDFILE and also the TIMEINT parameter, they will be ANDed to arrive at a global set of time intervals to be applied to all input files, although users who mix the two approaches are warned to take care to take into account the TIMEZERO keyword. The TIMEZERO keyword in the GTIANDFILES are added to the specified START and STOP times (as well as to the TSTART and TSTOP keyword) to calculate the absolute times. Also, the time-ranges in the GTIANDFILES are expected to lie between the specified TSTART and TSTOP values. If the above seems confusing, it may be safer to use either the GTIANDFILE approach _or_ the TIMEINT approach, but avoid combining them.
SEEXTRCT can only process bitmask files of the general form:
Event == b1xxxxxxxxxxxxxxxxxxxxxxx &&
Event == bxxxxxxxx1xxxxxxxxxxxxxxx &&
Event <= bxx001xxxxxxxxxxxxxxxxxxx &&
Event >= bxxxxxx11xxxxxxxxxxxxxxxx
SEEXTRCT does not support NOT (!) signs, OR (|) signs, or parentheses, since those are used to denote a hierarchy for the comparisons. Also, SEEXTRCT does not understand FORTRAN comparitors, so you MUST use C-like constructions. All input files must have the same data descriptors.
Note that the bit-mask is not only instrument dependent but also file dependent, and the extractor cannot check to ensure that the input is meaningful. User caution is recommended. For further information about the use of bitmasks, please study the relevant 'recipes' on the XTE GOF webpages, particularly those under the title "Screening and Selection".
Use of this parameter is recommended for those cases where the required binsize is small, and to specify it precisely in seconds would require digits beyond the sixth decimal place, which the extractors' timing routines cannot support due to the intrinsic limitations of computer arithmetic. (Keep in mind that XTE timing resolutions are really an inverse power of 2; eg., Good Xenon data has a timing resolution of 2^-20 seconds.) See the BINSZ parameter discussion for further details on binning resolution.
Use decimal expressions for sub-second increments, e.g. 0.004 for 4 milliseconds. If the supplied BINSZ is less than the minimum time resolution of the first file then the binsize will be reset to that time resolution. If subsequent files have larger minimum time resolutions, they will be skipped and not processed. Bin-sizes have a minimum resolution of 0.000001 seconds; anything smaller can cause problems both in accuracy and resolution due to the intrinsic limitations of computer arithmetic.
If BINSZ=INDEF (not recommended!), 100 evenly sized bins will be produced with a binsize of (TIMEMAX - TIMEMIN)/100.0.
Users needing fine timing resolution in their binning can set MULTIPLE=yes. If MULTIPLE=yes, the BINSZ parameter will be interpreted as an integer multiplier of the intrinsic timing resolution of the data (e.g. to rebin Binned Mode data up at precisely 4 times the intrinsic resolution of the data, users should supply "MULTIPLE=yes" and "BINSZ=4").
To address the situation when the user wants high-accuracy and does not plan on processing any produced data files with Xronos, the user can specify a TIMEZERO value of his liking, with TSTART and TSTOP shifted by the same TIMEZERO amount. The numerical value specified in TIMEZERO is globally subtracted from all time-stamps, TSTART, and TSTOP keywords. Alternatively TIMEZERO may be set to FLOAT, which will cause the first time-stamp in the output light-curve to be 0.0d0 and TIMEZERI and TIMEZERF to contain the value of the 0.0 timestamp.
Note that negative values for timestamps and keywords can occur if care is not taken when choosing an appropriate value for TIMEZERO.
If TIMEMIN is set to INDEF, all time intervals are included in creating the selected output.
See the usage notes supplied for the TIMEMIN parameter for further information. If TIMEMAX is set to INDEF, all time intervals are included in creating the selected output.
(The FTOOLS script TIMETRANS may be used to take ranges of times and convert them into absolute time, and create an ASCII file which can then be input into the extractor at the TIMEINT prompt using "@ascii_filename_containing_times".)
The TIMEINT file is an ASCII file containing pairs of start and stop times, one pair per line, with no dashes or other non-numerical characters.
If no CPIX information is found, a one-to-one correspondence is assumed between the bin channel and the energy detector channel, with the first channel in the file assumed to correspond to energy channel 0, and the Nth channel in the file is set to correspond to the energy channel (N-1).
Use the @filename option if you have an ASCII file containing Start_Chan Stop_Chan pairs. Such files may be created by the FTOOLS script CHANTRANS.
When analyzing data from binned modes, it is recommended that the boundaries supplied using CHINT line up with the energy channel boundaries specified in the input files processed. Otherwise, unexpected results may occur due to the intrinsic channel resolution of the data processed. For example, if CHINT specifies that original energy channels 150-200 be included but the energy binning of the file (specified by CPIX) is 0-155, 156-180, 180-255 then data from _all_ channels will be included in the PHA file -- presumably not the user's intent.
NB: The EPHEM, PERIOD and PHASEINT parameters are largely historical relics; we advise the use of the specific time intervals for cases where the cycle time is similar to the observation length (e.g. binary orbital periods), and the use of the FASEBIN suite of tools for cases where the cycle time is much shorter than the observation length (e.g. pulsars).
The code has been structured such that 1 billion light curve bins can presently be accumulated. If more than that are required, changing the parameter "itcycles" from 10,000 to something higher and recompiling will increase the size. However, no current FTOOLS can cope with such a large number of light curve bins.
Dates used must all be of a consistent system. MJD is used for all dates, TIMESYS is used to determine the time system in use and is automatically converted into seconds before processing begins.
Many small bug fixes have been made, such that one should acquire the absolute latest version before beginning processing of any large files. It is possible that a "bug" identified below does not exist in the latest version.
MAXMISS added, to improve processing speed when in a "cycling" mode - i.e., the user has specified a bin-size which results in more than 100K light-bins in the light curve. See information above.
The codes have been made truly case-insensitive but they will preserve the case of the columns as they are stored within the original files.
Previous warning messages that could cause the code to fail down stream have been made into termination errors. This will make it so that after the message about the offending behavior, the code will abort without attempting to "fix" the problem.
The code now will automatically attempt to APPLY the gti information that is contained in the second extension of XTE data files. This is called by setting GTIORFILE to APPLY. The user can override this by setting GTIORFILE to "-".
Known problems in this version:
A phase-interval problem. At the moment GTI's are detected by blank regions in the output, so that when the codes generate the GTI to append to the light curve or spectrum it does so by looking for blanks in the time-line. The maximum number allowed is 12800 at the moment, but if a user uses a small bin-size and a large period and filters on multiple phase intervals it is easy to over-flow this 12800 maximum. At the moment the only thing to do is for the user to divide the (TIMEMAX - TIMEMIN / PERIOD ) * number of phase intervals and ensure that this does not exceed 12000 or so. The code will abort if it does since that will cause a boundary violation. This is being worked on so that if the user specify phase filtering a "PHASE FILE" will be output in the form that Xronos expects and the GTI will no longer include such phase filtering information.
A minimum bin-size problem. Unfortunately, since double precision values only contain 16 digits of accurate information (see the IEEE standard on accuracy) and at the moment 8 digits are used to specify the number of seconds for MET, we are only left with 8 digits for the decimal values - thus the extractors are only accurate to 1.0D-8. Unfortunately, some of the files specify the timing resolution of the satellite beyond 8 digits (although the time-stamps can only show that level of accuracy, e.g., 6.554523228973598E+07 or 65545232.28973598 so anything after the ...98 is inaccurate) can cause problem when processing the data. Thus, rounding errors can occur. To avoid this, we recommend the user does not specify a bin-size to an accuracy greater than 1.0D-8.
2) The output light curve has been "shifted" so that the timestamps give the leading edge of the light-bin, rather than the center of the light bin as it did in the past. This change is reflected in the change in the TIMEPIXR keyword from 0.5 to 0.0. This change was made since Xronos no longer needs to have the light-bin's timestamp given at the center of the bin. Note that Xronos cannot accurately processed raw XTE data since it does not incorporate the TIMEZERO keyword in the same way as the XTE files. (See below for more information.)
3) In version 3.5.1 the usage of TIMEZERO was made to obey the standard XTE convention, i.e., that TIMEZERO must be applied not only to the TIME column but also to TSTART and TSTOP values, as recommended by the FITS working group. This causes an incompatibility with Xronos, which adds TIMEZERO to the TIME column, but NOT to the TSTART and TSTOP keywords. Thus, a "timezero" parameter was added to SA(SE)EXTRCT which can have any value (INDEF is the default and will set TIMEZERO = 0.0d0) and will be subtracted from the actual timestamps. If "timezero" is set equal to "FLOAT" then the value of TIMEZERO will "float" to the earliest value such that the first timestamp in the light-curve will be 0.0d0, and TSTART will also be 0.0d0 (since TIMEZERO must be added to TSTART as well as the timestamp).
4) SEEXTRCT had its timing algorithm completely rewritten, such that it should be faster and more robust than previous versions of the science event extractor.
5) ALL GTIANDFILEs and TIMEINT values are "merged" together to form one large GTI filter.
6) All GTI files input as GTIANDFILES and GTIORFILES are now pre-filtered and compared against the TSTART and TSTOP values given in the file. It was discovered that some users were inputting GTI ranges well outside of the TSTART and TSTOP (+ TIMEZERO) time-ranges, causing unexpected problems in calculating the valid ONTIME for science event data. This was solved by pre-filtering the GTI file based upon the TSTART and TSTOP keywords given in that extension. Basically, the codes attempt to make allowance for common user errors.
7) CHINT and CHBIN can now understand channels specified in the same format as the CPIX keyword, allowing the user to "copy" the pertinent parts of the CPIX keyword and input that into the extractors. This will rebin or channel filter the original data based upon these specifications. See the appropriate parameters for more information.
Version SEEXTRCT_3.6:
1) A problem that occurred for GTIANDFILES that contained only 1 row has been corrected, as well as now checking for the case where the user supplies a GTIANDFILE or TIMEINT file which contains NO DATA. Both of these cases would cause all of the data points in the files being processed to be filtered out.
2) Additional information is now written to the screen describing the GTI information that the code is supplied with from the user. This was necessary as some GTI's were correct such that the code was using non-applicable time-ranges when filtering data. This can give rise to serious problems for SEEXTRCT. Also, the TSTART+TIMEZERO and TSTOP+TIMEZERO specified in the GTI's are now rigidly enforced - these keywords should accurately describe the information in the extension, in the past users this was not the case and caused serious problems for the extractors under certain conditions.
3) If "no data" is selected the code will exit properly now.
4) The timing algorithm has been reworked such that on-time is now calculated by calculating the amount of valid time that is between the TSTART+TIMEZERO and TSTOP+TIMEZERO minus all of the filtered out time-regions. As a result of this the code is MUCH more sensitive to errors in the GTI's that are input. To compensate for this sensitivity the code performs many more checks on the GTI's to attempt to ensure that they were created in a valid manner. It is still possible to confuse the code if the user modifies such parameters as TSTART and TSTOP in the GTI file, removes them, or creates them in an unsanctioned manner, i.e., a way not listed in the ABC guide for data analysis. The code assumes that all input GTI's are valid, so it is up to the user to ensure that any forced modifications performed to the GTI's result in valid XTE GTI files. As long as the information input is valid and conforms to the XTE specifications, no problems should be encounted.
2) A problem with using TIMEMIN and TIMEMAX with GTI files which fell way outside of the TIMEMIN and TIMEMAX has been fixed.
3) Several redundant keywords, or incorrectly described ones have been removed, or fixed.
4) The timing algorithm has been re-worked to correctly allow multiple GTI's per light-bin, as well as a few other things. Originally, this was believed to be an occurance that users would not see with valid data, but the new HEXTE tools make this occur with regularity.
Please report problems to xtehelp@athena.gsfc.nasa.gov.
SAEXTRCT, CHANTRANS, TIMETRANS, SEPLOT, SESELECT, SEFILTER