This recipe describes how to reduce HEXTE archive or event mode data using the HEXTE tools hxtback and hxtdead released with FTOOLS v4.1. These tools now apply to both archive and event mode data but still require housekeeping files which are not included in realtime data. For this reason, there remains a section on how to do a simple, first order correction to the background for realtime data.
Introductory notes
- Files: As received by the guest observer, RXTE data are organized by observation
and observation segment. All of the hexte files necessary for these basic
steps are in the lowest-level directory called hexte. The HEXTE clusters
appear as separate instruments in the data stream, with file prefixes as in
the table below.
Each file name starts with the letter "F" and is followed by "S" for science
or by "H" for housekeeping. Next comes a two-digit hex number labeling the
data contents, and finally two 7-digit hex numbers giving the start and stop
times. Data of each type can appear in more than one file, but this is not
usual.
| Data Type | Cluster A (Cl0) | Cluster B(Cl1) |
|---|
| Science data | FS50... | FS56... |
| Standard Modes (Archive) data | FS52... | FS58... |
| Calibration data | FS54... | FS55... |
| 16s Housekeeping | FH53... | FH59... |
| 128s Housekeeping | FHfd... | FHfe... |
In addition to these, the user will require the RXTE spacecraft attitude data
which can be found both in the XTE Filter File, and in the raw attitude file
(FH0e...).
- Modes: The Science data files (FS50.., FS56..) contain HEXTE data in the telemetry
format selected by the user, either Science Event data (SE), or Science
Array (Binned) SA data (which includes Spectral Bin or Temporal Bin modes).
Can't remember which Science Mode was used? Use the Ftool 'fkeyprint' to
look at the value of the DATAMODE keyword. Names beginning with "E_" are SE,
names beginning with "B_" are binned (SA) data, either Spectral Bin or
Temporal Bin. Note that cluster A and B may have been configured (by
choice!) in different Science Modes, so check both FS50.. and FS56.. files
if you are uncertain.
For a more complete picture of the observing modes (now including the
HEXTE rocking and lower energy bounds) users should also consult the
appropriate OBSCAT for their observation.
The Standard Modes files contain two data products produced
automatically for all observations: Archive Spectral Bin (64-bin spectra
produced every 16s), and Archive Temporal Bin (4-band light curves with 1s
bins).
More information on the Science Modes and the Standard Modes data products
may be found in the HEXTE chapters of the proposal documentation.
- Configurations: The rocking behavior of the two HEXTE clusters can be selected by the proposer among a set of times and angles. By default, the two clusters rock alternately to a total of four offset positions, so each goes to a positive offset, then on source, then the negative offset, then back on, etc. While one cluster is on source, the other is offset. To determine if there is a confusing source in any of these fields, the Web tool HEXTErock will plot the offset positions on the sky and any ASM sources in the field.
The default configuration for HEXTE's rocking motion before UT 1998 Jan 9 00:00 was 16 second intervals of on-source pointing alternated with 16 seconds of slews and dwell for the off-source pointings. After this date, the default was changed to 32 second intervals. The default offset angle has remained 1.5 degrees. If you are unsure of the HEXTE configuration for a particular observation, you can find the rock and dwell modes in the 16 second housekeeping data for each cluster. These can be found through XDF or by looking for the FH53 (Cluster 0) and FH59 (Cluster 1) files in the hexte subsystem directory. The columns for the dwell time and rocking angle are cmdDwellApMod and cmdPosApMod, respectively. The information is encoded in an integer value which corresponds to a given mode according to this look-up table
- Changes:
One of the detectors (2) on Cluster 1 has lost its spectral capabilities. (For more information, see the HEXTE page on the subject.) This does not affect analysis of light curves; the detector retains its photon counting capabilities. For spectral analysis, however, its data are incorrect. In fact, this detector dumps all events into the first three PHA channels. When extracting light curves, you should in any case exclude those channels below the LLD (i.e. channels < 15) which will also exclude all the detector 2 events. Likewise, when you look at the full spectrum, your analysis should ignore these channels.
As of UT 1998 Jan 9 00:00, a lower energy bound of 10 keV was implemented as the default. For more information about the HEXTE status, please see the HEXTE status report at http://mamacass.ucsd.edu:8080/hexte/status/hexte_status_980109.html.
Separating background from source+background in raw FITS files
If the HEXTE cluster was configured for source/background beamswitching
observation (the default is a 32 second "rocking" configuration), users must separate the on- and off-source data
before accumulating spectra and/or light curves.
This step is the same for both archive and event modes.
- The script hxtback will produce one, two or three files for the
input Science file. These consist of the original data selected by rocking
position. All the data which is source-looking will appear in one file
whose name begins with the original data file name followed by "_src". If two-sided rocking was selected (as in the default configuration), two background files are created
with "_p" and "_m" suffixes, for "plus" and "minus" positions respectively.
Alternatively, users may elect to keep all the background data in a single
file (if you are not concerned about any possible nearby source or other asymmetry in the background pointings), by using the -b option, in which case the output file is appended
"_bkg". Note that one may add the two final background spectra back together
later using sumpha, after the dead-time
corrections has been performed (see below).
In addition to separating the on- and off-source Science Events,
hxtback also produces a good-time interval (GTI) extension
which specifies in full detail the time intervals on/off-source. This
information is essential for the extractor programs to calculate the correct
raw exposure time, which is later corrected to live time by
hxtdead. You may also look at this to determine the time interval of the rocking which in turn will tell you how to bin the light curve.
- The calling sequence is
hxtback -h
for a full list of options. For our Cluster 0(A) example:
hxtback -b -i FS50_xxx
You will see output which includes something like the following:
running fdump to get the GTI Extension header . . .
cluster string: ClstrPosition == 1 || ClstrPosition == 65 ||
ClstrPosition == 8 || ClstrPosition == 72
fselect for cluster position all source...
where the tool has determined the values of ClstrPosition corresponding to an on-source pointing and run fselect with that selection expression; it will use another expression for the background pointings.
- Run this tool for each of your FITS data files. It may take quite a while
for it to go through all the data.
Extracting light curves and spectra
What you need to know about binsizes: read even for SPECTRAL analysis !
The deadtime correction algorithm uses the GTI extensions of the extracted light curves and spectra. These extensions will depend on the chosen binsize, even for spectra, since the extractor's algorithm for creating them is the same regardless of which printmode you choose. The individual good time intervals cannot be smaller than the specified binsize, as each bin must be completely within good time, by definition. This means that, if you choose a binsize which is greater than the rocking interval, you loose the information about the on source versus the off source time intervals. This in turn causes an incorrect deadtime calculation.
Users should use a binsize equal to the rocking interval so that the GTI extensions are appropriate and hxtdead will have the information it needs.
HELPFUL HINT: Finding the rocking interval in your data:
To determine the rocking interval in your HEXTE observation,
examine the column CmdDwellApMod in the 16-second housekeeping files.
The integer value of this column, which reports the rocking interval
commanded to the HEXTE clusters, is a look-up table as follows:
CmdDwellApMod Dwell (s)
0 0
1 16
2 32
3 64
4 128
In the following example, the
value of CmdDwellApMod is 2, and so the
rocking interval is 32 seconds.
fdump prhead=no FH53_9490e93-94938e0.gz
Name of optional output file[STDOUT]
Names of columns[CmdDwellApMod]
Lists of rows[-]
cmdDwellApMod
1 2
2 2
3 2
4 2
5 2
After the deadtime correction is made, light curves should then be rebinned using
rebinlc. The multiple used to rebin should correspond to one
complete
cycle of rocking on and off source, or 4 times the dwell time. This
way, both source and background data will be present at each new time
step, so that the background can be subtracted.
For archive mode:
This step is very similar to extracting PCA Standard2 data using saextrct.
- If you have production data, you will want to apply the deadtime correction to the light curves and spectra. The current tools do not accept more than one science file at a time, so you should extract products from individual files. In other words, give the extractor only one file at a time.
If, however, you have realtime data and will be unable to apply the deadtime correction anyway, you may make a list of the source (or background) FITS files from which to extract one light curve and spectrum, and enter the name of the list preceded by an '@' character.
- For "GTI files to be OR'd", you must answer "APPLY" so that the resulting exposure correctly accounts for the rocking motion.
- Note on "GTI files to be AND'd": As for PCA Standard2, data filtering with only the GTIOR
file is not usually sufficient, which is why we recommend making a GTIAND file using additional
selection criteria. The example below shows how to create a user gti with maketime and
gives a selection expression that is usually suitable (note that -- depending on the observation
-- data up to typically 10-30 min after the SAA have to be excluded in order to minimize
residual background contributions).
maketime infile=FP_XXX.xfl outfile=good_hexte.gti
expr="offset.lt.0.01.and.ELV.gt.10.and.(time_since_saa.lt.0.0.or.time_since_saa.gt.10)"
name=NAME value=VALUE time=Time compact=no
- One difference from PCA Standard2 data is that the default GOOD columns to extract does not apply. You must make your own column list consisting of the following text (each line followed by a return and with no blank lines before or after):
SpecDet0
SpecDet1
SpecDet2
SpecDet3
and specify the name of this text file, prepended by '@', at the "COLUMNS to be accumulated" prompt.
Please note: On 1996 March 6 at 11:27:12 UT, HEXTE cluster B detector 2 lost the ability to measure spectral
information. Thus, if you are extracting cluster B data for spectral analysis from observations after this date, you should use a column
list that excludes detector 2, ie.:
SpecDet0
SpecDet1
SpecDet3
If you intend to use cluster B for lightcurve analyses (ie., photon-counting only), then you may still use the
full column list, including detector 2, since it still accurately detects photons.
- As discussed above, give a binsize equal to the rocking interval of your data.
- For the best results extracting a lightcurve, you should select only the channels with the highest signal to noise; for most sources, a range like 15-61 would be good. (This is above the LLD threshold and below where the background begins to dominate for a typical spectrum.) You should first extract a full spectrum and compare it to the background to determine which channels would be best for your particular source.
- You can run this from the command line or in a script; use "plist saextrct" to see the names of the parameters (there are many, but only "infile" and "outroot" will change):
saextrct infile=FS52_xxx_src outroot=FS52_xxx_src
gtiorfile=APPLY gtiandfile=good_hexte.gti binsz=16 chint=15-61 ...
along with the long (too long to put here!) list of other parameters.
For event modes:
Deadtime Correction
Now available for Archive and event modes,
but not for generic binned modes.
See Notes below for using realtime data.
For more information about the HEXTE deadtime, please see http://mamacass.ucsd.edu:8080/hexte/status/hexte_deadtime.html.
hxtdead corrects either light curves or spectra for the clock time durations. This is an
essential step in the reduction because the cluster's motion while rocking
is treated as dead time in the off-source data, and background subtraction
will be grossly in error without it. In addition to the file to be
corrected, the hxtdead programs require data from the original FITS file
and also a housekeeping file and coefficients file. They
calculate live time corrections and apply them to the .pha or .lc file.
In both cases, hxtdead applies an algorithm
developed by the HEXTE team to calculate the live-time based on 3
quantities: the live time counter in the FITS Science data file, FS5[0268]_xxx_{src,bkg}, and the so-called ULD and XULD particle rates in the 16-second Housekeeping file, FH5[39]. The coefficients file, hxtdead_200002_pwa.fits
for Cluster A(0) or hxtdead_200002_pwb.fits for Cluster B(1), contains some HEXTE instrument settings which determine the conversion factors used by this algorithm. These files are in the latest version of the HEXTE area of CALDB. See the CALDB Management Page for how to update your current CALDB. Alternatively, you may simply get these individual files from the CALDB part of the anonymous ftp are at https://heasarc.gsfc.nasa.gov/caldb/data/xte/hexte/bcf/deadtime/ and give the appropriate file at the prompt in stead of "CALDB".
Run hxtdead on both the source and background data files; this calculates the correct EXPOSURE keyword for the PHA
files, so at the end of this stage you will have for each FITS data file: one
source file and one or two background files, with appropriate keywords set. Note: make sure to use the corresponding hxtback'ed data file, i.e. the one with the _src or _bkg suffix, rather than the original FITS file. If you do not give the _bkg file, for instance, when correcting the background data, hxtdead will not know what correction factor to apply to account for the rocking time.
% hxtdead
Running HXTDEAD version 0.0.1
=============================================
Calibration file:[]caldb
File containing status apid values (FH5{39}):[] FH53_xxx
File containing events list/archive data (FS5{0268}):[] FS50_xxx_src
.pha/.lc file to be modified:[] FS50_xxx_src.pha
Current EXPOSURE = 1584.000000
Adjusted EXPOSURE = 907.844299
To correct a light curve is essentially the same, but the output is much longer, as it displays the original and corrected rate fore each time bin.
Again, as an example of running this on the command line:
% hxtdead calvalf=caldb engvalf=FH53_xxx eventarcf=FS50_xxx_src
phalcf=FS50_xxx_src.pha
Note: A common warning when running the tool is a message like the following:
apModPeriod undfined for index i = 455
uld or xuld undfined for indices i = 455 j = 0
...
Current EXPOSURE = 784.000000
Adjusted EXPOSURE = 366.632904
Detector 0: Uld data interpolated for 1 IDFs: 7796942 - 7796942
Detector 0: Xuld data interpolated for 1 IDFs: 7796942 - 7796942
...
The tool needs the housekeeping data to cover the entire time range of the science data, which is expected. Due to the different telemetry formats and rates, however, this isn't always the case. Occasionally, the science data will have an extra bin extending beyond the housekeeping, and hxtdead will simply interpolate the values in the housekeeping. This introduces only a negligible error and should in no way affect your analysis.
Another, less common warning which may appear is:
Error: Analysing SA type data.
XFF Revision # < 5.3.1
Continuing with analysis until error processing can be
established.
The tool may run to completion but with incorrect results. This is due to a problem in early processing of HEXTE data which was fixed with XFF version 5.3.1. If you are working with data from original processing that was taken before mid December 1996, you will see this message. If so, retrieve the
reprocessed data from the RXTE data archive.
Using the script hxtlcurv:
As hxtdead requires light curves and spectra to be deadtime corrected
one file at a time and it requires the several steps above for each
file, the HEXTE team has provided a Perl script to perform each of
these steps for one input data file. The script hxtlcurv takes
the science data file, the coefficients file, the 16s housekeeping
file, and several selection options which it feeds to the extractor
(gtifile,timeint file, chmin, chmax.) While originally intended to
produce only light curves, it has an option to have spectra produced
as well, as this does not slow the extractor.
The first version (v3.5) of this script was released with FTOOLS v4.2, but that version has problems and is not recommended. The new version is currently available from our ftp area at "https://heasarc.gsfc.nasa.gov/FTP/xte/software/hxtlcurv/". Follow the directions in the README for how to install it with your current FTOOLS.
A sample command line for running hxtlcurv would look like:
hxtlcurv -i FS52_xxx -k hxtdead_200002_pwa.fits -r FH53_xxx
-b 16 -g obs00.gti -t burst.tint -l 15 -u 61
where "-b 16" specifies the output bin size (in seconds), "-t burst.tint" is the time interval file produced by timetrans, "-l 15 -u 61" are the lower and upper energy channels (in absolute channels, of course.)
First, hxtlcurv runs hxtback (if it does not look like it has already been run, i.e. if files like FS50_xxx_src and FS50_xxx_bkg do not already exist), then it extracts both the on and off source data, then runs hxtdead on the light curves (and spectra if the -p option is added), and finally subtracts the background light curve from the on-source light curve.
The resulting files will be named for the input file, and for source or background, e.g. FS52_xxxxxxx-xxxxxxx will result in FS52_xxxxxxx-xxxxxxx_src.lc, FS52_xxxxxxx-xxxxxxx_bkg.lc, and FS52_xxxxxxx-xxxxxxx.lc (for the net, background subtracted light curve.)
Warning: The manual page for hxtlcurv ("fhelp hxtlcurv") specfies that bin size (-b) must be
16 seconds or less, in the case of event mode data, or 16 seconds only, in the case of archive mode data.
Users wishing for output files with larger bin sizes should use hxtlcurv to make a 16-second binned file,
and then use the FTOOL "rebinlc" to rebin the hxtlcurv output to a larger time value. See "fhelp
rebinlc" for further details.
Combining light curves and spectra Since the event mode data has to be extracted one FITS file at a time and deadtime corrected separately, this describes how to combine the results.
- For combining the individual spectra, use the tool (the values of CORRSCAL, AREASCAL, and BACKSCAL will be incorrect if not set directly on the command line):
% sumpha
Expression to be evaluated[] FS50_xxx_src.pha + FS50_yyy_src.pha
+ FS50_zzz_src.pha
Units algebraic expression to be performed in (C,R,F or Help)[C]
O/p PHA filename[] src.pha
Exposure time flag/value ({value},{file},CALC,NULL or Help)[] CALC
......... processing file: FS50_xxx_src.pha
......... processing file: FS50_yyy_src.pha
......... processing file: FS50_zzz_src.pha
... ... performing algebra in units of COUNTS
... written the PHA data Extension
** sumpha completed successfully
Warning: this tool will not operate on files whose names include the "-" character, so if you name them after the original FITS files which include this between the start and stop times, sumpha will try to interpret it as a minus sign.
After you have combined the various individual spectra to make one source file and one background file, you are ready to XSPEC. (Note: do not attempt to combine spectra from clusters 0 and 1 in this way; the two must be analyzed as different instruments.)
- For combining the light curves, you can use lcurve, which, given 1 for the number of time series, takes a list of input light curves and
appends them.
Generating net light curves from source and background data
In order to subtract the source from the background data, rebin both
source and background files.
The tool rebinlc can next
be used to rebin the source and background light curves. Do the
following
for both _src and _bkg files:
% rebinlc
Running REBINLC_v1.0
========================
Name of input light curve file[] hxt_src.lc
Ratio of number of input to output bins[1] 4
Name for output (rebinned) light curve file[] hxt_src_rb.lc
Now use lcmath to subtract source and background.
% lcmath
Name of input FITS file[] hxt_src_rb.lc
Name of background FITS file[] hxt_bkg_rb.lc
Name of output FITS file[] hxt_net.lc
Scaling factor for input[1.]
Scaling factor for background[1.]
Add instead of subract?[no]
Net or Sum light curve will be corrected
100% completed
You now have background-subtracted HEXTE light curves.
For more information on lcurve, rebinlc and lcmath, please
see the fhelp for these tools.
Form the appropriate RMF:
The easy route:
The large majority of HEXTE observations are made with the source
on-axis, and after the cluster B loss of detector 2 (March 6, 1996).
We have therefore generated "DEFAULT" .rmf, .fov, and .arf files
for this situation. You may use these defaults in your spectral
analysis, and bypass the HXTRSP step in your hexte data reduction.
Find the default HEXTE response matrices via anonymous FTP to
https://heasarc.gsfc.nasa.gov/FTP/xte/calib_data/hexte_files/DEFAULT.
Cluster A, use:
hexte_00may26_pwa.arf
hexte_97mar20c_pwa.rmf
hexte_00may26_pwa.fov
Cluster B, use:
hexte_00may26_pwb013.arf
hexte_97mar20c_pwb013.rmf
hexte_00may26_pwb013.fov
The not-so-easy but still straightforward route:
If your data are from before March 6, 1996, or are off axis, you will
need to use HXTRSP with the latest caldb versions of the calibration
files. In this case, use
the HEXTE response matrices via anonymous FTP at
https://heasarc.gsfc.nasa.gov/FTP/xte/calib_data/hexte_files/HEXTE_caldb. The current versions are hexte_97mar20c_pw*.rmf
and hextecaldb_00may26_pw*.[arf,fov].
- If you have 256-channel event list data, you may use the appropriate 256-channel HEXTE .rmf file and the corresponding .arf file found here.
- For any modes with spectra binned differently (e.g. array data, often in the form of 64 channel spectra), users must first rebin the .RMF file to match the spectral binning of their data. You will first use the ftool rddescr which reads the PHA file and produces a text file with the channel-binning information. This can then be used to rebin the full-sampling 256-channel RMF using the tool rbnrmf thus:
% rddescr phafil=FS50_xxx_src.pha chanfil=FS50_xxx.bins
% rbnrmf binfile=FS50_xxx.bins infile=hexte_97mar20c_pwa.rmf
outfile=FS50_xxx_src.rmf
% fparkey add=yes value="FS50_xxx_src.rmf"
fitsfile=FS50_xxx_src.pha keyword=RESPFILE
The final command writes the name of the RMF into the PHA file for use with
XSPEC.
There now exists a script hxtrsp which accomplishes the same thing as the 3 commands (rddescr, rbnrmf, and fparkey). An example
of the command is:
% hxtrsp -i FS50_76b07d6-76b1e60_src.pha
-a 20804-01-06-00/stdprod/FP_76b07d6-76b6ad2.xfl
In addition, running hxtrsp to create your response matrix will also call hxtarf which will call the latest hxtarf file produced by the HEXTE team in May 2000. This file ensures a more consistent calibration between HEXTE and PCA fluxes than previously. In fact, tests run on 15-30 keV Crab data give HEXTE fluxes that are within about 5 percent of PCA fluxes. Running hxtrsp as just indicated will spawn the following hxtarf call:
% hxtarf chatter=5 clobber=yes phafil= "FS50_76b07d6-76b1e60_src.pha"
inarf=CALDB collcube=CALDB detnam="PWA"
xtefilt="20804-01-06-00/stdprod/FP_76b07d6-76b6ad2.xfl"
outarf="FS50_76b07d6-76b1e60_src.arf" ...
The keywords for the updated hxtrmf and hxtarf files are written to the .pha header.
Details and pointers on running hxtrsp can be found in the
HEXTE Response Recipe
.
Notes on the background:
- Check the background spectra:
If you did not use the -b option in HXTBACK, you may try using either the _p or _m background files. The spectra and/or light curves of these files should be the same, to within the error, unless a contaminating source appears in one of them. One easy way to check the two backgrounds against each other is to enter both as two data sets in XSPEC; if you plot them both this way, you should see that they lie exactly on top of one another. If there is any difference, this could indicate another source in the field. (To find out the exact pointing of the off-source rockings, use the web tool HEXTErock.) Likewise, you can check the background against the on-source spectrum. They should look the same at the highest energies (where there are presumably no source counts). If there is any offset, this could be an indication that there is some mis-match in the way the two were extracted or that the deadtime correction is not right.
Assuming that the two background spectra look OK, you can use sumpha to
add them together before performing a spectral analysis on the source.
- Scaling the background for realtime data:
Though realtime data do not include the necessary housekeeping information for the deadtime correction, they can still be used for some basic analysis. You can at least make the simple correction for the exposure difference introduced by the clusters slewing from source to background pointings. It takes 2 seconds to slew from one pointing to the other, and this time is deadtime in the background data. So, if you bin the source and background lightcurves as described above, you must take into account the fact that there is 4 seconds less exposure for the background per dwell.
For 16 second rocking, for instance, you should scale the background while subtracting in lcmath by entering a factor of 1.33 for the background light curve. (I.e. us the tool to multiply the count rate by 4/3, the ratio of 16 seconds on source to 12 seconds background, where the difference is the two slews, each of 2 seconds.) For the spectra, the total exposure should be corrected by the inverse of this amount. Simply take the value of the EXPOSURE keyword for the background PHA file, obtained using fkeyprint, and change it in the header using fparkey to 0.75 times the original exposure. (The fhelp for these tools describes the simple usage.)
Now you can deadtime correct realtime HEXTE data
It is now possible to calculate HEXTE deadtimes from the realtime data
products made available by the SOF shortly after the observations are
performed. Since March 2000, the FH53 and FH59 files necessary for
calculating HEXTE deadtimes have been included with the realtime data
(available at
https://xtesof.nascom.nasa.gov/pub/FITS) and the more
complete SOF "production" data (
https://xtesof.nascom.nasa.gov/pub/FITS/production). (Technical
issues
prevented these files from being included previously.)
With these files, you can now use the HXTDEAD tool, and perform a
complete extraction of HEXTE spectra and lightcurves by following the
normal procedures for
working with realtime data.
If you have a question about RXTE, please send email to one of our
help desks.
|
