HEASARC Staff Scientist Position - Applications are now being accepted for a Staff Scientist with significant experience and interest in the technical aspects of astrophysics research, to work in the High Energy Astrophysics Science Archive Research Center (HEASARC) at NASA Goddard Space Flight Center (GSFC) in Greenbelt, MD. Refer to the AAS Job register for full details.

Search:

[Advanced Search]



Things to watch out for with XRISM data processing and analysis

Email the XRISM Science Data Center (SDC) helpdesk with any questions: XRISM-SDC-help@lists.nasa.gov

General

  • Products
    • The files in the 'products' directory for each instrument are meant for a quicklook preview, and not for any detailed analysis. Spectra and responses should be produced for each source using the XRISM ftools.
    • The Xtend region file should correspond to a circle with radius 2.5' (85 pixels), but the radius is 3.83' (130 pixels), although the name of the file implies the former radius. You can edit the file to change the radius. Note that the circles in the Xtend images also have a radius of 3.83', not 2.5'.
    • There are some known cosmetic errors in the pipeline preview products. All of these errors will be fixed soon.
  • Response files
    • The canned responses (see Response files for proposers) are designed for simulations. Although they have the standard energy channels and can be loaded in Xspec with spectra extracted from flight data, the actual responses (RMF and ARF) are observation-dependent and should be generated for each application for proper spectral fitting.
  • xapipeline / rslpipeline / xtdpipeline
    • 'xapipeline' is the user pipeline tool that calls each instrument pipeline tool, 'rslpipeline' and 'xtdpipeline', to reprocess the data. Any of these tools can be run by hand. See the help for each of these tools for more information.
    • In general, the SDC recommends that users save reprocessed files or analysis results outside of the original archive data directory. Otherwise a FATAL error may result due to the presence of duplicate files within the parent (input) directory tree.
      • For example, use 'xapipeline indir=000145000 outdir=000145000_resolve_reproc', instead of 'outdir=000145000/resolve_reproc'

Resolve

  • Data format and naming conventions
    • The event files (and the spectra extracted from them) are split according to the filter wheel setting:
      • p0px0000 UNDEF
      • p0px1000 OPEN
      • p0px2000 Polyimide
      • p0px3000 ND
      • p0px4000 Be
      • p0px5000 Fe55
    • In general, you should use 'p0px1000' events for most science or calibration data analysis, unless the ND filter was used.
  • Gain correction and calibration
    • The energy scale drift correction tool, rslgain, can sometimes fail in its attempt to find the fitting solution used to derive the gain drift at a specific time for a specific pixel. This may result in an incorrect energy scale for some events in that pixel.
    • The energy scale quality reports, which are described and linked from the main data analysis page, can be used to identify these and more generally assess the per-pixel gain.
    • This situation may also be identified by examination of the pipeline processing log, e.g., xa300003010_joblog.html (for OBSID 300003010) in the archive log directory. If the "TOTAL NUMBER OF SUCCESSFUL FITS" is less than the "TOTAL NUMBER OF FITTED GROUPS" (also expressed by text such as 'rslgain: INFO: Failure fitting group with splitval = 0' for pixel 0) in screen output such as that shown below, the energy scale for that pixel may warrant additional scrutiny. Note that the rslgain tool is generally run multiple times - once for the calibration pixel 12 and once when the filter wheel Fe55 source is used in assigning energy for every pixel in the array. It is the per-pixel case used for energy assignment that is of interest.


    • There is a gain shift between Hp and Mp events. Correction strategies are under development but, for now, energy scale accuracy may only be assured for Hp events.
    • Pixel 27 exhibits, as yet unexplained, gain excursions. Due to the relative infrequency of gain fiducial measurements, the calculated gain drift correction may not capture these excursions, rendering the Pixel 27 energy scale less reliable than for other pixels.
    • The Resolve team recommends a new RISE_TIME screening expression, and this is introduced in version 2.1 of the Quick-Start Guide (updated 6 May, 2024). However, the impact on very bright sources of the possible (energy-dependent) removal of source events by this and other screening is under investigation.
  • ARF
    • Resolve region file outputs from xamkregion should not be used as input regions to the ARF generator (xaarfgen or xaxmaarfgen) because the pixels in these region files are slightly rotated by different amounts relative to each other and some of the pixels overlap with each other. This is a design flaw inherited from Hitomi, based on mis-use of the XRISM ground calibration data and will be fixed in the future. However, note that the Resolve DET coordinates region file currently output from the pipeline is created with xamkregion, so should not be used as input to the ARF generator, so should only be used for overlaying Resolve FoV on images from other instruments.
    • Sub-array XMA effective area. The XMA effective area energy dependence shows some variation on the size scale of Resolve pixels, but for point sources that have < 1 ct/s (0-30 keV, all grades), the Resolve pixels beyond the central four make a diminishing contribution to the net effective area for an Hp spectrum from the whole array. However, as the source count rate increases, pixels exposed to the outer parts of the PSF make an increasing contribution to the net Hp effective area relative to pixels exposed to the PSF core. A more accurate net effective area will be obtained by making ARFs and RMFs separately for groups of pixels, and then combining the ARFs and RMFs into a single response (RSP) file, using appropriate weights for the ARFs. For example, the array could be divided into three pixel groups, corresponding to the four central pixels, an inner ring of pixels that surround the central four, and an outer ring consisting of the remaining pixels. Note that smaller regions on the detector require correspondingly larger values of the 'numphoton' parameter in xaarfgen.
  • RMF
    • Resolve RMF size. The Resolve X-large RMF should not be made with the full 0.5 eV per bin resolution as doing so will result in an unreasonably large size of 7 Gb. Instead, users are strongly urged to use the option to split the response function into two matrices (e.g., with command like 'rslmkrmf splitrmf=yes elcbinfac=16 whichrmf=X splitcomb=yes'). Read the help files for rslrmf and rslmkrmf for details.
    • Resolve RMF normalization and net effective area. It has been found that there is an excess of Ls (ITYPE=4) events in Resolve data that are not due to the source. In addition, for very bright sources, the observed branching ratios into different grades do not agree with theoretical expectations, possibly due to screening out of true source events. These phenomena are under investigation. They affect the calculated effective area function and the relative weights of the line-spread functions in different pixels. A related issue for bright sources is that the XMA effective area per pixel varies by such a large amount over the Resolve array, that the grade fractions vary significantly pixel-to-pixel. Large-amplitude variability in bright sources further complicates the analysis. A more complicated method of constructing response files is required and is currently in development. Also, for phase-resolved spectroscopy, different response functions may be required for different phases. Additionally, for now, it is recommended not to fit any spectra above 12 keV because branching ratio anomalies are worse at higher energies.

      For now, it is recommended to make Resolve RMFs using a custom cleaned event file that has Ls events removed, and has ALL events that have PI values lower than 6000 and higher than 20000 also removed. This custom event file is to be used for RMF generation only, and not for analysis. For sources that have more than 1 ct/s in the 0-30 keV band (summed over all 5 grades), the procedure will result in an upper limit on the effective area (corresponding to lower limits on derived fluxes). An approximate lower limit on the effective area can be obtained by making RMFs using a different custom event file that does NOT have Ls events removed, but has all events with PI<10000 and PI>16000 removed. For sources that have <1 ct/s, the lower and upper limits on the effective area functions converge. Until solutions are found for solving the anomalous Ls events and branching ratios problems, the uncertainties in effective areas have to be treated as systematic errors.
    • Pixel-to-Pixel Spectral Differences. It has been found that in some sources (including point sources), there are broad and significant differences in spectra extracted from different Resolve pixels. The magnitude and energy dependence of the differences varies from source to source and from pixel to pixel, and is not dependent on count rate. The origin of these spectral differences is not yet understood, and is under investigation. The figure below shows the ratio of spectra from 31 pixels that exclude the central four, to the spectra from the central four, for two non-variable point sources. The left panel below shows a case for which the spectral ratio is flat (so any spectral differences are smaller than the statistical errors), and the right-hand panel shows a case for which the spectral ratio changes by as much as a factor of ~2.3 between 2 and 12 keV. Until the origin of the spectral differences is understood, they will have to be treated as systematic errors. If the photon statistics are sufficiently high, spectral differences may be observed between smaller groups of pixels, or even between individual pixels. At this time, it is not known how to determine which pixels or group of pixels yield spectra that are most faithful to the "true" spectrum. The pixel-to-pixel variation of the XMA effective area is insufficient to explain the observed spectral anomalies. Moreover, an XMA origin would impose similar spectral differences for all on-axis point-source observations, but is found not to be the case. Extended sources present a greater challenge, because while it is possible to directly observe sub-array spectral differences in point-sources, for extended sources we cannot assume that the spectra in all pixels should intrinsically be essentially the same. This means that at this time it is not possible in general to gauge the severity of sub-array spectral distortion in extended sources, and this also complicates spatial-spectral mixing (SSM) analyses. In addition, it is possible that "jumps" in spectral parameters may occur between pixels that imply unphysical spatial variations.
  • rslpipeline
    • In rare cases, on some platforms, rslpipeline (and xapipeline) may halt or stall when running the rslctfluct task due to a memory issue. This is under investigation and is expected to be fixed in the future. Users may run these scripts with calc_fluct=no to avoid this issue.

Xtend

  • Data format and naming conventions
    • Xtend can have multiple input files split by the operating modes. The Xtend file naming convention is:
      • xaNNNNNNNNNxtd_pRzdmcccca_cl.evt
      • xa000127000xtd_p031100010_cl.evt (example 1)
      • xa000127000xtd_p032000010_cl.evt (example 2)

      • where:
      • NNNNNNNNN = 9-digit OBSID
      • p = 'p' for pointed observation, 's' for slew observation, 'a' for all (will usually be 'p')
      • R = index for data divided into multiple files due to (e.g.) filesize. R is in the range [0-9a-zA- Z] and set to 0 if no division is needed.
      • zdmccca = dataClass, which describes the CCDs used and their configuration, where:
        • z = placeholder always set to 3 for Xtend in-flight data
        • d = DETNAM: 0 = CCD, 1 = CCD12, 2 = CCD34
        • m = DATAMODE: 0 = WINDOW1 (full window), 1 = WINDOW2 (1/8 window), 2 = WINDOW1BURST (full window + 0.1 sec burst mode), 3 = WINDOW2BURST (1/8 window + 0.1 sec burst mode)
        • cccc = hexadecimal encoding of on-board instrument settings (thresholds, area discrimination, ADC chains used, readout nodes used, charge injection settings, etc.)
        • a = reserved bit, 0 for now
    • Some common dataClass values you may encounter:
      • 30000010: the event list contains all four CCDs in full window mode
      • 31100010: the event list contains CCD1 and CCD2 in 1/8 window mode
      • 31300010: the event list contains CCD1 and CCD2 in 1/8 window + 0.1 sec burst mode
      • 32000010: the event list contains CCD3 and CCD4 in full window
      • The first three digits of this (300*, 311*, 313*, 320*) are the important ones that tell you the mode and the contents of the event list, whether it contains data from all CCDs or just a pair. You can also look at the DETNAM, DATAMODE, and DATACLAS header keywords.
      • DATAMODE = WINDOW1BURST (full window + 0.1 sec burst mode) is not supported for science observations, only for Xtend IT usage.
    • Pairs of CCDs share clock drivers, so CCD1+CCD2 always operate in the same mode, and CCD3+CCD4 always operate in the same mode.
    • CCD2 is the aimpoint CCD, and only the CCD1+CCD2 pair is allowed to operate in 1/8 window or 1/8 window+0.1 sec burst mode; the CCD3+CCD4 pair always operates in full window mode.
      • You should always see a 320* event list along with a 311* or 313* event list.
      • You will generally only see a single 300* event list when all CCDs are in full window mode.
  • Regions in 1/8 window mode
    • In 1/8 window mode, the region for input to the ARF generator must be confined to be completely inside the boundaries of the window, otherwise the ARF will not be correct. The image below in DET coordinates shows an example of a correct region.
  • Flickering pixels, cosmic-ray echo events, anomalous pixels
    • It has been found that the tool searchflickpix, used for removing events from flickering pixels and cosmic-ray echo events does not work well with XRISM data in an automated setting, so it is not applied in the XRISM pipeline. This is because there is a risk that true source events might be removed. However, the pipeline still runs searchflickpix without applying it to the data, but produces the .fpix file that lists the flagged pixels. This file is used by xaexpmap (which then is used by xaarfgen). Therefore, the .fpix file from the pipeline should not be used as input to the xaexpmap. The user should follow the Quick-Start Guide on mitigation strategies for anomalous pixel removal, and either make their own .fpix file, or specify NONE as input to xaexpmap. Note the following corrections to the Quick-Start Quide v2.1:
      • All of the searchflickpix commands should have the following parameter settings added:
        • xcol=DETX ycol=DETY
      • If you are directed to section 5 from step 3 in the Appendix (i.e., if you are applying a simple threshold cut), add the following parameter settings to both searchflickpix commands:
        • grade=ALL


  • Obscured portion of the field of view (added 2024-10-03)
    • The corner of the Xtend FoV farthest from the aimpoint is blocked by a structure of the camera. While there may be counts detected in this region, they are all from unfocused non-X-ray background and not from celestial X-rays. The approximate location of this region can be seen below in a DET-coordinate image of the sunlit limb of the Earth from the 'day Earth' trend archive.
    • The XRISM SDC will release a DET coordinates region file that users can employ to exclude this region.


Miscellaneous

  • The SDC has prepared ds9 'template' region files that include the XRISM instrument fields of view. These can be displayed on any image with WCS information, e.g. from ROSAT, Chandra, XMM, eROSITA, DSS, SDSS. These regions should be used only for display purposes, not for extracting products for analysis.