Search:

[Advanced Search]



Things to watch out for with XRISM data processing and analysis

Email the XRISM Science Data Center (SDC) helpdesk with any questions: XRISM-SDC-help@lists.nasa.gov

General

  • Products
    • The files in the 'products' directory for each instrument are meant for a quicklook preview, and not for any detailed analysis. Spectra and responses should be produced for each source using the XRISM ftools.
    • The Xtend region file should correspond to a circle with radius 2.5' (85 pixels), but the radius is 3.83' (130 pixels), although the name of the file implies the former radius. You can edit the file to change the radius. Note that the circles in the Xtend images also have a radius of 3.83', not 2.5'.
    • There are some known cosmetic errors in the pipeline preview products. All of these errors will be fixed soon.
  • Response files
    • The canned responses (see Response files for proposers) are designed for simulations. Although they have the standard energy channels and can be loaded in Xspec with spectra extracted from flight data, the actual responses (RMF and ARF) are observation-dependent and should be generated for each application for proper spectral fitting.
  • xapipeline / rslpipeline / xtdpipeline
    • 'xapipeline' is the user pipeline tool that calls each instrument pipeline tool, 'rslpipeline' and 'xtdpipeline', to reprocess the data. Any of these tools can be run by hand. See the help for each of these tools for more information.
    • In general, the SDC recommends that users save reprocessed files or analysis results outside of the original archive data directory. Otherwise a FATAL error may result due to the presence of duplicate files within the parent (input) directory tree.
      • For example, use 'xapipeline indir=000145000 outdir=000145000_resolve_reproc', instead of 'outdir=000145000/resolve_reproc'

Resolve

  • Data format and naming conventions
    • The event files (and the spectra extracted from them) are split according to the filter wheel setting:
      • p0px0000 UNDEF
      • p0px1000 OPEN
      • p0px2000 Polyimide
      • p0px3000 ND
      • p0px4000 Be
      • p0px5000 Fe55
    • In general, you should use 'p0px1000' events for most science or calibration data analysis, unless the ND filter was used.
  • Gain correction and calibration
    • The energy scale drift correction tool, rslgain, can sometimes fail in its attempt to find the fitting solution used to derive the gain drift at a specific time for a specific pixel. This may result in an incorrect energy scale for some events in that pixel.
    • The energy scale quality reports, which are described and linked from the main data analysis page, can be used to identify these and more generally assess the per-pixel gain.
    • This situation may also be identified by examination of the pipeline processing log, e.g., xa300003010_joblog.html (for OBSID 300003010) in the archive log directory. If the "TOTAL NUMBER OF SUCCESSFUL FITS" is less than the "TOTAL NUMBER OF FITTED GROUPS" (also expressed by text such as 'rslgain: INFO: Failure fitting group with splitval = 0' for pixel 0) in screen output such as that shown below, the energy scale for that pixel may warrant additional scrutiny. Note that the rslgain tool is generally run multiple times - once for the calibration pixel 12 and once when the filter wheel Fe55 source is used in assigning energy for every pixel in the array. It is the per-pixel case used for energy assignment that is of interest.


    • There is a gain shift between Hp and Mp events. Correction strategies are under development but, for now, energy scale accuracy may only be assured for Hp events.
    • Pixel 27 exhibits, as yet unexplained, gain excursions. Due to the relative infrequency of gain fiducial measurements, the calculated gain drift correction may not capture these excursions, rendering the Pixel 27 energy scale less reliable than for other pixels.
    • The Resolve team recommends a new RISE_TIME screening expression, and this is introduced in version 2.1 of the Quick-Start Guide (updated 6 May, 2024). However, the impact on very bright sources of the possible (energy-dependent) removal of source events by this and other screening is under investigation.
  • ARF
    • Resolve region file outputs from xamkregion should not be used as input regions to the ARF generator (xaarfgen or xaxmaarfgen) because the pixels in these region files are slightly rotated by different amounts relative to each other and some of the pixels overlap with each other. This is a design flaw inherited from Hitomi, based on mis-use of the XRISM ground calibration data and will be fixed in the future. However, note that the Resolve DET coordinates region file currently output from the pipeline is created with xamkregion, so should not be used as input to the ARF generator, so should only be used for overlaying Resolve FoV on images from other instruments.
  • RMF
    • Resolve RMF size. The Resolve X-large RMF should not be made with the full 0.5 eV per bin resolution as doing so will result in an unreasonably large size of 7 Gb. Instead, users are strongly urged to use the option to split the response function into two matrices (e.g., with command like 'rslmkrmf splitrmf=yes elcbinfac=16 whichrmf=X splitcomb=yes'). Read the help files for rslrmf and rslmkrmf for details.
    • Resolve RMF normalization and net effective area. It has been found that there is an excess of Ls (ITYPE=4) events in Resolve data that are not due to the source. In addition, for very bright sources, the observed branching ratios into different grades do not agree with theoretical expectations, possibly due to screening out of true source events. These phenomena are under investigation. They affect the calculated effective area function and the relative weights of the line-spread functions in different pixels. A related issue for bright sources is that the XMA effective area per pixel varies by such a large amount over the Resolve array, that the grade fractions vary significantly pixel-to-pixel. Large-amplitude variability in bright sources further complicates the analysis. A more complicated method of constructing response files is required and is currently in development. Also, for phase-resolved spectroscopy, different response functions may be required for different phases. For now, it is recommended not to fit any spectra above 11 keV.
  • rslpipeline
    • In rare cases, on some platforms, rslpipeline (and xapipeline) may halt or stall when running the rslctfluct task due to a memory issue. This is under investigation and is expected to be fixed in the future. Users may run these scripts with calc_fluct=no to avoid this issue.

Xtend

  • Data format and naming conventions
    • Xtend can have multiple input files split by the operating modes. The Xtend file naming convention is:
      • xaNNNNNNNNNxtd_pRzdmcccca_cl.evt
      • xa000127000xtd_p031100010_cl.evt (example 1)
      • xa000127000xtd_p032000010_cl.evt (example 2)

      • where:
      • NNNNNNNNN = 9-digit OBSID
      • p = 'p' for pointed observation, 's' for slew observation, 'a' for all (will usually be 'p')
      • R = index for data divided into multiple files due to (e.g.) filesize. R is in the range [0-9a-zA- Z] and set to 0 if no division is needed.
      • zdmccca = dataClass, which describes the CCDs used and their configuration, where:
        • z = placeholder always set to 3 for Xtend in-flight data
        • d = DETNAM: 0 = CCD, 1 = CCD12, 2 = CCD34
        • m = DATAMODE: 0 = WINDOW1 (full window), 1 = WINDOW2 (1/8 window), 2 = WINDOW1BURST (full window + 0.1 sec burst mode), 3 = WINDOW2BURST (1/8 window + 0.1 sec burst mode)
        • cccc = hexadecimal encoding of on-board instrument settings (thresholds, area discrimination, ADC chains used, readout nodes used, charge injection settings, etc.)
        • a = reserved bit, 0 for now
    • Some common dataClass values you may encounter:
      • 30000010: the event list contains all four CCDs in full window mode
      • 31100010: the event list contains CCD1 and CCD2 in 1/8 window mode
      • 31300010: the event list contains CCD1 and CCD2 in 1/8 window + 0.1 sec burst mode
      • 32000010: the event list contains CCD3 and CCD4 in full window
      • The first three digits of this (300*, 311*, 313*, 320*) are the important ones that tell you the mode and the contents of the event list, whether it contains data from all CCDs or just a pair. You can also look at the DETNAM, DATAMODE, and DATACLAS header keywords.
      • DATAMODE = WINDOW1BURST (full window + 0.1 sec burst mode) is not supported for science observations, only for Xtend IT usage.
    • Pairs of CCDs share clock drivers, so CCD1+CCD2 always operate in the same mode, and CCD3+CCD4 always operate in the same mode.
    • CCD2 is the aimpoint CCD, and only the CCD1+CCD2 pair is allowed to operate in 1/8 window or 1/8 window+0.1 sec burst mode; the CCD3+CCD4 pair always operates in full window mode.
      • You should always see a 320* event list along with a 311* or 313* event list.
      • You will generally only see a single 300* event list when all CCDs are in full window mode.
  • Regions in 1/8 window mode
    • In 1/8 window mode, the region for input to the ARF generator must be confined to be completely inside the boundaries of the window, otherwise the ARF will not be correct. The image below in DET coordinates shows an example of a correct region.
  • Flickering pixels, cosmic-ray echo events, anomalus pixels
    • It has been found that the tool searchflickpix, used for removing events from flickering pixels and cosmic-ray echo events does not work well with XRISM data in an automated setting, so it is not applied in the XRISM pipeline. This is because there is a risk that true source events might be removed. However, the pipeline still runs searchflickpix without applying it to the data, but produces the .fpix file that lists the flagged pixels. This file is used by xaexpmap (which then is used by xaarfgen). Therefore, the .fpix file from the pipeline should not be used as input to the xaexpmap. The user should follow the Quick-Start Guide on mitigation strategies for anomalous pixel removal, and either make their own .fpix file, or specify NONE as input to xaexpmap. Note the following corrections to the Quick-Start Quide v2.1:
      • All of the searchflickpix commands should have the following parameter settings added:
        • xcol=DETX ycol=DETY
      • If you are directed to section 5 from step 3 in the Appendix (i.e., if you are applying a simple threshold cut), add the following parameter settings to both searchflickpix commands:
        • grade=ALL


  • Obscured portion of the field of view (added 2024-10-03)
    • The corner of the Xtend FoV farthest from the aimpoint is blocked by a structure of the camera. While there may be counts detected in this region, they are all from unfocused non-X-ray background and not from celestial X-rays. The approximate location of this region can be seen below in a DET-coordinate image of the sunlit limb of the Earth from the 'day Earth' trend archive.
    • The XRISM SDC will release a DET coordinates region file that users can employ to exclude this region.


Miscellaneous

  • The SDC has prepared ds9 'template' region files that include the XRISM instrument fields of view. These can be displayed on any image with WCS information, e.g. from ROSAT, Chandra, XMM, eROSITA, DSS, SDSS. These regions should be used only for display purposes, not for extracting products for analysis.