NAME
searchflickpix - Search for anomalous flickering pixels in event files from
CCD-type detectors
USAGE
searchflickpix infile outfile cellsize impfac logporb1 logprob2 bthresh
DESCRIPTION
The searchflickpix task is a mission-independent task that reads an input
event file to search for flickering pixels, which are pixels that exhibit
an anomalously high pulse height over a short period of time (usually less
than an observation). The task outputs a file containing events affected
by the detected flickering pixels in the first EVENTS extension. If the
parameter 'cleanimg=yes', the converse is recorded, namely the clean events
that are not affected by flickering pixels. In the second extension (named
PIXELS), the location of each flickering pixel is recorded, along with the
number of counts, duration, and average pulse height of the flickering
pixel. This task is based on the ftool cleansis, originally developed for
ASCA/SIS, but it is significantly altered from the original task to have
more universality and flexibility. It is a mission-independent task, but
is only meant to be used for event data from CCD-type detectors, such as
ASCA/SIS, Suzaku/XIS, Hitomi/SXI, and XRISM/Xtend.
The main routine for flickering pixel detection is a two-stage process, similar to
that in the ASCA task cleansis. This procedure is performed separately for each
detector chip, as specified by the 'chipcol', 'firstchip', and 'lastchip' parameters.
After the event file is read, event counts in each pixel are compared to the mean
counts in the full chip, and the pixels for which the Poisson probability (calculated
using 'impfac') exceeds the threshold given by the 'logprob1' parameter are flagged.
The pixel locations for this comparison are defined by the 'xcol' and 'ycol'
parameters, which can be in any coordinate system that is fixed to the detector frame
and which uniquely specifies pixels in the detector chip. For Hitomi/SXI, 'xcol=ACTX' and
'ycol=ACTY' should be used for further flagging of flickering pixels with sxiflagpix.
However, if the output event file (made with 'cleanimg=no') is to be used as input
to the tasks ahexpmap or xaexpmap, DET coordinates should be set using xcol=DETX and ycol=DETY.
For any mission, SKY coordinates should not be used in general, since changes in
spacecraft attitude will change the correspondence between SKY pixels and physical
detector pixels with time. Note that 'chipcol' does not need to refer to an entire
CCD, but can refer to parts of the CCD as long as pixel coordinates are unique. For
example, for a Suzaku/XIS event list, 'chipcol=SEGMENT', 'xcol=RAWX', and 'ycol=RAWY'
will search for flickering pixels separately in each of the four CCD segments and
output them to the same PIXELS extension in 'outfile'. This is a very flexible tool,
and care must be taken to correctly specify 'chipcol', 'firstchip', 'lastchip',
'xcol', and 'ycol' so that they uniquely identify physical pixels.
Next, the counts in each pixel are compared to the pixels in a surrounding
box of size ('cellsize' x 'cellsize'), and anomalous pixels are flagged
similarly to the first step, but with another Poisson probability
parameter, 'logprob2'. If the surrounding pixels yield no counts, a zero
background threshold specified by the parameter 'bthresh' is applied
instead. This second Poisson search is repeated if 'iterate=yes'. Last,
if 'flagedge=yes', pixels along the chip edges are flagged as flickering
pixels for the full duration of the observation.
The input events can be filtered several ways before the flickering pixel
search is done. By setting 'chancol' to a column such as PHA or PI in the
input event list and adjusting 'chanmin' and 'chanmax', a pulse-height
filter (>= chanmin && <= chanmax) is applied to the events
used for the flickering pixel search. The 'chancol' parameter may also be
an element of a vector column, such as 'chancol=PHAS[0]' to specify only
the central pixel of each XRISM/Xtend or Hitomi/SXI event island. No
filtering is done if 'chancol=NONE'. Similarly, spatial filters can be
applied with 'xmin', 'xmax', 'ymin', and 'ymax'. The default parameter
settings use the TLMIN and TLMAX header keywords corresponding to the
appropriate column 'xcol' and 'ycol', effectively applying no filter. A
grade filter may be applied using the 'gradecol' and 'grade' parameters.
Finally, the 'n_division' parameter may be used to divide the observation
into equally-spaced time bins and search each duration separately. This is
useful to search for short-duration flickering pixels that might not be
flagged in a long observation with high background signal.
The parameter 'cellsize' must be an odd integer or zero, and it is
recommended to be about 1/6 of the PSF HPD (half-power diameter),
which corresponds to about 7 pixels for the Hitomi/SXI (for an HPD
of 1.2 arcmin, see Maeda et al. 2018, PASJ, 70, 19).
If 'cellsize=0', a simple counts-per-pixel cutoff is applied,
so any pixels in the chip image containing more than 'bthresh' counts are
flagged as flickering. Note that for this mode of operation requires 'grade=ALL',
otherwise not all events that exceed the cut-off threshold will be flagged.
The recommended value for 'logprob1' is
log10(1/'chipsize'), where 'chipsize' is the total number of pixels in a
single detector chip ('logprob1' is about -5.6 for Hitomi/SXI). For these values,
'bthresh=3', where 3 is recommended, although a lower threshold may be set,
based on the expected background rate.
After the flickering pixels are determined, and as an update from cleansis,
this task calculates and outputs a time duration
(or START and STOP times) when the flickering events are observed for each
detected pixel to the PIXELS extension. This function is disabled when the parameter
'timecol=NONE'. In most cases, START and STOP might be identical to TSTART
(observation start time) and TSTOP (observation stop time) header keywords, respectively.
However, given that flickering pixels are not persistent but repeat on
timescales ranging from hours to days, the duration can be shorter than the
entire observation time, especially for long-exposure observations. If
'duration=yes', a detailed duration search is performed to
determine how long each flickering pixel is actually flickering. This
information is output as the FRACTION column in the PIXELS extension, and it
is the fraction of time the pixel flickers between START and STOP. The
time-averaged pulse height in each flickering pixel is also output to the
'chancol' column in the PIXELS extension, unless 'chancol=NONE'.
PARAMETERS
- infile [filename]
- Input event file. The EVENTS extension must be present.
- outfile [filename]
- Output event file.
- (timecol = TIME) [string NONE|time column name]
- Time column (or NONE). This column provides the time information used
to determine first and last times when flickering events occur in each
detected flickering pixel, outputting START and STOP columns in the
PIXELS extension. If set to NONE, this time information will not be in the output.
- (chipcol = CCD_ID) [string NONE|CCD ID column name]
- Chip identification column name (or NONE). If set to NONE, then all
events are searched together for flickering pixels. Otherwise, events are
searched separately by 'chipcol' grouping, but only for 'chipcol' values
between 'firstchip' and 'lastchip', inclusive. For an instrument with one
physical detector per event list, the parameter should be set to NONE; for
instruments with multiple detectors in a single event list, the proper
column name should be specified, otherwise different physical pixels will
be mixed together in the search for flickering.
- (xcol = ACTX) [string]
- X coordinate column. This should be a coordinate fixed to the detector
frame that uniquely identifies pixels. If the output flickering pixel event
file is to be used as input to ahexpmap or xaexpmap, choose xcol=DETX.
- (ycol = ACTY) [string]
- Y coordinate column. This should be a coordinate fixed to the detector
frame that uniquely identifies pixels. If the output flickering pixel event
file is to be used as input to ahexpmap or xaexpmap, choose ycol=DETY.
- (chancol = PI) [string NONE|channel column name]
- Name of input and output event channel (or pulse height) column (or
NONE). This column is used along with the parameters 'chanmin' and
'chanmax' to filter the events used for the flickering pixel search. An
element of a vector column can be specified, such as 'chancol=PHAS[0]' to
specify only the central pixel of each Hitomi/SXI or Suzaku/XIS event
island. The task also calculates the average pulse height values of the
flickering events for each detected pixel and outputs this value as the
'chancol' column in the PIXELS extension. If set to NONE, no pulse height
filtering is done, and the average pulse height value is not output.
- (gradecol = GRADE) [string NONE|grade column name]
- Name of input event grade column (or NONE). If this name is specified,
only events with specific grades given in the 'grade' parameter are used in
accumulating the chip image for the flickering pixel search. If set to
NONE, all grades are used, and the 'grade' parameter is ignored.
- (grade = 0) [string ALL|grade list]
- Event grade(s) for cleaning (or ALL). This should be a list of grade
values from the 'gradecol' column that are used in accumulating the chip
image for the flickering pixel search. Values should be comma-separated,
and ranges can be specified by a dash. For example, 'grade=0,2,3,4,6' and
'grade=0,2-4,6' are equivalent. If set to ALL, all grades are used. This
is the same result as setting 'gradecol=NONE', and if that is specified,
the 'grade' parameter is ignored.
- (n_division = 1) [integer]
- Divide total observation time into the given number.
- (cleanimg = no) [boolean yes|no]
- If set to yes, the cleaned (non-flickering) events are output instead
of the flickering events.
- cellsize = 7 [integer]
- Poisson clean cell size (odd integer > 1). This is given in units of
pixels and it must be an odd integer greater than 1. If
set to 0, this invokes the simple counts-per-pixel cutoff.
- impfac = 320 [double]
- Impact factor for the gamma function used for the Poisson search.
In cleansis it was fixed at a value of 320, which is the default here.
- logprob1 = -5.6 [double]
- The LOG (base 10) of the Poisson probability threshold for the
first-step flickering pixel search (using the entire chip). It must be
negative. If 'cellsize=0', this parameter is ignored.
- logprob2 = -5.6 [double]
- The LOG (base 10) of the Poisson probability threshold for the
second-step flickering pixel search (comparing to the surrounding pixels).
It must be negative. If 'cellsize=0', this parameter is ignored.
- (iterate = yes) [boolean yes|no]
- If set to yes, iterate the second step of Poisson cleaning.
- (flagedge = no) [boolean yes|no]
- If set to yes, pixels along the chip edge are flagged as flickering for
the entire duration of the observation.
- bthresh = 3 [integer]
- Zero background cutoff threshold, applied when the local mean
background level is zero. Check this number against the actual background
count rate. If 'cellsize=0', this parameter gives the fixed count cutoff
threshold.
- (duration = no) [boolean yes|no]
- If set to yes, a detailed search for flickering pixel duration is performed.
- (sigma = 3.0) [double]
- Significance level for flickering pixel duration.
- (firstchip = TLMIN) [string TLMIN|CCD identification]
- Minimum value for the 'chipcol' column. If set to TLMIN, the task
searches for the header keyword TLMIN of 'chipcol' and uses this value. If
'chipcol=NONE", this parameter is ignored.
- (lastchip = TLMAX) [string TLMAX|CCD identification]
- Maximum value for the 'chipcol' column. If set to TLMAX, the task
searches for the header keyword TLMAX of 'chipcol' and uses this value. If
'chipcol=NONE", this parameter is ignored.
- (xmin = TLMIN) [string TLMIN|number]
- Minimum value for the 'xcol' column to allow spatial filtering of input
events. If set to TLMIN, the task searches for the header keyword TLMIN of
the related column, and uses this value.
- (xmax = TLMAX) [string TLMAX|number]
- Maximum value for the 'xcol' column to allow spatial filtering of input
events. If set to TLMAX, the task searches for the header keyword TLMAX of
the related column, and uses this value.
- (ymin = TLMIN) [string TLMIN|number]
- Minimum value for the 'ycol' column to allow spatial filtering of input
events. If set to TLMIN, the task searches for the header keyword TLMIN of
the related column, and uses this value.
- (ymax = TLMAX) [string TLMIN|number]
- Maximum value for the 'ycol' column to allow spatial filtering of input
events. If set to TLMAX, the task searches for the header keyword TLMAX
of the related column, and uses this value.
- (chanmin = TLMIN) [string TLMIN|number]
- Minimum value for 'chancol' to use in accumulating the chip image for
flickering pixel search. If set to TLMIN, the task searches for the header
keyword TLMIN of the related column, and uses this value. If
'chancol=NONE', this parameter is ignored.
- (chanmax = TLMAX) [string TLMAX|number]
- Maximum value for 'chancol' to use in accumulating the chip image
for flickering pixel search. If set to TLMAX, the task searches for the
header keyword TLMAX of the related column, and uses this value. If
'chancol=NONE', this parameter is ignored.
- (clobber = no) [boolean yes|no]
- Overwrites the existing output file if set to yes
- (chatter = 2) [integer 0|1|2|3]
- Chatter level for output. Set to 0 to suppress output, or to 1, 2, or 3 for increasing the chatter of the output.
- (logfile = !DEFAULT) [string]
- Log file name. If set to DEFAULT, uses the name of the task and, if preceded by "!", overwrites the file if it exists. If set to NONE, no log file is created.
- (debug = no) [boolean yes|no]
- Diagnostic output is printed to the screen if set to yes.
- (history = yes) [boolean yes|no]
- Records task parameters in HISTORY.
EXAMPLES
- Detect flickering pixels from a Hitomi/SXI event file (OBSID 100050020) using a 5x5
background cell and a 3-count zero background threshold. The output
file contains the flickering pixels (not cleaned events, as was output
by cleansis) in the EVENTS extension.
searchflickpix infile=xa100050020xtd_p0100004b0_uf.evt outfile=xa100050020xtd_p0100004b0_flickering.evt cellsize=7 \
impfac=320 logprob1=-5.6 logprob2=-5.6 bthresh=3
- Same as above, but write out the cleaned (non-flickering) event list
in the output file.
searchflickpix infile=xa100050020xtd_p0100004b0_uf.evt outfile=xa100050020xtd_p0100004b0_clean.evt cellsize=7 \
impfac=320 logprob1=-5.6 logprob2=-5.6 bthresh=3 cleanimg=yes
- Same as example 1, but a Suzaku/XIS event file is used as input. Each
XIS event file contains data from one CCD, but with a larger chip size
(1024x1024).
searchflickpix infile=input_xis.evt outfile=output_flickering.evt cellsize=5 impfac=320 logprob1=-6.0 \
logprob2=-6.0 bthresh=3 chipcol=NONE xcol=DETX ycol=DETY
- Clean a generic event file to global fixed threshold of 20 counts per pixel.
Exclude PHA values greater than 1200.
searchflickpix infile=input.evt outfile=output_clean.evt cellsize=0 impfac=320 logprob1=-6.0 logprob2=-6.0 \
bthresh=20 chancol=PHA chanmin=0 chanmax=1200 cleanimg=yes
SEE ALSO
cleansis,
sxiflagpix
xtdflagpix
LAST MODIFIED
August 6, 2024