NAME

batclean - read in a DPI and clean out a fit to the diffuse background

USAGE

batclean infile outfile incatalog [detmask] [options]

DESCRIPTION

This tool fits a simple background or source plus background model to a detector plane image. The code contains a default background model which is described below. The user can also specify one or more background models to be fit using the bkgmodel parameter. The output file can be either the cleaned focal plane (default), the fit to the detector plane or the original input file. The fitting is performed as a linear combination of background components, via singular value decomposition.

The tool can also, optionally, balance the detector plane either before or after or instead of cleaning. See below under the "balance" parameter for more details.

This code reads in a Detector Plane Image (DPI) file as well as an optional detector quality mask file. If no detector mask file is provided, then all detectors with non-zero data are included in the fit.

Source Cleaning

Since BAT is a coded mask instrument, bright sources can contribute counts to a large portion of the detector. These source introduce their own 'coding noise' which is approximately 1% of the source intensity. By modeling and subtracting the contributions of bright sources, the coding noise can be reduced significantly (Although the effects cannot be completely removed.)

If one or more sources are to be cleaned from the DPI (default), then set srcclean='YES' and enter an input catalog ('incatalog'). All sources with significance greater than 'snrclean' will be modeled and cleaned. batclean which uses a forward projection to create a model template for each source; the fit coefficient is the normalization (intensity) of the source.

The input catalog must contain columns "IMX," "IMY," "CATNUM," (or "SOURCE_ID"), "NAME," and "SNR." The tool reads the source position in BAT image coordinates from this file and uses the source position to forward project a model of the source flux onto the focal plane.

Background Cleaning

The purpose of background cleaning is to model and remove smoothly varying gradients in the detector plane image. These gradients are typically dominated by the cosmic X-ray background, which peaks at the center of the detector plane image. If these trends are not removed, the coded mask image will contain increased 'pattern noise' which will reduce the sensitivity to real sources.

By default, a simple background model is used, which contains linear and quadratic spatial gradients. It has the following parameters:

The code creates a background template maps based on the model above (which can be optionally output using the backexp parameter). In addition to the simple model, the user can supply custom model template files, and these will be fitted with a linear normalization component. The custom model templates must be in the same format as a DPI (i.e. 286 x 173 image extensions), one image extension per template.

The tool then uses a singular value decomposition method to fit a linear combination of all source and background templates to the counts data of the DPI. The fit coefficients are then used to create a model background map.

Balancing

batclean is also capable of "balancing" the detector plane counts. Balancing involves subtracting DC biases from certain combinations of detectors which are thought to have systematic offsets. The BAT flight software itself also does balancing on-board Swift. The most obvious case of this is to subtract the global-average counts from all detectors, but there are other possible combinations. The purpose of balancing is to further reduce pattern noise due to systematic features in the detector counts (for example, each detector module is biased slightly differently). Since these structures do not correspond to the shadowgrams of real sources, they only contribute noise, and can be subtracted without reducing sensitivity to real sources. If a detector quality mask is given (via 'detmask'), then only non-masked detectors are included in the balancing.

If any balancing operation is enabled via the 'balance' keyword parameter, the task will also balance the entire array, or subsets of the array, as specified in the table below. The balancing that is requested is given by supplied a comma-separated list of any or all of the following values. The balance is performed in the order in which the values are listed. Balancing is not commutative, so order is significant.

Allowed balance values:

Output File

The output file will contain detector plane maps with the result of the cleaning operation, i.e. a series of DPIs, one for each input DPI. The actual meaning of the data values depends on the 'outversion' parameter:

Depending on the 'maskfit' parameter, the output data values may or may not be masked by the quality map. batclean writes the fit coefficients to the output file as keywords, as follows:

PARAMETERS

infile [filename]
Input file name. The name of the input DPI file. This is expected to be an image array with dimensions equal to the number of detector columns by the number of detector rows (set to 286 X 173). If there are multiple BAT_DPI image extensions then all are cleaned and multiple cleaned images are written to the output file.

outfile [filename]
Output file name. See above for description of output file contents, which depends on the 'outversion' parameter.

incatalog = "NONE" [filename]
Input source catalog. The format of the catalog file is that output by the BAT tool batcelldetect. See above for the format description. By default the first table extension is opened, unless an explicit extension is provided. If no source cleaning is requested, then set incatalog='NONE'. See also the 'srcclean' parameter.

(aperture = "CALDB:FLUX") [filename]
Aperture file. The default is the aperture file in the CALDB area. This parameter may be omitted or set to "NONE" if only background cleaning is required (i.e., the srcclean parameter set to "NO"). If several aperture types are available in the CALDB, the user can specify CALDB:apertype. Valid values of apertype are currently: FLUX or DETECTION, for apertures optimized for flux reproducibility or detection sensitivity, respectively.

(detmask = "NONE") [filename]
Name of a detector quality mask file. This should be an image file with the same dimensions as the focal plane map. A pixel value of 0 indicates the detector is enabled for imaging, and any other value indicates the detector should be ignored. A default value of 'NONE' implies all detectors are on, except for the BAT detector gap regions. It is strongly recommended to use a detector mask file if one is available. The quality of the fit, especially to calibration data is greatly improved if only working detectors are included in the fits.

(wtmapin = "NONE") [filename]
Name of an input error map. This is used to provide weights for the fitting in case one is cleaning a DPI which does not have Poisson statistics (such as an already cleaned DPI). The typical sequence would be to create an error map using the wtmapout parameter on the first cleaning and then read this error map in on the second cleaning. If an already cleaned image is cleaned without an input error map than uniform weighting is used.

(wtmapout = "NONE") [filename]
Name of an output error map. If a filename is specified then the tool writes out a DPI containing the calculated weights for each detector. This can be used on subsequent cleanings as the input wtmapin error map.

(backexp = "NONE") [filename]
If the name of a file is given here, then the background model will be output to this file. The format is a binary table with one column for each model parameter. The data is written in DPH format, or as a 286 X 173 element array, one number per detector. This table gives the model which was fit to the data, not the actual fit to the data. The final column in the table is the covariance matrix from the fit.

(bat_z = 0) [real]
This is the z-component of the distance to the source in BAT_Z coordinates (the vertical distance of the source from the origin of the BAT coordinate system). This parameter is ignored if srcclean = "NO." If the incatalog contains a BAT_Z keyword, then the value of the keyword is used and the value of the bat_z parameter is not used. Note that the default value of 0 corresponds to a source at infinity.

(bkgmodel = "simple") [string]
This optional parameter indicates the background model to be fitted to the input counts. The possible values are: "simple" (simple background model as described above); "none" (no background fit); or user-specified custom templates. For user-specified custom templates, bkgmodel should be a list of background templates, which can be: a comma-separated list of one or more template filenames, or a filename such as "@filename.txt" where the specified file contains a list of model files separated by carriage returns. By default the 'simple' model is combined with user-specified templates, unless the special string "useronly" is specified as one of the filenames.

(srcclean = "YES") [boolean]
Determines whether sources are cleaned from the input DPI or only background. If this parameter is "YES," then an incatalog and aperture file must be supplied.

(cleansnr = 6.0) [real]
Only sources with a signal-to-noise ratio as read from the incatalog higher than this value are cleaned from the DPI. If no sources have a signal-to-noise ratio above the cleansnr value, then only the background will be cleaned.

(snrcol = "SNR") [string]
Name of catalog column to be used with 'cleansnr' for signal to noise thresholding. By default it is "SNR", but any column can be used to trigger cleaning.

(ignore = "UNKNOWN") [string]
Unless this parameter is set to "NONE" or "none," the cleaning code will not clean any source in incatalog whose NAME matches the ignore string. The user can either specify a particular source name, the strings "IGNORE" or "NONE", or the name of a file (preceded by "@"). If a file name is supplied, then the file must contain a list of sources to be ignored, once source per line. Note that source names to be ignored must match exactly (including spaces, capitalization, etc.) the names in the catalog.

(balance = "none") [string]
Comma-separated list of balancing operations to perform, in order, or 'none' to indicate no balancing. See the 'balancing' section above for more information.

(balfirst = "NO") [boolean]
If 'no' then balancing is done after cleaning. However, by setting this parameter to 'yes' then balancing can be done before cleaning.

(maskfit = "NO") [boolean]
If this parameter is set to "YES" and outversion="fit" then the output file contains the fit parameters for all detectors even those masked using a detmask. The quality map is always applied during the fitting and balancing operations regardless of the setting of 'maskfit'. 'maskfit' only alters how the data values in the output file are written.

(leadedge = "NO") [boolean]
If this parameter is set to "YES," then the leading edge detectors (those on the two edges of detector module sides facing toward the source) are fit separately from the rest of the detectors. Since the current forward projection code does not correctly handle these edge detectors, setting this parameter does not significantly affect the overall clean.

(eff_edge = 0.2) [float]
The depth of edge (cm) to consider effective. Range from 0.03 to 0.2. Normally this

(corrections = "none") [string]
Comma-separated list of corrections to apply to the mask weighting, or "none" if no corrections are to be applied. Note that corrections are only applied to the forward projection for sources. This parameter has no effect if only the background is to be cleaned.

Possible correction types are: "flatfield" (basic flat fielding correction accounting for both cosine-type and r-squared-type effects); "nearfield" (for ground testing results where the source is at a finite distance -- requires bat_z != 0); "cosine" (cosine effects of off-axis illumination only); "rsquare" (r-squared effects only); "opaque" (activates a new algorithm for calculating the mask transmission function); "sides" (effects of sides of detectors XXX not implemented).

(outversion = "cleaned") [string]
This optional parameter indicates the meaning of the data values in the output file. See the "Output File" section above for more information. Choice of outversion parameter dictates what value is written to the HDUCLAS2 keyword. Possible values are:
(clobber = NO) [boolean]
If the output file already exists, then setting "clobber = yes" will cause it to be overwritten.

(chatter = 1) [integer, 0 - 5]
Controls the amount of informative text written to standard output. Setting chatter = 4 or higher will produce detailed diagnostic output; chatter = 2 prints out a basic diagnostic message. The default is to produce no printed output.

(history = NO) [boolean]
If history = YES, then a set of HISTORY keywords will be written to the header of the specified HDU in the output file to record the value of all the ftcopy task parameters that were used to produce the output file.

EXAMPLES

Note that when commands are issued on the Unix command line, strings containing special characters such as '[' or ']' must be enclosed in single or double quotes.

The following examples illustrate ways in which batclean can be used.

1. Fit the simple background model to a DPI with a detector mask file. Output the cleaned DPI and a background exposure map. 

    batclean input.dpi cleaned.dpi detmask=bat.mask backexp=backexp2.dph 
srcclean=NO
2. Fit a set of source models along with a simple background model to a DPI, using the default aperture file. Output the fit to the focal plane instead of the cleaned focal plane. A flat-fielding correction is applied in the forward projection. 

    batclean input.dpi fit.dpi catalog.tbl detmask=bat.mask outversion="fit"
corrections="flatfield"
3. Fit two user-supplied background models in addition to the hard-coded model. 

 
   batclean input.dpi cleaned.dpi detmask=bat.mask bkgmodel=model1.dpi,model2.dpi
4. Same example, except the user gives the two model files as lines in the text file "models.lst." 

   batclean input.dpi cleaned.dpi detmask=bat.mask bkgmodel=@models.lst
5. Balancing is performed after cleaning, applying the Module and InOut balancings. 

    batclean input.dpi cleaned.dpi detmask=bat.mask balance=Module,InOut

BUGS

SEE ALSO

batcelldetect, batmaskwtimg

REFERENCES

Based on Netlib code http://www.netlib.org/c/numcomp-free-c 

Name        : svd.c.Z (8704 bytes)
Where       : in pub/C-numanal on ftp.usc.edu
Description : SVD based on pascal from J. C. Nash book
Author      : Bryant Marks (bryant@sioux.stanford.edu)
              Brian Collett (bcollett@hamilton.edu)
Version     : 14 April 1993

Statistics for low count rates from K. Mighell (ApJ 518, p. 380, 1999)

LAST MODIFIED

Dec 2008