NAME
ftcalc - Calculate values for a table column.
USAGE
ftcalc infile[ext] outfile column expression
DESCRIPTION
'ftcalc' is a general table calculator utility that computes values for
a table column based on an arbitrarily complex expression,
which may be a function of the value of other columns or keywords in
the table. The computed values are written into the specified column
in an output copy of the input file. A new column will be created and
appended to the table if the specified column does not already exist.
The expression is evaluated for each row of the table
and the result is written to the specified column. The expression
may evaluate to a numeric value (or a vector of values) or it
may evaluate to a character string, depending on the data type
of the output column.
See the 'calc_express' help file for a full description of the
allowed calculator expression syntax. In general, the calculator
expression may contain any of the following elements:
- Column and keyword names:
- The value of the column (in the current row) or the keyword will
be substituted into the expression. Precede keyword names with a
single pound sign, '#', if necessary to distinguish it from a column
with the same name. If the column or keyword name contains a space or
a character that could be interpreted as a mathematical operator,
enclose the name in dollar sign characters, as in the column $MAX-PHA$,
or the keyword #$DATE-OBS$. NOTE: column names that begin with the
letters 'b', 'o', or 'h' followed by numeric digits must be enclosed in
'$' characters, otherwise they will be interpreted as numeric constants
in binary, octal, or hex notation, respectively.
To use a table
entry in a row other than the current one, follow the column's name
with a row offset within curly braces. For example, 'PHA{-3}' will
evaluate to the value of column PHA, 3 rows above the row currently
being processed. One cannot specify an absolute row number, only a
relative offset. Rows that fall outside the table will be treated as
undefined, or NULLs.
To use a single element of a vector column within a calculator expression,
follow the column's name with a element number (beginning with 1 for
the first element) within
square brackets, as in 'PHAS[1] * 10'. Note that this expression
evaluates to a scalar quantity (a single number), whereas the expression
'PHAS * 10' evaluates to a vector that is created by
multiplying each element of the PHAS vector by 10.
- Mathematical operators:
- +, -, *, /, ** or ^ (exponentiation)
- Boolean operators in C or Fortran-type notation:
- .eq., ==, .ne., !=,
.lt., <, .le., <=, .gt., >, .ge., >=, .or., ||, .and., &&, .not., !,
and ~ (approximately equal, to within 1E-07)
- Math library functions:
- abs(x), cos(x), sin(x), tan(x), arccos(x), arcsin(x), arctan(x), arctan2(y,x),
cosh(x), sinh(x), tanh(x), round(x), floor(x), ceil(x)
exp(x), sqrt(x), log(x), log10(x), x%y (modulus), erf(x), erfc(x), gamma(x)
random() (returns random number >= 0 and < 1),
randomn() (returns Gaussian distribution with zero mean and unit
standard deviation),
randomp(x) (returns a Poisson random distribution whose expected
number of counts is X. X may be any positive real number of expected
counts, including fractional values, but the return value is an integer.)
min(x,y), max(x,y),
accum(x), seqdiff(x), angsep(ra1, dec1, ra2, dec2) (all in degrees).
- Numerical constants:
- Numeric values are assumed to be in decimal notation. Integer
constants may also be in hexidecimal (0x123f7), octal (0o12737) or
binary (0b1001010). Such integer constants are 32-bit integers.
The following predefined constants may also be
used: #pi (3.1415...), #deg (#pi/180), #e (2.7182...), #row
(substitutes the current row number into the expression). Two special
constants, #null and #snull, can be used for testing if the expression
value has an undefined numeric value or an undefined string value,
respectively.
- String constants:
- enclose string values in quotes, e.g., 'Crab', 'M101'
- Datatype casts to convert reals to integers or integers to reals:
- (int) x, (float) i
- Conditional expressions:
- 'b?x:y' where expression 'x' is evaluated if
'b' is TRUE (not equal to zero), otherwise expression 'y' is evaluated.
- Test for near equality:
- near(value1, value2, tolerance) returns 0 if
value1 and value2 differ by more than tolerance.
- Bit masks:
- Bit masks may be used in logical expressions, and be of arbitrary
length. Bit masks may be binary (b110001), octal (o44712) or hexidecimal
(h0f3D). The 'x' character represents a wild card at that position.
- Good time interval test:
- The gtifilter() function returns 1 if the time value
lies within one of the good time intervals, otherwise it returns 0.
Specifying 'gtifilter()' is equivalent to 'gtifilter("", TIME,
"*START*", "*STOP*")' and uses the GTI extension in the current FITS
file to filter the TIME column using the START and STOP columns in the
GTI extension. The gtifind() function takes the same arguments
as gtifilter() but returns the GTI entry number that brackets
the time sample instead of true/false (or returns -1 if no GTI
brackets the time sample).
- Good time interval overlap calculation:
- The gtioverlap() function returns the amount of overlap
time between a user requested time range and a GTI extension.
See calc_express for more
information.
- Spatial region test:
- This function returns 1 if the spatial position associated
with that row of the table is located within the region defined by
the specified region file. Specifying 'regfilter("region.reg", xpos, ypos)'
uses the xpos and ypos table columns (and any associated World Coordinate
System keywords) to determine the position, and the region file named 'region.reg'.
- Vector column operators:
- These funcions operate on a vector and return a scalar result:
MIN(V), MAX(V), AVERAGE(V), MEDIAN(V), SUM(V), STDDEV(V),
NELEM, and NVALID(V) (number of non-null values), NAXIS(V)
and NAXES(V,n).
See the 'calc_express' help file for more information.
PARAMETERS
- infile [filename]
- File name and optional extension name or number enclosed in square
brackets of the input table to be operated on (e.g.,
'file.fits[events]'). If an explicit extension is not specified then
the first table extension in the file that is not a GTI (Good Time
Interval) extension will be used. Additional filter specifiers can be
appended to the file name, also enclosed in square brackets, to create
a virtual input table as shown in some of the examples. The input file
is not modified and instead, the calculations are performed on the
output file copy.
- outfile [filename]
- Name of the output file that will contain the modified copy of the
input file. To overwrite a preexisting file with the same name, prefix
the name with an exclamation point '!' (or '\!' on the Unix command
line), or set the 'clobber' parameter = YES.
- column [string]
- Name of the column to which the calculated values will be written.
If the column does not already exist, a new column with this name
will be appended to the output table. If a vector column is specified,
and if the calculation evaluates
to a scalar quantity (not a vector), then the 'element' paramenter (described
below) may be used to specify which element of the vector should
be used to store the result.
- expression [string]
- The expression used to calculate the output column's values.
The expression can evaluate to any of the following data types:
boolean, integer, double, string, or bit. If a vector column
is used in the calculation expression, then the result will also be
a vector with the same length. A text file containing
the expression can be specified by preceding the filename with
'@', such as '@file.txt'. The expression in the file can be
arbitrarily complex and extend over multiple lines of the file. Lines
that begin with 2 slash characters ('//') will be ignored and may be
used to add comments to the file.
- (rows = "-") [string]
- A comma-separated list of rows or row ranges that will be operated
on (e.g., "10-20, 25, 30-40"). By default, all the rows in a table are
calculated.
- (element = 0) [integer]
- If the calculation evaluates
to a scalar quantity (not a vector), the result may be written
to a single element of an existing vector column by specifying the name
of the column with the 'column' parameter and the
element number (beginning with 1 for the first element) with
this 'element' parameter. In this case the 'rows' parameter
is ignored, and the calculation is performed on every row of the
table. In all other cases 'element' should
be set to the default value of 0.
- (tform = "") [string]
- Optional value string for the TFORM keyword which defines the
datatype of the output column, if a new column is being created. If
'tform' is blank, then a default datatype as determined by the input
expression will be used. The specified tform value must conform to
the rules specified in the FITS standard for the TFORMn keyword, which
depend on whether the table being operated on is a binary FITS table
or an ASCII FITS table. For binary tables, legal TFORMn datatype codes
are
L - Logical
X - Bit
B - Byte
I - 16-bit Integer
J - 32-bit Integer
A - Character string
E - Single precision floating point
D - Double precision floating point
The datatype code may be preceded by an integer vector length (usually 1).
Examples of legal TFORMn values for an ASCII table (where the first
number represents the column width, and the second number represents
the number of numeric digits after the decimal place) are:
A10 - Character string
I6 - Integer
F12.2 - Fixed decimal floating point
E12.5 - Exponential notation floating point
D16.9 - Double precision exponential notation
- (clobber = NO) [boolean]
- If outfile already exists, then setting "clobber = yes" will cause
it to be overwritten by the new file.
- (chatter = 1) [integer, 0 - 5]
- Controls the amount of informative text written to standard output.
Setting chatter = 5 will produce detailed diagnostic output, otherwise
this task normally does not write any output.
- (history = NO) [boolean]
- If history = YES, then a set of HISTORY keywords will be written
to the header of the output table to record the value of all the ftcalc
task parameters that were used to produce the output file.
EXAMPLES
Note that when commands are issued on the Unix command line, strings
containing special characters such as '[' or ']' must be enclosed in
single or double quotes.
1. Calculate the values for a column named AREA in the first
table extension of a the input file by taking the product of the
'Xlen' and 'Ylen' column values in each row. Write the result to
the new file 'outfile.fits'.
ftcalc input.fits outfile.fits AREA Xlen*Ylen
2. Compute a 3 point running mean of the values in the 'Rate' column
and write result to a single precision floating point column called
'Smooth'. A double precision column would have been created by default
if the 'tform' parameter had not been specified.
ftcalc input.fits outfile.fits Smooth '(Rate{-1}+Rate+Rate{1})/3.'
tform='1E'
3. First, create a virtual input file that contains only the X and Y
columns in the table, then compute the distance between a point whose X
and Y coordinates are defined by the XCENTER and YCENTER keywords, and
the positions defined by the X and Y column values in each row of the
table. Store the result in a column called 'Radius'.
ftcalc 'input.fits[col X;Y]' outfile.fits Radius
'sqrt( (X - #XCENTER)**2 + (Y - #YCENTER)**2 )'
4. Conditionally compute the 'Rate' column. If 'Counts' is greater than
zero then Rate is calculated by dividing the 'Counts' column
value by the 'EXPOSURE' keyword value. If the 'Counts' value is less than
or equal to zero, then the Rate value is set = 0.
ftcalc input.fits outfile.fits Rate 'Counts>0?Counts/#exposure:0.'
SEE ALSO
calc_express,
ftcopy,
ftselect,
ftsort,
filenames,
colfilter,
rowfilter,
binfilter
fv, the interactive FITS file editor, can also be used to calculate values
for table columns.
The design of this task is based on fcalc
in the ftools package and on the CXC dmtcalc task.
LAST MODIFIED
March 2002