How to Install a Calibration Database

Michael F. Corcoran
Code 662,
NASA/GSFC,
Greenbelt, MD20771

Lorraine Breedon
Code 668,
NASA/GSFC,
Greenbelt, MD20771

Version: 2009 Aug 04

SUMMARY

This document describes the steps required to install and build a local Calibration Database (CALDB) system: construction of the filesystem, transfer of the desired calibration files from the HEASARC calibration database and installation of the calibration files into the `Calibration Index Files' (CIFs). Once set-up, the continued maintenance/management of a local CALDB is covered in OGIP Calibration Memo CAL/GEN/92-015.

This document is available in both PDF and HTML formats.

LOG OF SIGNIFICANT CHANGES

Release	Sections Changed	Brief Notes
Date
1997 Sept 30	All	First Public Version
2005 Dec 08	All	General Update
2007 Feb 27	§§ 2.1; 3	refer to `https://heasarc` rather than `ftp://legacy` in URLs
2009 Aug 04	§ 3	note that tar files are listed on Supported Missions page
2018 06 11	all	changed http:// links to https://

1 Introduction
2 Setting up the CALDB Environment
2.1 Downloading and Installing CALDB setup files
2.2 CALDB Setup
3 Downloading and Installing CALDB Calibration Data from the HEASARC CALDB
4 Including Non-HEASARC-Supported Missions in your CALDB
5 Additional Information

1 Introduction

This document describes how to set up a local copy of the HEASARC's calibration database (CALDB) system.

The High Energy Astrophysics Science Archive Research Center (HEASARC) Calibration Database (CALDB) is a modular system for storing and retrieving calibration data on an instrument-by-instrument basis. It is designed to be flexible and installable on any Unix workstation, desktop or laptop. The CALDB is integrated with the HEAsoft software package in that the HEAsoft tasks can access appropriate calibration data from the CALDB with little input from the user.

Installing all or part of the CALDB is a matter of setting up the CALDB environment variables and downloading and untarring the calibration data. These steps are described below.

Note that calibration data which is archived in the HEASARC CALDB is directly available to all HEAsoft users with an internet connection via Remote Access. Remote Access means that users have access to all data in the HEASARC CALDB without having to download the CALDB data to their local disk. More information on setting up Remote Access to the HEASARC CALDB is available at https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_remote_access.html.

2 Setting up the `CALDB` Environment

Define the environment variable $CALDB to point to the directory where the calibration data will be located (we'll call this the top-level CALDB directory). For example, if the CALDB is being installed under the directory /local/home/user/caldb you need to define the $CALDB environment variable as:

% setenv CALDB /local/home/user/caldb

2.1 Downloading and Installing `CALDB` setup files

Change your current working directory to $CALDB and download the file
https://heasarc.gsfc.nasa.gov/FTP/caldb/software/tools/caldb_setup_files.tar.Z
either using your web browser or using a download utility like wget or curl. Then uncompress and untar the file in your $CALDB directory. For example:

% cd $CALDB
% wget https://heasarc.gsfc.nasa.gov/FTP/caldb/software/tools/caldb_setup_files.tar.Z
--14:02:20--  https://heasarc.gsfc.nasa.gov/FTP/caldb/software/tools/caldb_setup_files.tar.Z
           => `caldb_setup_files.tar.Z'
Resolving heasarc.gsfc.nasa.gov... 128.183.240.223
Connecting to heasarc.gsfc.nasa.gov[128.183.240.223]:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 8,172 [application/x-tar]

100%[========================================================================>] 8,172         --.--K/s             

14:02:20 (12.37 MB/s) - `caldb_setup_files.tar.Z' saved [8172/8172]

% tar -zxvf caldb_setup_files.tar.Z

This will create two $CALDB configuration files, which are:

$CALDB/software/tools/alias_config.fits
$CALDB/software/tools/caldb.config

and a CALDB initialization file, $CALDB/software/tools/caldbinit.csh (for unix C-shell users) or $CALDB/software/tools/caldbinit.sh (for unix Bourne shell users). Note that there is also an initialization file (caldbinit_iraf.unix) for users who might want to install the CALDB under IRAF¹, and even an initialization file for VMS users. The IRAF and VMS initialization files are supplied for largely historical reasons, and are currently deprecated.

Open the appropriate intialization file (caldbinit.csh or caldbinit.sh, depending respectively on whether you are using C-shell or Bourne shell) in an editor (like vi or pico for example), and edit the definition of the $CALDB environment variable to point to your top-level CALDB directory. In our example the first line of caldbinit.csh should read:

setenv CALDB /local/home/user/caldb

2.2 `CALDB` Setup

Before accessing calibration data (using the HEAsoft analysis tools, for example) you need to set up the CALDB environment variables by sourcing the appropriate CALDB initialization file:

source $CALDB/software/tools/caldbinit.csh

for c-shell users, for example. If you include this line (or source $CALDB/software/tools/caldbinit.sh for Bourne shell users) in your .login file, then the CALDB will be available to you every time you log in.

3 Downloading and Installing `CALDB` Calibration Data from the HEASARC `CALDB`

This section describes how to download calibration data from the HEASARC for a particular mission and instrument, and how to store the data so the CALDB software tools can access it.

While we discuss downloading data for a single mission and instrument, analysis of data from a single instrument sometimes requires calibration data from another instrument - for example, constructing an effective area for a particular instrument on a given mission might require not only the quantum efficiency for the instrument, but also calibration information describing the telescope performance, or information concerning other components which may be in the optical path. Please consult the relevant mission pages, data analysis guides, guest investigator facilities or science support centers for more information.

Follow these steps to download and install the CALDB for a particular mission/instrument combination from the HEASARC CALDB:

Change directory to $CALDB. If you haven't defined the environment variable $CALDB, you should do so, as specified above.
For each < mission > / < instrument > combination (for example the rosat/pspc) download the tar files listed on https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_supported_missions.html. CALDB tar files are usually single files of the form
```
goodfiles_<mission>_<instrument>_tar.Z
```
although this single file may be split into multiple compressed tar files if the total volume of calibration files is large (typically more than a few gigabytes). You should refer to the CALDB Supported Mission web page, https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_supported_missions.html, for the list of CALDB tar files for each mission and instrument.
You can download the tar files via web access from https://heasarc.gsfc.nasa.gov/FTP/caldb/data. For example, if you want to install the CALDB for the ROSAT PSPC, you should download the file /caldb/data/rosat/pspc/goodfiles_rosat_pspc.tar.Z.

Uncompress and untar each tar file you downloaded to $CALDB. This will create the

$CALDB/data/< mission>/< instrument>

directory tree. For example:

% cd $CALDB
% uncompress goodfiles_rosat_pspc.tar.Z
% tar -xvf goodfiles_rosat_pspc.tar
% rm -i goodfiles_rosat_pspc.tar

Or:

% cd $CALDB
% tar -zxvf goodfiles_suzaku_xis_bcf.tar.Z
% tar -zxvf goodfiles_suzaku_xis_cpf.tar.Z
% tar -zxvf goodfiles_suzaku_xis_index.tar.Z
% rm -i goodfiles_suzaku_xis_*.tar.Z

This will create the CALDB directory structure for each mission and instrument with the currently-available calibration data.

Run the FTOOL caldbinfo to check that your installation for this mission and instrument is correct. Use the command-line syntax

% caldbinfo INST <mission> <instrument>

where INST tells caldbinfo to use the INSTRUMENT mode. For the SWIFT XRT:

% caldbinfo INST SWIFT XRT
  ** caldbinfo 1.0.1
  ... Local CALDB appears to be set-up & accessible
  ** caldbinfo 1.0.1 completed successfully

4 Including Non-HEASARC-Supported Missions in your `CALDB`

This section is for users who have set up the CALDB structure as described above, and who want to add their own missions and instruments to their CALDB.

Make sure the CALDB environment variables are defined as described in Section 2.2.
Check the file $CALDB/software/tools/caldb.config for the presence of the < mission > / < instrument > combination in which you're currently interested. For example, if you want to add your own response file for the BBXRT2 XRS to your CALDB, then you could do the following:
```
% grep BBXRT2 $CALDB/software/tools/caldb.config | grep XRS
```
If the < mission > / < instrument > combination does exist in $CALDB/software/tools/caldb.config, you can go on to step 3. If grep does not find the specified < mission > / < instrument > string in the caldb.config file you'll need to add an entry for this particular mission/instrument combination. To do this for a particular < mis > / < inst > , you need to add the following lines to caldb.config:
```
#<mis> 
<mis>  <inst> CALDB data/<mis>/<inst> caldb.indx CALDB data/<mis>/<inst>
```
For example to add an entry for an instrument called the xrs on a mission called BBXRT2, you should add the following lines:
```
# BBXRT2
BBXRT2   XRS CALDB data/bbxrt2/xrs caldb.indx  CALDB  data/bbxrt2/xrs
```
to your caldb.config file.
Create the appropriate directory structure data/ < mission > / < instrument > under $CALDB if it doesn't already exist. For the example above:
```
      % cd $CALDB
      % mkdir data
      % mkdir data/bbxrt2
      % mkdir data/bbxrt2/xrs
```
Put the calibration data in the directory $CALDB/data/ < mission > / < instrument > /. For example
```
      % cp /local/home/xrs_970929.rsp $CALDB/data/bbxrt2/xrs/.
```
If the file $CALDB/data/ < mission > / < instrument > /caldb.indx doesn't exist create it with the FTOOL crcif:
```
      % cd $CALDB/data/bbxrt2/xrs/
      % crcif
```

Change directory to the appropriate $CALDB/data/ < mission > / < instrument > / directory and update the caldb.indx file with the FTOOL udcif:

      % cd $CALDB/data/bbxrt2/xrs/
      % udcif
      Name of file containing dataset[]bbxrt2.rsp 
      Name of Calibration Index File[] ../caldb.indx
      Dataset:     MATRIX
      Quality value for the dataset being entered[] 0

Check that the installation was successful using the FTOOL caldbinfo:

      % caldbinfo
      Mode (BASIC, INST, or FULL)[] INST
      Name of Mission[] BBXRT2
      Name of Instrument[] XRStth -utth -utth -u
       ** caldbinfo 1.0.1
       ... Local CALDB appears to be set-up & accessible
       ** caldbinfo 1.0.1 completed successfully

5 Additional Information

Users who are interested in generating calibration data in format which is compatible with the CALDB should see the web page, "Producing CALDB Data Files" at https://heasarc.gsfc.nasa.gov/docs/heasarc/caldb/caldb_producers.html.

More information on managing a CALDB is available from "How to Manage a Calibration Database" (calibration document CAL/GEN/92-015).

Footnotes:

¹the Image Reduction and Analysis Facility, http://iraf.noao.edu/

File translated from T_EX by T_TH, version 4.12.
On 11 Jun 2018, 18:10.

How to Install a Calibration Database

Michael F. Corcoran Code 662, NASA/GSFC, Greenbelt, MD20771 Lorraine Breedon Code 668, NASA/GSFC, Greenbelt, MD20771

SUMMARY

LOG OF SIGNIFICANT CHANGES

Contents

1 Introduction

2 Setting up the CALDB Environment

2.1 Downloading and Installing CALDB setup files

2.2 CALDB Setup

3 Downloading and Installing CALDB Calibration Data from the HEASARC CALDB

4 Including Non-HEASARC-Supported Missions in your CALDB