The ASCA Archive

N.E. White (HEASARC, ASCA), E.V. Gotthelf (ASCA),

& N. Oliverson (NSSDC)

The ASCA archive opened on 15 November 1994 and is accessed via the online service of the High Energy Astrophysics Science Archive Research Center, HEASARC. This article describes the archive contents and how to retrieve the data. The analysis of ASCA data is fully described in the document "An abc Guide to ASCA Analysis" which is available in the anonymous ftp area on heasarc.gsfc.nasa.gov under asca/docs.

1 The ASCA Archive

The following propriertary periods apply to the ASCA data:

• PV data taken between April 1993 and approximately December 1993 is made public 18 months from the observation date.
  
• US and joint US/Jp data (i.e. with a US PPI) that was delivered prior May 2 1994 will be public 1 year after that date.
  
• US and joint US/Jp data delivered after May 16 1994 will be public 1 year + 2 weeks from dispatch to the PI.
  
• Jp and joint Jp/US data (with a Japanese PPI) will be delivered 18 months + 2 weeks from dispatch to the PI.
  

A list of the currently available data, and release dates for future data is available from the BROWSE database (http://heasarc.gsfc.nasa.gov/W3Browse. , database ASCAPUBLIC).

A text dump of the archive contents is also available by anonymous FTP at heasarc.gsfc.nasa.gov under asca/docs/ascapublic.txt.

The data will initially be released in 1 month batches, around the 15th of each month, with the first batch released on 15 November 1994. As the archive population becomes routine, the releases will be made more often, on a weekly basis.

1.1 Accessing the ASCA data

Public ASCA data may be retrieved in any one of the following ways:

• By anonymous FTP at heasarc.gsfc.nasa.gov under the asca/data/[sequence number] directories (see Section 3, below).
  
• By logging into heasarc.gsfc.nasa.gov, username XRAY, BROWSE database ASCALOG. Data products can be located and extracted via the BROWSE database interface (Section 3)
  
• By the World Wide Web at http://heasarc.gsfc.nasa.gov/W3Browse/. Select the mission ASCA and then the database ascapublic. Data products can be located and extracted via this interface (Section 3).
  
• By an e-mail request to the NASA's NSSDC archive service (Section 4), Address: ndadsa.gsfc.nasa.gov, Subject: REQUEST ASCA DATA_TYPE, Message: SEQUENCE_NUMBER.
  

In all cases the same basic file structure (described in Section 2, below) is used to store the data.

Intrepid explorers of the ASCA public archive should note that the PV data, especially those taken in the first few months, are not always optimized for scientific investigation of the targeted object. Many observations were made for instrument calibration and contain frequent mode changes and non-standard parameter setting for which instrument responses are not available. The observation modes and instrument settings during the latter PV and AO phase are more stable. If in doubt, please ask the ASCA GOF.

2 Archive File Structure

ASCA data is most conveniently stored (and labeled) by the 8-digit SEQUENCE NUMBER which is assigned to each pointed observation and generally consists of a day's worth of consecutive data.

The data for a particular observation sequence is quite large (up to 200 Mbytes or more) and thus the data is divided into six manageable sub-directories (TELEM, AUX, RAW, SCREENED, UNSCREENED, and PRODUCT). For a typical analysis not all files are necessary. The following is summary of each sub-directory and what is needed for a typical analysis:

• TELEM: Contains the telemetry (FRF) files which makes up each sequence (typically one file, occasionally more). These are by far the largest files and these data are NOT required; they are here for completeness.

• RAW: Contains the fundamental FITS format event list files, with one file per mode change per instrument. These are the basic data, which are required for a complete analysis of the data. You'll want this if you plan to start the analysis from scratch, using ascascreen and tkascascreen. But before downloading this relatively large number of files, its wise to take a quick look at one of the processed products.

• AUX: Contains the calibration and filter files. The filter files (extension .mkf) are necessary to time filtering the data. To make any rescreening of the data the .mkf file must be retrieved. For barycentric timing correction the frf.orbit file is required. The other files contain calibration information used during the processing and are typically not required for a standard analysis.

• UNSCREENED: Contains the modally merged event list files for each instrument, but not screened to remove bad data (no time filter). This is a convenient starting point for basic data analysis, especially if you want to rescreen the data using different cleaning criteria.

• SCREENED: Contains the modally merged event list files - screened using current standard filter criteria. These files are useful for quick analysis to extract standard spectra, images, and, light curves.

• PRODUCTS: contains the product catalog, information, and contents files. A good getting started point with basic observation catalog information, pre-digested quick look images, and summary plots.

The individual ASCA data files are archived in FITS format and two basic naming conventions are used to identify them:

a) The Telemetry FRF (First Reduction Files) convention:


 root:  ft930408_0413_0603 = ft[OBS-DATE]_[START(UT)]_[STOP(UT)]


b)  AD (Modally merged files):


 root:      ad10004010 = ad[SEQUENCE-NUMBER]


c) Appendage (added on for both a and b):


              s100312h = [s1] [003] [12] [h]


                       = [instrument]  [index] [inst. mode] [bitrate]


                       = [s0/s1/g2/g3] [0-999]    [code]     [l/m/h]

                                                        ||

       Some common modes:                     01 = SIS FAINT

                                              02 = SIS BRIGHT

                                              03 = SIS FAST

                                              70 = GIS PH

d) Extensions

The following extensions are commonly used to denote different types of files:

• .fits indicate raw FITS files containing photon event data.
  
• .unf are unscreened FITS events files.
  
• .evt are screened FITS events files.
  
• .img are FITS images in sky coordinates.
  
• .detimg are FITS images in detector coordinates.
  
• .gif are gif format images. The file names are appended with _detimg or _img to indicate if the images are sky or detector coordinates.
  
• .mkf are the "makefilter" filter files, used to screen the data.
  
• .txt contain ASCII files.
  
• .log are the output from the processing logs.
  
• .ps are postscript output from the quicklook analysis.
  
• .cat are the observation catalogs, which list all the modes and detector configurations. These can be used via xselect to choose the files of interest.

3 HEASARC Archive Access

ASCA data archived at the High Energy Astrophysics Science Archive (HEASARC) is available directly via anonymous FTP at heasarc.gsfc.nasa.gov under asca/data/rev1/[seq_num]. A database interface is also available in the same account by logging into legacy and typing browse and selecting the ASCALOG database. A World Wide Web (WWW) interface to browse on legacy is also available.

3.1 Anonymous FTP

Data can retrieve by anonymous FTP in the usual way (% = UNIX prompt}:

%ftp heasarc.gsfc.nasa.gov
(username: anonymous, password: your_e-mail_address)
%cd asca/data/rev1/seq_num/sub_dir
%binary
%[prompt]
%[m]get file_name[s]
%quit

To find the sequence number of an observation, use BROWSE (see below) or the file in asca/docs/ascapublic.txt.

Example 1: Getting the cleaned products.

mget asca/data/rev1/60000000/product/*.gif    <-- get all the gif images
mget asca/data/rev1/60000000/product/*.evt    <-- get the merged cleaned
						  event list data 
mget asca/data/rev1/60000000/product/*.img    <-- get all the sky FITS
images

Example 2: Getting all the screened event files

mget asca/data/rev1/60000000/screened/*       <-- get all the screened event
 						  files plus the observation
						  catalogs.

mget asca/data/rev1/60000000/unscreened/*     <-- get all the unscreened event
						  files, and the observation
						  catalogs
mget asca/data/rev1/60000000/aux/*.mkf.Z      <-- get the mkf
files

mget asca/data/rev1/60000000/raw/*s0*h.fits.Z <-- get all the raw sis0
						  high bit rate data 
mget asca/data/rev1/60000000/raw/*g2*m.fits.Z <-- get all the raw gis2
						  medium bit rate data 
mget asca/data/rev1/60000000/aux/*.mkf.Z      <-- get all the mkf files

3.2 The HEASARC Browse

The HEASARC BROWSE facility can be used to find out which data are available, and also to download the data. Browse can be accessed either via remote login or via the WWW.

3.2.2 World Wide Web

To access the WWW browse interface go to

http://heasarc.gsfc.nasa.gov/W3Browse/

click on ASCA, and select Archival Data. Then simply type the name or coordinates of the object. The hyperlinks then lead to the data, if its available. Data can be downloaded either file by file, or as a tar file. The gifs can be displayed, to give a quick look at the data.

3.3 NDADS Access

ASCA data are being archived at the National Space Science Data Center on optical disk (WORM) platters. The ASCA/NDADS archive can be accessed by submitting e-mail request or via the World Wide Web (WWW) to NDADS. Requests for ASCA data on tape will also be supported.

3.3.1 ASCA Data on NDADS

ASCA data in the NSSDC Data Archive and Distribution System, NDADS are organized by the 8-digit observation sequence number and sub-directory structure described above. The sub-directories are referred to here as "data-types" and are archived on NDADS in unix tar containers. When untar-ed, the appropriate directory structure is reproduced [seq_num/data-type]. Each tar file also contains the original bill-of-lading (BOL) file delivered to NSSDC with each data-type by the ASCA project. The BOL file contains a list of the individual files within each data-type and sequence number and several other fields used to ingest the ASCA data to NDADS (see data-type `LOG' below). All BOL files are in ascii format. In addition, some processing logs and tape contents lists have also been archived in ASCII format.

Also archived on NDADS are six logs (listings) of the files within each tar container. These logs are derived from the original BOL's from the project and contain the following fields; original file name delivered by the project, compressed byte count, decompressed byte count, original release date as supplied by the ASCA project, an archive flag used to ingest the data, and the object name. These logs are archived as ASCII files separate from the tar container files.

3.3.2 NDADS E-Mail Requests

ASCA data can be requested electronically through the Automated Retrieval Mail System (ARMS). For general information on NDADS send the following e-mail message (nothing needs to be entered in the body of the message):

  To:       NDADSA::ARCHIVES (or archives@ndadsa.gsfc.nasa.gov)
  Subject:  HELP

To request specific information on the NDADS ASCA archive send the following e-mail message (nothing needs to be entered in the body of the message):

  To:       NDADSA::ARCHIVES
  Subject:  HOLDINGS ASCA

NDADS requests for ASCA data should include the NDADS entry-ids (8-digit observation sequence numbers), the data-type, and the project name (ASCA). A listing of the ASCA/NDADS data-type's and sample file names are given in Table 4.1. An e-mail request to NDADS for data should use the following syntax:

  To:       NDADSA::ARCHIVES
  Subject:  REQUEST Project_Name Data_Type

For example, a request for the "SCREENED" tar file for sequence number 12345678 would look like:

  To:       NDADSA::ARCHIVES  (or archives@ndadsa.gsfc.nasa.gov)
  Subject:  REQUEST ASCA SCREENED

Then in the body of the e-mail message enter:

12345678
<end of message>

More than one sequence number can be requested in a single archive request. However, multiple requests must be made to retrieve more than one data-type.

3.3.3 Downloading Requested Data

After receipt of the e-mail message sent by the user, ARMS (Automated Retrieval Mail System) will extract the data to the NDADS staging area or send the requested data to your home computer if remote data transfer was requested. ARMS sends an e-mail message notifying the user that the data are available and includes a list of all the files retrieved. The archive user can then retrieve the files via anonymous ftp from ndadsa.gsfc.nasa.gov.

The basic steps are:

a) connect to NDADSA via anonymous ftp. Give your e-mail address as the password.
b) change directory to data_dist.asca
c) check the directory to make sure the expected files are there.
d) to ensure the proper transmission of the data, tar files should be sent in binary mode and the BOL files should be sent in ASCII mode.
e) get the desired files.

3.3.4 Remote transfer of Data

ASCA data requests can be transferred directly to the requesters home computer. This method avoids the need to place the data on the NDADS staging disk, but requires the requester to supply remote node information. This feature is available either through the e-mail interface or the WWW. To request data be ftp-ed to your home computer send the following e-mail message:

  To:       NDADSA::ARCHIVES
  Subject:  REQUEST Project_Name Data_Type  [Data_format  [Remote_access]]

The syntax for the remote access information is:

   Node_Name|User_Name|Password|Destination_Directory

Note that when requesting remote node data transfer, the Data_format field must also be specified. Initially, for ASCA, the Data_format field should be set to "None". Further details are contained in the NDADS help information and/or via the WWW.

3.3.5 NDADS World Wide Web Interface

The ASCA data on NDADS and related information can be accessed via the WWW using Mosaic. Requests for ASCA data utilize the Mosaic forms capability.

The URL for the ASCA NSSDC page is:

http://nssdca.gsfc.nasa.gov/asca/asca_nssdc.html

3.3.6 Restoring the Files on Your (UNIX) Computer

Place the ASCA tar files in the directory you wish. Then cd to this directory and enter:

tar xvf asca_SequenceNum_data-type.tar

e.g.:

tar xvf asca_12345678_screened.tar

This will create a subdirectory named by the sequence number and under this directory another subdirectory named by the `data-type' which, in turn, contains all the component files. To decompress all the files cd to the data-type subdirectory, e.g.:

cd 12345678 cd screened uncompress *.Z

3.3.7 Tape Requests

Individuals who need ASCA data on magnetic tape should submit their requests to the NSSDC/CRUSO office (address below). Requesters can receive the data on 4mm, 8mm or 9-track tape. Note that there is a significant charge for off-line media but that this charge may be waived for NASA-funded researchers. Contact the CRUSO office for the latest price lists. Note that off-line requests for data will take substantially more time than direct requests to NDADS, so if possible, requesters are urged to use one of the electronic request methods.

3.3.8 VMS Requests

ASCA researchers using the VMS operating system who experience problems with detarring and/or decompressing the ASCA tar files should contact Dr. Nancy Oliversen at the address below for assistance.

     NDADS 

     Data-type   Component files                  Sample file names

 AUX         ASCA_nnnnnnnn_AUX.tar            ASCA_12345678_AUX.tar
 PRODUCT     ASCA_nnnnnnnn_PRODUCT.tar        ASCA_12345678_PRODUCT.tar
 RAW         ASCA_nnnnnnnn_RAW.tar            ASCA_12345678_RAW.tar 
 SCREENED    ASCA_nnnnnnnn_SCREENED.tar       ASCA_12345678_SCREENED.tar
 TELEM       ASCA_nnnnnnnn_TELEM.tar          ASCA_12345678_TELEM.tar
 UNSCREENED  ASCA_nnnnnnnn_UNSCREENED.tar     ASCA_12345678_UNSCREENED.tar

 LOG         ascii log of files in each tar container.


	     ASCA_nnnnnnnn_RAW.BOL            ASCA_12345678_RAW.BOL
	     ASCA_nnnnnnnn_TELEM.BOL          ASCA_12345678_TELEM.BOL
             ASCA_nnnnnnnn_SCREENED.BOL       ASCA_12345678_SCREENED.BOL
             ASCA_nnnnnnnn_UNSCREENED.BOL     ASCA_12345678_UNSCREENED.BOL
             ASCA_nnnnnnnn_AUX.BOL            ASCA_12345678_AUX.BOL
	     ASCA_nnnnnnnn_PRODUCT.BOL        ASCA_12345678_PRODUCT.BOL


where nnnnnnnnn = 8 digit observation sequence number

If you have any questions, comments, suggestions or problems using the ASCA archive please send e-mail to ascahelp@athena.gsfc.nasa.gov, or contact Eric Gotthelf directly at 301-286-1587.

Users are welcome to visit the ASCA GOF to analyze both ASCA archival and guest data. To do so please contact Karen Smale at ksmale@lheavx.gsfc.nasa.gov or 301-286-7612 to book the facility.

For information regarding the ASCA NDADS archive and obtaining tape copies of data contact Dr. Nancy Oliversen (noliversen@nssdca.gsfc.nasa.gov or 301-286-1598). Tape copies can also be directly requested from NSSDC/CRUSO via request@nssdca.gsfc.nasa.gov or 301-286-6695.

5 Acknowledgements

The creation of the ASCA archive has been a joint effort by the ASCA GOF in the Laboratory for High Energy Astrophysics, and the ASCA Processing Team in the Astrophysics Data Facility (ADF). We would like to thank Richard Fink, Don Jennings, Ed Seufert, Ron Watson and Rich Pisarski for their efforts to make sure the archive was released on time. We also thank the other ASCA GOF members for their contributions and, of course, the entire ASCA team in Japan and the US for making this mission so successful.

Proceed to the next article

Return to the previous article

Select another article