skip to content
 
NASA/GSFC & ESA GSFC ESA   INTEGRAL Helpdesk
  INTEGRAL What's New
  HEASARC Site Map
  NASA Homepage

HEASARC HOME INTEGRAL HOME ARCHIVE DATA ANALYSIS PROPOSALS & TOOLS STUDENTS / TEACHERS / PUBLIC
INTEGRAL U.S. Guest Observer Facility
ABOUT INTEGRAL GOF SERVICES STAFF USER'S COMMITTEE RELATED SITES

Low-Level Data Download Cookbook

1. Overview

  • Users should search for data products in the HEASARC mirror to the INTEGRAL Public Data Archive using Browse.

  • If one is interested in all public low-level data for a particular source then we recommend searching the INTEGRAL Science Window Catalog or the INTEGRAL Public Pointed Science Window Catalog>.

2. Searching either the INTEGRAL Science Window Data or INTEGRAL Public Pointed Science Window Data Catalog

  1. Select INTEGRAL from the list missions on the main Browse form.

  2. Click on the "Detailed Mission/Catalog Search" button at the top of the page. This gives you a list of available INTEGRAL catalogs.

  3. Select one of the INTEGRAL Science Window Data or INTEGRAL Public Pointed Science Window Catalogs in section 1 of this page.

  4. Fill in the relevant fields in section 2 of this page. Note that a source can be specified either by name or position. The choice of search radius is dependent on the instrument of interest. We recommend use of the following: SPI = 12 deg, IBIS = 10 deg, JEMX = 3 deg. If all instruments are of interest then choose a 12 degree search radius for the data download. The data can be limited to a smaller angular offset from the source at the beginning of data analysis when you choose which SCWs to include in your observation group. You can use Browse to help you choose the apropriate SCW subset when building your observation group.

  5. Click on "Specify Additional Parameters." This will display all the data columns in the specified catalog. Searching the INTEGRAL Science Window Data Catalog can return a large number of SCWs, particularly if the source is located near the Galactic Center. Some of these SCWs are of limited scientific use, e.g. slew data. For this reason we recommend retrieving only public, pointing data. This is done by setting the "Query Terms" for "scw type" and "status" equal to "pointing" and "public," respectively. In addition, you may want to specify a good time >0 for instruments of interest in order to ensure that those instruments were operational during each SCW. This is done by specifying values >0 for "good spi", "good isgri", etc. We also suggest sorting the search results by "scw id." This done by clicking on the circular button next to "scw id."

    Some of these selections have already been done for you if you have chosen to search the INTEGRAL Public Pointed Science Window Data Catalog. This catalog contains only public pointing data. When searching this catalog the user should still specify good times >0 for instruments of interest. Sorting the results by "scw id" is also recommended when searching this catalog.

  6. Click on "Start Search." This will produce a list of SCWs conforming to your search criteria.

  7. Select the SCWs of interest by clicking on the appropriate boxes on the right side of the table. Note that there may be multiple rows for the same SCW. This will occur if a particular SCW is for an amalgamated observation in which targets for multiple observers have been combined into one observation. When this occurs, the number of rows for the SCW will equal the number of amalgamated targets. Do not worry if you select the same SCW multiple times, e.g. by using the "Select All" button. Browse will know what to do.

  8. After selecting the SCWs of interest, the user should choose to produce a shell script that will download the low-level data products by clicking on "Create Download Script." We urge the user to select both the science wondow and the auxiliary data products. This will produce a set of commands that should be cut and pasted into a local shell script. Once the script is created, move it to the directory in which you wish to create your data tree.

  9. You now need to determine whether you need to download the 'ic', 'idx', and 'cat' data trees from the archive. If you have performed previous INTEGRAL data analyses these data may already exist on your local disks. If this is the case, then create soft links using 'ln -s' to these pre-existing data. WARNING: This should only be done if the existing 'ic', 'idx', and 'cat' data were from analyses containing data from spacecraft revolutions later than the current download

  10. If appropriate 'ic', 'idx', and 'cat' data do not already exist locally, then you should download these data from the HEASARC archive. This is achieved by using this shell script: ic-idx-cat_download.

  11. Run the download script(s) in the directory where you wish to create your data repository. It will create 'aux' and 'scw' subdirectories filled with newly downloaded data as well as 'ic', 'idx', and 'cat' directories if they were needed.

  12. Set the environmental variable REP_BASE_PROD to the directory containing these newly created directories.

  13. An ASCII list of SCWs suitable for the creation of an Observation Group (OG) can be derived from the data download script using this Perl script, make_og_list.


Last updated:Friday, 25-Feb-2011 13:13:29 EST .