How to copy a subset of INTEGRAL data
=====================================
(Volker Beckmann, Exploration of the Universe Division at NASA/GSFC)

Here I describe how to copy a subset of data from the ISDC. To do so, you need an existing INTEGRAL data set at your site, a user account at the ISDC, and the data rights to access the INTEGRAL data you want to copy. Even though this is very similar, I give the explanation for copying near-real-time (NRT) data, and for copying consolidated data (see further down).


Copy NRT data
=============

Create a new data directory:

mkdir data_rep_nrt
cd data_rep_nrt
setenv REP_BASE_PROD $PWD

Link the idx/ and ic/ directories of local consolidated data you have at your site:

ln -s /whereever/consolidated/ic ic
ln -s /whereever/consolidated/idx idx

Create an aux/ directory with the adp/ subdirectory:

mkdir aux
cd aux
mkdir adp

Link the org/ directory from your local consolidated data here:

ln -s /wherever/consolidated/aux/org org

and the ref/ directory in aux/adp/:

cd adp
ln -s /whereever/consolidated/aux/adp/ref ref

Now go back to the aux/adp/ directory:

cd ..

and copy the auxiliary data for the revolutions you want to copy from ISDC here:

scp -r your_username@isdcsf4.unige.ch:/isdc/pvphase/nrt/ops_1/aux/adp/XXXX.000 .

where XXXX is the revolution number (e.g. 0180). Now you have to copy the science window data:

cd $REP_BASE_PROD
mkdir scw
cd scw

If you want to copy a complete revolution:

scp -r your_username@isdcsf4.unige.ch:/isdc/pvphase/nrt/ops_1/scw/XXXX/ .

In case you want to download only part of the data:

mkdir XXXX
(where XXXX is the revolution number, e.g. 0180)

cd XXXX
scp -r your_username@isdcsf4.unige.ch:/isdc/pvphase/nrt/ops_1/scw/XXXX/rev.000 .

And now copy the science window directories here:

scp -r your_username@isdcsf4.unige.ch:/isdc/pvphase/nrt/ops_1/scw/XXXX/science_window_number .

NOTE! scp does not allow the wildcard ('*' or '?') when copying from another machine. BUT you can use the wildcard when you are on the system where to copy from, i.e. log onto ISDC, go to the scw/XXXX/ directory and use
scp -r science_window_* username@your_home_computer.wherever.it:.


Copy consolidated data
======================

Create a new data directory:

mkdir data_rep_cons
cd data_rep_cons
setenv REP_BASE_PROD $PWD

Link the idx/ and ic/ directories of local consolidated data you have at your site:

ln -s /whereever/consolidated/ic ic
ln -s /whereever/consolidated/idx idx

Create an aux/ directory with the adp/ subdirectory:

mkdir aux
cd aux
mkdir adp

Link the org/ directory from your local consolidated data here:

ln -s /wherever/consolidated/aux/org org

and the ref/ directory in aux/adp/:

cd adp
ln -s /whereever/consolidated/aux/adp/ref ref

Now go back to the aux/adp/ directory:

cd ..

and copy the auxiliary data for the revolutions you want to copy from ISDC here:

scp -r your_username@isdcsf4.unige.ch:/isdc/arc/rev_1/aux/adp/XXXX.001 .

where XXXX is the revolution number (e.g. 0180). Now you have to copy the science window data:

cd $REP_BASE_PROD
mkdir scw
cd scw

If you want to copy a complete revolution:

scp -r your_username@isdcsf4.unige.ch:/isdc/arc/rev_1/scw/XXXX/ .

In case you want to download only part of the data:

mkdir XXXX
(where XXXX is the revolution number, e.g. 0180)

cd XXXX
scp -r your_username@isdcsf4.unige.ch:/isdc/arc/rev_1/scw/XXXX/rev.000 .

And now copy the science window directories here:

scp -r your_username@isdcsf4.unige.ch:/isdc/arc/rev_1/scw/XXXX/science_window_number .

NOTE! scp does not allow the wildcard ('*' or '?') when copying from another machine. BUT you can use the wildcard when you are on the system where to copy from, i.e. log onto ISDC, go to the scw/XXXX/ directory and use
scp -r science_window_* username@your_home_computer.wherever.it:.


In case of questions and comments contact me:
beckmann@milkyway.gsfc.nasa.gov