Access to raw data for PATT, UH, Japanese and Korean Observers
|Access to raw data for PATT, UH,
Japanese and/or Korean observers
Each night, WFCAM observers usually acquire very large amounts of
data (hundreds of Gbytes). We therefore
recommend that ALL users obtain their WFCAM data, fully
reduced, from the WFCAM Science Archive (see links on
previous page). If you are happy with this arrangement, you do not
need to read the rest of this page.
However, we recognise that in some exceptional cases there may be
good reason why observers need immediate access to raw data.
Unfortunately, due to staffing shortages and other priorities, UKIRT
scientists are NOT available to copy raw data to tape and/or
disk. Instead, this job is the
responsibility of the observer.
All observers wishing to
take raw data
away with them after their run must do one of the following:
- Observers bring their own USB-2 interfaced hard disks (or lap-top
sufficient disk space) and connect this to a summit machine at UKIRT.
Please read very carefully the notes on our
disk connection policy.
- Observers bring their own hard disks or lap-top and copy data
when they are back down in Hilo AFTER their run. Suitable workstations
are available. Obviously observers MUST allow sufficient time after
their run to copy their data; for a typical WFCAM run, we recommend at
least TWO DAYS to complete this process.
- Observers FTP their data to their home institute while at the
summit, or from Hilo after their run. Note, however, that ALL FTP jobs
from the summit MUST be completed or halted before the end of the
normal observing night, and certainly before the 14 hr summit rule is
broken. Observers should not expect UKIRT staff to stay late at the
telescope, just so they can FTP their data.
Finally, if you do want access to raw data, it is extremely
important that you discuss your plans with the Support
Scientist assigned to your project BEFORE you come to Hawaii. In
particular, observers should check the suitability of the laptop
and/or USB disk they intend to use for data transfer.
Data format and conversion
At the summit, raw WFCAM data are stored as Starlink-format NDF files
(.sdf extension). WFCAM writes four files at a time - one for each
array. Each NDF file is 17 Mb in size, and on a good night 2,000-3,000
observations (8,000-12,000 files) may be acquired!
These NDF files are converted to smaller, rice-compressed FITS
files (4 Gb each) at the end of the night, and are ftp-ed directly to
CASU in the U.K. These smaller FITS files are therefore only
available on disk AT THE SUMMIT for a period of about 30 hours after
they are created.
Raw NDF files are copied to Hilo on a daily basis. These data
remain on disk, in Hilo, for a period of 2-3 weeks.
Note that the raw data format - Starlink NDF (.sdf files)- will
need conversion to FITS at the PI's home institution, unless Starlink
is to be used for data processing. If you do not have a copy of the Starlink software package
an ISO image is available upon request. This software is supported for
UKIRT users by the JAC.