UKIRT
Show document only
UKIRT Home
Contact Info
OMP
UKIRT Staff wiki
Weather
Web Cameras
____________________

Survey Observing
Other Observing
Observing Tool Manual
Schedule
Applying For Time
Get UKIDSS Data
Get non-UKIDSS Data
Data Reduction
Acknowledge Us
Telescope
WFCAM
Cassegrain
Technical Reference
UKIRT Publications
Graphical Weather Server
EAO Safety Manual
UKIRT Waiver Form
TSS Priority Page
imaging_reduction_cookbook
Manual Reduction of Imaging Data Using KAPPA and CCDPACK



Lead Author: Watson Varricatt.

For any of this to work, you must first install the Starlink software collection, which is available from http://starlink.eao.hawaii.edu/starlink/. Links to detailed manuals for KAPPA and CCDPack are given at the bottom of the page.

Step I - Dark corrections

Run ORACDR on the raw data

Look at all the darks:

During the observation, just before a particular region of interest, if there was hard saturation on some objects in the field, they are going to leave latency on the array. If you take a dark after that without flushing the array, the darks will have bright residuals which are going to get subtracted from the images when you do the dark subtraction. This will also spoil the flats. If that is the case, see if you have darks taken with the same exposure time on the other objects, either before or after the object of interest. If those darks are good, use them. If there are good darks taken before and after the bad dark, use the one closest in time to the bad dark. If those are separated by nearly the same time interval before and after the bad dark, average them and use.

If there are no darks taken that night with the same exposure time as the faulty one, patch the latent images in the current dark using GAIA and use that. Patching like this on the dark has a problem that if there are some hot pixels in the same region as the latency, they may also go, which is not what we want. So use a patched dark, only if there is no good dark taken that night with the same exp time. While doing the patching, make sure to look at the position of the feature that you are patching, cross check the position of that with the previous frames and convince yourself that there was indeed a saturating object there!.

There will be an ascii file called index.dark in your ORAC_DATA_OUT directory. If you are using a different or modified dark, add the file name of this dark (without .sdf extension) to index.dark.

Step II

Re-reduce your object frames with the new dark - e.g.:
oracdr -list 10:36 -calib dark=newdark
If the original darks were perfect without any latency, you do not have to do this !

Now, again, be careful about using the median combined sky-flats produced by the pipeline if there are very bright objects in the field which give hard-saturation. They can leave considerable residuals up to 3-4 subsequent frames (sometimes even more). If we median combine these frames and produce sky flats, when there are residuals, these very bright objects will not get removed completely and produce residuals in the flats. If that happens, patch the residuals out before median combining them to produce sky flats.

If you have only one jitter pattern on a object, then proceed to step III

If you have more than one jitter patterns on an object with the same filter, then you can improve upon the quality of the flat fielding to some extent by averaging the different flats on the same object taken with the same filter. For eg. if you are observing an object in K and you are doing 3 X 9 point jitters, ORACDR with generate flats from these three sets separately and generates mosaics

If you look at the files, you will find separate flats for each sets - eg:

 flat_K98_groupno.sdf
flat_K98_groupno_c1.sdf
flat_K98_groupno_c2.sdf

average these three flats and make a combined flat - eg:

 flat_K98_groupno_all.sdf

modify the "index.flat" file in your ORAC_DATA_OUT directory and add the name of this flat to the list of the flats.

now re-reduce the data using oracdr with the new dark and flat. e.g.:

 oracdr -list 10:36 -calib dark=newdark,flat=flat_K98_groupno_all

Step III

Caution: For crowded fields, there may be overlapping objects or nebulosity, which will again not get properly removed when you median combine. Also, if your exposure time is too low (which often happens with bright objects, or even with moderate exposure times using narrow-band filters), the count level on the sky may be too low and will be dominated by the noise. In these situations, use a separate flat taken using the same filter with sufficient exposure time as close in time to the target observation as possible and re-reduce the data using ORACDR

Step IV

Use the flat fielded individual frames to construct your final mosaic using kappa and ccdpack.

To load kappa, type

 kappa

from your terminal. (If you need help on any of the KAPPA utilities, type kaphelp and give the utility name when prompted for.)  put the names of the all the files that you want to check into a file (inp) (without the .sdf extension) usually the default coordinates produced by the oracdr are 'sky' coordinates. Some of the tasks that we describe below are to be done in 'pixel. coordinates. For that type the kappa command

 wcsframe

when it asks for the the input file, type

 NDF - The NDF to be modified /@test/ > 
type
^inp

to let it know you want the whole inp list processed.

next question is

 FRAME - Required co-ordinate Frame /'sky'/ >

if the default coming is pixel, press the enter key. If not type pixel.

Now all the frames will have pixel coordinates assigned as default. Before doing the mosaicking, you have to check individual frames and remove any imperfections on the image for getting a good mosaic, some of these are common for all the frames and can be efficitntly removed using Kappa.

If your imaging is done with ufti, one obvious problem is that the central 2 and the last 2 columns are usually not well flat fielded and so you will have to blank them off. You might want to blank of the central row too in UFTI.

This can be done by using the kappa task zaplin

you can make zaplin run on a large number of files

  • list all the file that you want to zap (without the sdf extension) to a file "inp"
  • list the names under which the above files are to be modified and stored to a file "out"
  • list the range in row and column of the above files to be zapped to a file "temp"

Suppose you want to blank off columns 511-512, 1023-1024 and row 512 in all the files. create a file "temp" with any editor with the contents:

 510.5 511.5 c
1022.5 1023.5 c
511.5 511.5 l

now run zaplin on these files:

 zaplin zaptype=b ^inp ^out mode=file colin=temp

Now you have to do the mosaicking. This is done using four separate tasks of the starlink package CCDPACK - pairndf, register,tranndf, makemosaic

for using the CCDPACK utilities, first invoke CCDPACK by typing

 ccdpack

on the command line

(If you need help on any of the CCDPACK utilities, type ccdhelp and give the utility name when promted for.)

There are 3 tasks you run to produce the final mosaic :

pairndf: to match the position of the images. This produces a set of .DAT files which tells the relative positions of the centroids of the objects to be matched when you do the mosaicking

Put all the files that you want to mosaic into a file (without the .sdf extension)

 pairndf
IN - List of NDFs /@^inp3/ >^file

now ccdpack will draw a window which is divided into 2 columns. all your files will be listed on the left of the left column and on the right of your right column

step A

Pick one file among the list, with which you like to match the rest of the files. On the left widow, click on the name of that particular file. That image will get displayed on the window. Adjust the contrast of that so that the point sources can be seen well.

step B

Select one of the rest of the images in the right winow… that gets displayed, adjust its contrast. On the right window, click on one of the common stars in both the frames, bring the mouse cursor on the star and left click. Now you can drag that image … drag it and bring it to the top of the same star on the left window and release the mouse. Now you can see the images over-plotted with the stars in the overpapping region nearly matched. Now move your mouse around in the overlapping region and left click on some of the common stars; markers will appear on them. (Right-click on any marked star will remove the selection if you selected a wrong object). Try to select sufficient stars spread throughout the overlapping region if available… this depends on the type of fit that you want to do.

After you have selected and marked sufficient stars, press "done" at the bottom of the wind. Now the image will disappear from the right window and the names of this pair of files will get boldfaced in the file list. Keep the same image in the left and select another one in the right window and again match them as before. Repeat this for all the remaining files. When all the reset of the files are matched with the one in the left window, pairndf task will exit, after creating a set of .DAT files which contain the relative positions of the images.

Now you have to run the task register which will determine the transformation between the images from the determined object centroids. You can give the same file name as input and select the order of transformation (1-6)..go for higher orders only if you have sufficient number of overlapping objects marked in all the images. you can give the same file name as input as ^file. It will look for the .DAT files in the same directory, get the positions and match the centroids.

 USEWCS - Offsets are in Current coordinate frame? /TRUE/ >
INLIST - Names of position list files /@^file/ >

Run the task tranndf and give the same file name as above as input. This will do the transformation between the image pairs. Tranndf will ask for the names under which the transformed images are to be stored.

Next step is doing the final mosaicking. Normally, we should be able to go ahead and mosaic the transformed images. -but you have to take care while doing it on UFTI narrow band images… especially, if you have images taken with the H2 (2.122 micron) or Br Gamma filter of UFTI, if there are bright objects in the field, you will get ghosts (probably arising out of some internal reflections) which move around the frame when we jitter. You will be able to identify them, if you subtract one transformed frame from the next. Patch them from the transformed images, if needed be, before doing the mosaicking.

Now we can produce the mosaics using the makemos task of ccdpack. You can mosaic them either using mean or median combining. Do zero corrections if required .. this will take care of slight fluctuations in the background.

 makemos method=mean(or median) zero

MAKEMOS
=======
IN - List of input NDFs /@^inp2/ >^inpfilelist
OUT - Output mosaic NDF >name of mosaicked image

Patching

In Gaia, from the pull down menu, select "image analysis" and then "patch images"

In the menu that comes up, default is for patching with a gaia-defined sky region around the region to be patched. Go with that option in less crowded regions. In extremely crowded regions, you might want to define your own replacement-regions. Go to options and click on "fit background using annulus" to unselect that.

(if you find that you clicked patch and it overdid the job, click "undo", then again either change either the position, size or shape of the patch box and try again).

Documents:


Contact: Tom Kerr. Updated: Mon Oct 19 14:05:37 HST 2015

Return to top ^