Cookbook for reducing IR Images data using DoCIRIM/DoIRTF

F.M. Walter
June 1997
updated March 2000

Table of Contents

  1. Introduction
  2. Basics
  3. Basic Image Processing
  4. Correcting the Astrometry
  5. Standard Star Photometry
  6. The Aperture Correction File
  7. Aperture Photometry
  8. Photometry Reference Files
  9. Automated post-processing
  10. Ancillary Software
  11. The Software
  12. Limitations

I. Introduction

This software was designed for the purpose of permitting the user to reduce IR images obtained with the CTIO and IRTF IR cameras, and to undertake basic photometric analysis of the data.

This software is written in IDL, and runs under IDL version 5+. It has evolved to the point where it is fairly robust, the images look good, and I trust the photometry.

There are two drivers:

The only substantial differences have to do with the different keywords in the FITS headers.

Return to top.

II. The Basics

It is assumed that If you did not do all these things, you need to write your own software.

II.a Data file naming convention:
The raw data must be in .FITS files, and be named ROOTxxx.fits (ROOTxxx.A for IRTF data), where ROOT is some common root name and xxx is a sequence of numbers. For ease in the subsequent reductions, all the images for a particular observation should be numbered sequentially, e.g., root036,root037,root038, etc.

The flat files should be named jflat, hflat, and kflat (all .fits files). The file names must be lowercase on UNIX/LINUX systems. If these files are not found, a dummy flat file (a 256X256 array set to 1.0) is created.

II.b Raster patterns:
DoCIRIM deals with raster patterns by reading the telescope offsets from the FITS header (or the absolute positions from the IRTF header). For CIRIM data, the default pattern uses 6 images and the standard star pattern uses 4. DoIRTF expect a five-element pattern. Use the NOBS keyword to specify any other value. By default, there must be NOBS observations at each of K, H, and J. The order of the observations is unimportant. If the data were not taken in this manner, you can still process the data band-by-band using the KONLY, HONLY, or JONLY keywords.

If the data image numbers are non-sequential, use the IMAGE keyword to specify the numbers. Such data must be processed by individual bands. NOBS is determined from the size of the IMAGE array.

To first order the images are shifted by the offsets specified in the FITS header. Then they are cross correlated. If you do nothing, an image will appear and you will be asked to use the cursor to mark a source on which the images will be co-aligned. If you do not want to do this, you can use the RECTIFY keyword. /RECTIFY (or RECTIFY=1) tells the program to cross-correlate on the center of the image. This often works, but may give problems if the source is weak or there are lots of uncleaned cosmic rays. RECTIFY=[x,y] specifies that the cross-correlations are centered at pixel [x,y]. By default, once you specify the region for the K image, the same region is used for the J and H images. To override this and
a.) mark a source in each band interactively, set /MARKALL.
b.) mark the source in each individual image, use the SETOFFSET keyword.

Return to top.

III. Basic Image Processing

For CIRIM data, the basic calling sequence is:
DoCIRIM,root,im0
where root is the root name and im0 is the number of the first image to be processed. A third (optional) parameter returns the output data array.

The analogous program for IRTF/NSFCAM data is called DoIRTF.

DoCIRIM/DoIRTF has a large number of options specified by keywords. These include: 

DEBUG:             If set, the program stops a couple of times so you can
                   check intermediate results
DIREC:             name of raw data directory if different from current. This
                   is useful if you want to write the reduced data to a
                   new directory. The data are written to the current directory.
EXACT:             The program assumes that IM0 is the start of the nobs image
                   sequence. If you specify JONLY, DoCIRIM will start at
                   image IM0+2nobs. Specify /EXACT to force DoCIRIM/DoIRTF
                   to begin processing with image IM0.
HCRSIG:            high CR sigma cut, def=15. After shifting the images, the
                   median and sigma images are produced. The sigma image is
                   smoothed with a 5x5 median filter. Then the individual
                   images are compared to the median image, and points
                   discrepant by more than SIGMA standard deviations are
                   zeroed. This can lead to points being excluded from the
                   cores of bright sources, especially if the seeing varies or 
                   images are trailed. This will have an adverse affect on the
                   photometry. To mitigate against this, points in the sigma
                   image with a value exceeding HCRSIG times the median of that
                   image are not deleted, on the assumption that high points 
                   in the sigma image are bright sources.
IMAGES:            list of input images. Supercedes IM0. Should be used with
                   one of the /xONLY keywords. Use this in the case of a
                   non-sequential list of observations.
                   If the images were taken at different times, the central
                   coordinates may have shifted: use the SETOFFSET keyword.
JONLY,HONLY,KONLY: set one of these keywords to process 1 of the 3 filters.
                   The default is to do all 3 bands for a given target
LABEL:             label to append to output FITS file
LOG:               set to direct output to log file, def=DOCIRIM.LOG'
MARKALL:           set to specify the cross-correlation region for each
                   band. By default, the region selected for the first 
                   band is used for the others.
MAXSHIFT:          set to then maximum permissible shift, in pixels. The default
                   is 20*SHIFT pixels. If the cross-correlation shift exceeds
                   MAXSHIFT it is set to zero. Use this together with the
                   SETOFFSET keyword if you are having problems cross-correlating
                   on a faint object.
MEANDISPLAY:       set to display mean image; default=median image
NEWOBJ:            optional output file name
NODISPLAY:         set to skip TV display of final image
NOFILTER:          set to negate median filtering.
NOFITS:            set to skip writing .FITS output
NOPS:              if not set, postscript plot will be generated
RECTIFY:           2-element vector with coords of bright star for image
                   cross-correlation, /RECTIFY designates center, default is
                   interactive
SETOFFSET:         if set, you will be prsented with a composite of the images.
                   On these, mark (in sequential order) a star. The relative
                   offsets override the offsets in the header. Use this in the
                   case where there are large telescope drifts. You should 
                   use this when you get warnings of Xaxis or Yaxis fit
                   failures, or when the image comes up with multiple images.
                   The cursor positions are used: they are not centroided.
SHIFT:             set to 2 or 3 to expand cross correlation range
SHOW:              set to show centroid fits for image cross-correlation
SIGMA:             level for median filter cutoff; def=50, 1 for none
STANDARD:          set to use standard sequence. This writes a fifth image,
                   the unshifted, coadded image, to the output .FITS file.
                   It also appends the image number to the output file
                   name. Setting this in DoCIRIM sets NOBS to 4.
WARN:              set to stop on warning messages in CIRIM_SHIFT.

To run DoCIRIM, go to the directory you want the output data to be written to, start IDL, and then type DoCIRIM,root,obs1. Type DoCIRIM,/help for brief on-line help.

III.a Reduced data output files:
The output is a single .FITS file. The name of the file is taken from the OBJECT line of the .FITS header, and the band is appended. Thus a typical output name will be OBJECTX_K.FITS for the observation in the K band. For standard stars, because they are typically observed many times, the image number is also appended, so a typical output file name will be HD123456_200_K.FITS, if the observation sequence starts with file ROOT200.FITS. Use the NEWOBJ keyword to specify a new filename

The procedure CIRIM_OBJ strips the object line of all objectionable characters prior to generating the file name. Depending on what's in the object field, this can lead to gibberish, or a not-very-useful name. If you would rather specify the output file name, use the NEWOBJ keyword.

The output file contains 4 images (5 for a standard star observation) in the form of a nXmX4 (or 5) array.

The procedure RCIRIM reads and displays these files. Use the /AV or /MD keywords to select the average or median image.

The function GET_CIRIM can be used to extract and display individual images.

To set up the astrometric coordinate system, use SET_COORDS,/CIRIM,hh where hh is the fits header. This is automatically set in RCIRIM, GET_CIRIM, CIRIM_APPHOT, and CIRIM_POS.

Return to top.

IV. Correcting the Astrometry

The coordinates written to the FITS header by the TCS are approximate. To place the coordinates on the USNO A2.0 reference frame, use CIRIM_POS. The calling sequence is CIRIM_POS,root.

You will need to have a copy of the USNO A2.0 CDs, and will need to have device CDR defined to your CD drive.

You will be asked to mark first a star in the image (J band, if doing all 3 bands), and then the corresponding USNO star. The program will shift the central coordinates, centroid all the USNO star positions (and throw out the ones it doesn't detect), and reset the plate scale. If only one USNO star exists, the plate scale is not reset.

Invariably, there will be USNO A2.0 stars that turn out to be close visual pairs, or which have significant proper motion. At present, there is no way to exclude such stars from the fit. If there are enough "good" stars, then errors from one or two bad cases will be minimized.

Specify the minimum magnitude (in either the B or R bands) for the astrometric comparisons with the MAG keyword. If fewer than MINSTARS stars are found, MAG is reset to 22. If more than MAXSTARS stars are found, only the MAXSTARS brightest stars (from the brightest of the two bands), are used. MINSTARS and MAXSTARS are keywords which default to 7 and 25, respectively.

CIRIM_POS changes the RA, DEC, and plate scale fields in the header, and rewrites the .FITS file, unless the /NOWRITE keyword is set. The original values are saved in the *-ORIG keywords. By default, CIRIM_POS loops through the 3 bands, but the /KONLY, /HONLY, and /JONLY keywords will restrict the analysis to a single band. Unless you specify the /MARKALL keyword, you will identify stars only on the first image. CIRIM_POS resets the common astrometry grid.

Return to top.

V. Standard Star Photometry

To do the standard solution for photometry, use CIRIM_STD and CIRIM_EXT.

CIRIM_STD reads in the standards one at a time, but uses all observations of each standard. It used aperture photometry to extract the count rate. Note that the .FITS files read by CIRIM_STD must all be of the form root_obs_band.fits, where root is the name of the star, and obs and band are added in DoCIRIM.

The star name root must match one of the names of the stars listed in the file CIRIM_STDS.DAT. The disk location of this file is given by the envirionment variable CIRIMSTDS. Currently these standards include all the Elias (CIT) and FS (UKIRT) standards. If you observed different standards, feel free to edit the data file, adding K, H-K, and J-K along with the star name.

CIRIM_STD produces a text file called root.std. A typical .std file follows: 

      1.75127       55720.0      236.286       0 21 02 1997 00 07 53.8
      1.74075       61386.8      247.816       1 21 02 1997 00 09 18.9
      1.72960       36893.8      192.219       2 21 02 1997 00 10 50.4
      1.36363       56789.8      238.548       0 21 02 1997 01 36 12.4
      1.36066       62929.4      251.170       1 21 02 1997 01 37 34.0
      1.35743       38594.7      196.621       2 21 02 1997 01 39 03.5
      1.31959       58852.7      242.562       0 21 02 1997 03 39 55.6
      -1.32154       57987.4      241.112       1 21 02 1997 03 41 18.0
      1.32355       39104.1      197.925       2 21 02 1997 03 42 40.1
      1.68420       57903.8      240.896       0 21 02 1997 05 23 12.6
      1.69332       61640.9      248.449       1 21 02 1997 05 24 35.6
      1.70434       38723.8      196.794       2 21 02 1997 05 26 12.4

        (1)           (2)          (3)        (4)      (5)
   .STD file contents: column (1): air mass
                    column (2): net counts extracted
                    column (3): uncertainty on net counts extracted
                    column (4): band (0=K, 1=H, 2=J)
                    column (5): UT day, month, year, hour, minute, second
     note the negative air mass on one entry. CIRIM_EXT ignores entries with
     negative air masses. You can edit this file to remove questionable points.

CIRIM_EXT looks for all the *.std files reads them in, and does the photometric solution. There are a number of options for excluding bad data points, and a number of plot options, set using keywords. Among these are

      ALL:    set to include all data, including negative air masses.
      AMFIT:  fit air mass variations only
      FORCE:  force standard AM solution, and solve for time variation.
      IGNORE: number(s) of standards to ignore. The numbers of the standards
              found are listed by the program.
      NONUMBER: set for clean plot, otherwise points are labeled by standard
              number.
      ORDER:  order of polynomial temporal fit (requires /FORCE)
      REFDATA: name of solution file, def=CIRIM_EXT
      SAVESOL: set to write photometric solution to CIRIM_EXT.SOLUTION
      SHOW:    show existing solution only
      TFIT:    fit temporal variations only
      TIME:    set to plot versus time, default plots versus air mass.
      WIN:     plot window number

CIRIM_EXT transforms the UKIRT FS standard colors to the CIT scale (the Elias standards), so the solution should be on the CIT scale. Use the /RAWCOL keyword to override this transformation.

CIRIM_EXT produces a file called CIRIM_EXT.SOLUTION, which is used by CIRIM_APPHOT to do photometrically-calibrated aperture photometry. A typical CIRIM_EXT.SOLUTION file follows: 

       2
0 20.000 0.0548  0.171   0.674  -0.080 0.0     0.013    0.333    -0.002    0.035
1 20.000 0.0254 -0.157   0.571  -0.040 0.0    -0.018    0.335     0.002    0.035
2 20.000 0.0472 -0.332   0.722  -0.100 0.0     0.039    0.356    -0.003    0.036

(1) (2)   (3)     (4)      (5)   (6)   (7)      (8)      (9)      (10)     (11)
The first line gives the number of terms in the polynomial fit to the temporal
variation.
column (1): band. 0=K, 1=H, 2=J
column (2): Reference magnitude
column (3): Standard deviation of the fit in magnitudes
columns (4,5): intercept B of fit (at 0 AM, 0 time) and its uncertainty
columns (6,7): Air mass terms and uncertainty; (7) = 0 if fit is forced
columns (8,9): linear term to temporal variation fit, and error
columns (10,11): second order term to temporal variation fit, and error

This file is read by CIRIM_APPHOT to determine the magnitude correction.

Return to top.

VI. The Aperture Correction File

The default aperture size for the standard stars is 20 pixels radius. Often you will use a smaller aperture radius to measure targets, because the S/N is generally better for a smaller aperture. The program IR_APCOR generates an aperture correction table by extracting the counts for a specified target at many aperture sizes, and ratioing them to the reference aperture.

The calling sequence is IR_APCOR,ROOT,REFAP=REFAP, where ROOT is the root name of the file to be measured, and the keyword REFAP is the size of the reference aperture (default=20). The output file is IR.APCOR.

This aperture correction, if it exists, is automatically used in CIRIM_APPHOT.

Return to top.

VII. Aperture Photometry

CIRIM_APPHOT does aperture photometry. It is identical to APPHOT, except that it measures the 3 bands and then converts the count rates to magnitudes. The calling sequence is:
CIRIM_APPHOT,file (do not specify the band: if the file is OBJECT_K.fits, then the file is OBJECT). Type CIRIM_APPHOT,/help for on-line help. 
   KEYWORDS:
     CLIP:     default=0.01. Controls look of plot. plots are scaled linearly,
               with the brightest and faintest CLIP fraction ignored.  
     CONT:     if set, the program loops until you stop it with the right
               mouse button.
     EXPAND:   expansion factor for image, default=1. EXPAND=2 gives you about
               a 600x600 image. It may look nice, but it can be slow.
     GAP:      width of gap between radius and annulus, default=5 pix
     IDALL:    set to mark source in each band, def= K only
     MAGS:     set to output magnitudes, def=colors.
     MARK:     if set, uses MARKS to mark sources from a catalog.
     NOCENTRD: if not set, source will be centroided. If set, you mark the
               position with the cursor.
     NSIG:     sigma for upper limits, def=1
     RADIUS:   extraction radius, default= 12 pixels
     REFDATA:  name of photometric solution file, default=cirim_ext.solution
     RCENT:    radius of centroiding box, default=7 
     KONLY, JONLY, HONLY: set to analyze single bands.
     UPDATE:   set to add magnitudes to .IRPHOT file

Note that the data are stored in units of counts/second, because the exposures are not uniform due to the rastering and due to exclusion of bad points. You must do aperture photometry in units of counts to get correct estimates of the errors. CIRIM_APPHOT does this, by reading the nominal exposure time from the header and converting the image to units of counts. If you use APPHOT to measure any data reduced using this package, you must use the keyword EXPOSURE in the FITS header to convert the counts/second array to counts.

Return to top.

VIII. Photometry Reference Files

The photometry reference files are the extinction solution file, produced by CIRIM_EXT, with a default name of cirim_ext.solution, and the aperture correction file, produced by IR_APCOR, with a default name of ir.apcor. CIRIM_APPHOT uses the following protocol in determining the appropriate photometry reference files.

The extinction solution

  1. A valid extinction solution passed via the REFDATA keyword.
  2. A valid file name contained in the FITS header keyword EXT_SOL
  3. cirim_ext.solution in the current directory.
  4. If none of these files are found, CIRIM_APPHOT quits.

The aperure solution

  1. A valid extinction solution passed via the APCORREF keyword.
  2. A valid file name contained in the FITS header keyword APCOR
  3. ir.apcor in the current directory.
  4. If none of these files are found, CIRIM_APPHOT continues, but no aperture correction is applied.
The procedure ADDEXTSOL can be used to add the EXT_SOL and APCOR keywords to the FITS file headers. This can be useful for data that is to be moved to another directory after processing, but prior to analysis. Note that if you use ADDEXTSOL, the two correction files must have the same root and the default extensions. Type ADDEXTSOL,/help for on-line help.

Return to top.

IX. Automated post-processing

Return to top.

X. Ancillary Software

I have written a few procedures to simplify some housekeeping tasks. Among these are:

XI. The Software

The software is available via html download from this web site. You will need the 3 gzipped tar files and the standard star data file.

Some of these procedures call routines in the IDL astronomy users library. You will need to have this in your IDL_PATH. If you do not have the IDL astronomy users library, you can get it from here.

Because I have modified up a few commonly-used routines (including readfits and gaussfit), you should place these procedures in your IDL_PATH prior to the IDL astronomy users library and the RSI IDL libraries. For example, I place my procedures in directories LIBS/UTIL, LIBS/CIRIM, etc., the IDL Astronomy Users Library in the ASTROLIB directory, and then set up my IDL_PATH as follows:
setenv IDL_PATH +~/LIBS:+/usr/local/rsi/idl_5.2/lib:+/usr/local/rsi/idl/lib:+~/ASTROLIB:+~/idl:

The software was most recently updated on 2000 April 26. Click here for a list of corrections and changes.

Return to top.

XII. Limitations

Software is never perfect. The nice thing about anything written in IDL is that you can go into the code and tinker with things. Feel free, and let me know what works.

If you want to examine variables, consider using the /STP and /DEBUG keywords. Setting STP=1 stops the program at the end of (but inside) a number of the major subroutines. Set STP=2 to stop more frequently. DEBUG has a similar effect. Among the known limitations are:

Return to top.

Thats all for now.

fwalter@astro.sunysb.edu

Prof. Frederick M. Walter
Astronomy Program
Dept. of Physics and Astronomy
State University of New York
Stony Brook, NY 11794-3800