TFIT User's Guide
Version 1.2 -- March 2008
This user’s guide is
focused on how to run the software and obtain improved results. For more detailed
discussion, consult Laidler et al., “TFIT: A Photometry Package Using
Prior Information for Mixed-Resolution Datasets,” PASP 119:1325.
- Basic
Principles
- Preparing
the data
- Running the pipeline
- Utility
scripts
- Examining the diagnostics
- Step
by Step
- Other
common modes of operation
Appendix 1: Glossary
Appendix 2: Parameter file
Appendix 3: Error messages
Appendix 4: Installation guide
Basic Principles
Given a high resolution image (HRI) and a low resolution
image (LRI) of the same galaxy, we can use the information from the HRI to
constrain our interpretation of the LRI.
More importantly, since galaxies don't occur in isolation, we can use the
information from HRI of several galaxies, close enough
on the sky to overlap in the LRI, to obtain more accurate photometry of these
galaxies in the LRI by simultaneously fitting the several HRIs
to the LRI.
We begin with a library of FITS files, each containing a single object, and a
corresponding catalog of xy positions and isophotal fluxes. These images have been cut out from a
high resolution image containing many objects which has been cataloged by a
tool such as SExtractor (Bertin
& Arnouts, 1996) that produces a segmentation map
or some other means by which to exclude pixels that don't belong to the object
in question from the cutout. The local sky background has been subtracted from
each cutout, and for convenience, each cutout has also been normalized to have
a total flux of 1. Each FITS file contains a WCS that accurately locates the
image on the sky.
We convolve each cutout with a convolution kernel selected to transform the HRI
to the LRI. If the HRI is sufficiently high resolution, it can be treated as
``perfect'', and the point spread function (PSF) of the LRI can be used as the
convolution kernel. Since the LRI is generally at a coarser pixel resolution
than the HRI, we rebin the convolved images to the
LRI pixelation. The resulting library
of FITS files, convolved to the spatial resolution and rebinned
to the pixel resolution of the LRI, are referred to as templates.
We now consider an area of the LRI that is large enough to contain all the
light produced by an object close to its center, but no larger. This area is
typically twice the FWZI (full width, zero intensity) size of the PSF, and is
likely to contain several objects. Such a region is referred to as a cell.
Using the aforementioned catalog of the high resolution image to guide us, we
select the templates for all objects which occur in this cell. We use their WCS
information to place them correctly with respect to the LRI. This allows us to
create a matrix equation
LRI = A0 + A1T1 + A2T2 + ... + AnTn
where the Ti are the templates for the n objects that occur in the region.
(Actually, Ti = Ti/Ei, where Ei
is the RMS error value for the relevant pixels
in the LRI.) We solve this equation to determine the best fit flux for each
object in the LRI, plus a background term for the cell.
The flux that is produced in this way is something like an isophotal
flux, but there is a subtlety if the bandpasses of
the LRI and HRI differ, because the isophotes are
defined by the templates, and are thus defined in the band of the HRI.
In addition to an output catalog, these fluxes are used to prepare a model image,
or collage, using the templates as a diagnostic product. The collage is
then subtracted from the LRI to produce a residual image. This residual image
is the primary diagnostic product. It can be inspected for accurate background
subtraction, PSF matching, and registration. Objects which appear in the LRI
but do not have corresponding detections in the HRI will also appear in this
residual image.
If, as is frequently the case, the LRI has astrometric
properties (such as distortion) not captured by the WCS CD matrix, this will be
evident from the residual image. In a final diagnostic step, the collage is
cross-correlated against the LRI in rectangular zones. New transfer kernels are
then created which incorporate this shift, and the
convolution and fitting stages can be repeated to obtain improved results.
Preparing the data
Requirements and constraints
TFIT requires some careful preparation of the data sets
before you attempt to run. Specifically these requirements are:
·
Both lores and
hires images must have a valid WCS specified in RA and Dec
·
The CRVALs of both
images must be identical values.
·
The CRPIXs of both
images must be half-integer values.
·
The images must not be rotated with respect to
each other.
·
The pixel scale of the lores
image must be an integer multiple of the pixel scale of the hires image.
·
The lores image should
be background-subtracted as accurately as possible.
You will also need to prepare at least one transfer kernel
suitable for convolving the hires image cutouts to produce templates that are psf-matched to the lores image.
Accuracy of this kernel will significantly affect the accuracy of the final
results. If the images have very different resolutions, one may use the PSF of
the lores image; otherwise, the PSF of the hires image
should be deconvolved from the lores
PSF. The resulting convolution kernel must be:
·
a FITS image with a pixel scale identical to the
pixel scale of the hires image
·
centered in its array
If your lores image has significant spatial variation in its PSF,
you may prepare multiple kernels, each in its own FITS file, along with a
lookup table defining the rectangular zones on the lores
image that correspond to each kernel.
Data inputs
To run TFIT, you need to have:
- a lores and hires image prepared as above. The data must
be located in the first image extension of the FITS file.
- a
FITS file containing the transfer kernel prepared as above.
- an
object catalog of the hires image created by SExtractor
or a similar program. Catalog columns must include:
- object id
- object x,y
- object xmin,xmax,ymin,ymax
- local background
- isophotal
flux
- a segmentation map of
the hires image created by SExtractor or a
similar program, where this image uses the same object IDs as specified
by the catalog
- an RMS map for the lores image - ideally, a source-weighted rms map, where each pixel value is
sqrt (sky_rms**2 + source_rms**2), and source_rms**2 = (flux in counts/sec)/(exptime*gain).
- optionally, a flag map
for the lores image, to exclude dubious pixels
from the templates and fitting process
The task parameter file
A sample task parameter file will be installed in the param/ directory. (It can also be found in the lib/
directory of the source tree.) The parameters are divided into sections, and
each section is annotated with explanations of the parameters. You should copy
this file into your work area and edit it appropriately.
An annotated version of the task parameter file is presented in an appendix to this document.
Many of the parameters are filenames, and many of these
files function as the output from one stage and the input to another stage. This must be well understood in order to
successfully re-run parts of the pipeline. The information is included in the annotations and also in the table below.
Running the pipeline
The TFIT pipeline is designed
to be run from the shell command line. It produces a great deal of output that is
useful mainly for debugging when something goes wrong, so the output should be
directed to a file. If installed according to the directions in this manual,
the command usually given is:
tfit taskparfile.param >& logfile.log &
Output data products
TFIT produces the following files:
- a
tfitcat file. This file contains one
line per object per fit.
- a
tfitcat_best file. This file contains
one line per object, and is generally the file of most interest.
- a
tfitcell file. This file contains one
line per cell, and includes the chi^2 value for the cell, the cell
background (if fit), and number of objects per cell.
- a
modelfile file. This file contains an
output collage of the best fit for each object. No background terms are
applied.
- a
resid_modelfile file. This file is
created by subtracting the output collage from the lores
image, and is the primary diagnostic product.
- a
ddiagfile.ps file. This is a postscript
file showing the measured shifts from the registration dance step.
- a
tfitcovar file. This file contains a
verbose listing of the covariance matrix for each cell, as well as the
chi^2 ellipsoid.
Utility scripts
In addition to the TFIT pipeline itself, the TFIT software
package provides several utility scripts that can be run to provide additional
diagnostic information.
applydance.py
This
script can be used to apply the results of a registration dance to a new
convolution kernel. It generates a new set of shifted kernels and a new lookup
table that can be used by a pass2 (multikernel) run. This
may be desirable if significant PSF improvements are made after the first pass
through the pipeline has already been run.
Usage:
applydance.py <name of new kernel file> <ddiags filename>
cell2region.py
This
script generates a DS9 style region file that can be used to visualize the
boundaries of the overlapping cells.
Usage:
cell2region.py <name of tfitcell file>
covar_proc.py
This script generates individual
FITS files containing the covariance matrix for each cell from the covariance
diagnostic file produced by the fitter. These files can be examined with any
FITS viewer, and are used by the add_covar_info.py script.
Usage:
covar_proc.py <tfitcovar file> <dirname for covar FITS files>
make_collage.py
This
script can be used to generate a collage (named collage.fits) from a
set of templates and a _best file, without rerunning the diags stage
and re-creating the _best file. It can be useful if the diags stage
crashed after creating the _best file but before successfully writing
the collage.
It can also be used without applying the
fitting results contained in the _best file, to generate a collage of
normalized templates. This is rarely but occasionally useful.
If the _best file is not supplied, the
collage will be made from all templates in the directory; if it is
supplied, the _best file will be used to select the objects.
Usage:
make_collage.py <template directory> <lores filename> [<tfitcat_best file>]
psf_collage.py
This
script can be used to generate two matching collages named
hrpsf_collage.fits and lrpsf_collage.fits. Both collages have the
fitted scale factor supplied; both are at the pixel resolution of the
hires image. The hrpsf_collage.fits file is created from the
unconvolved cutouts; the lrpsf_collage.fits file is created from the
convolved, but high-resolution, templates which may be optionally
created in the convolve stage.
The resulting pair of collages may be useful in PSF investigations.
Usage:
psf_collage.py <cutout directory> <hr_template directory> <hires filename> <tfitcat_best file>
set_covar_info.py
This script can be used to add the covariance index (in a
column named maxcvratio) to the "best"
file. The object id of the most influential object is also added (maxcvid), as is the signal to noise (sn). This information can be used when working with
the catalog to identify objects that are effectively isolated or whose
photometry is significantly affected by their neighbors. You must first run the
cover_proc.py
script to generate individual FITS files for the covariance matrices.
Usage:
set_covar_info.py <tfitcat_best file> <dirname for covar FITS files>
Examining the diagnostics
- Examine the residual image
with an image viewer such as DS9. Set the display scale to a narrow range
centered on zero, such as -0.05 to +0.05. Look for:
- a
"sea" of faint background objects where nothing appears on the lores image. This may indicate that the lores image is imperfectly background subtracted; you
may want to improve its background subtraction and repeat the fit and
diagnostic stages.
- "yin-yang"
patterns for the objects, indicating that there are registration
difficulties. You may want to run the registration dance, if you haven't
already, and proceed to a second pass of processing to compensate for the misregistration.
- hot
or cold cores in the residual images, indicating that the convolution
kernel is too broad or too narrow. You may wish to tinker with the kernel
and reprocess.
Note that objects that were
very bright in the hires image are unlikely to subtract well, if the template
is dominated by instrumental effects rather than intrinsic object morphology.
- You may wish to simultanously display the lores
image and the collage, and blink back and forth between them.
- Examine the plot of measured
shifts produced by the plotdance stage (if
performed). Look for a sensible pattern of measured shifts that
corresponds to the yin-yang patterns in the residual image, and check that
any interpolated shifts make sense.
- Plot the xy
positions of the objects in the final (best) catalog. Check that there are
no gaps in coverage that might result from errors in the input catalogs,
templates, or problems with some cells.
Step-by-step
The table below summarizes the actions performed within each
of the TFIT processing stages, and identifies their input and output products
by the associated keyword in the task parameter file. The filenames of some
output products are constructed from others; this is indicated by providing the
prefix or suffix in <angle brackets>. Square brackets indicate optional products that may be produced depending on the task parameters.
Note that the psflookup
file is an output from the dance stage, and an input for the convolve stage
only when the multikernels
option is set to true. The posframe parameter determined whether the pixel coordinates in the psflookup file are in hires or lores coordinates. When using a file produced by the dance stage, posframe should be set to lores.
|
Stage
|
Substages
|
Input files
|
Output files
|
|
(before beginning)
|
Prepare datasets
Catalog HRI
|
|
hirescat
hiresseg
|
|
cutout
|
Cut out objects from HRI
Background-subtract
Normalize
|
hiresfile
hirescat
hiresseg
|
cutoutcat
cutoutdir
|
|
convolve
|
Create template library:
- Convolve cutouts with kernel
- Resample to match pixel scale of LRI
|
cutoutcat
cutoutdir
psffile
(psflookup)
(posframe)
|
templatedir
(templatedir</old>)
[hr_templatedir]
|
|
transform
|
Transform catalog to LR frame
|
cutoutcat
|
lrfcat
|
|
cull
|
Cull template library
Remove templates that fall entirely outside LRI
Trim templates that fall partially outside LRI
Adjust LR catalog accordingly
|
hiresfile
loresfile
lrfcat
templatedir
|
templatecat
templatedir</culled>
templatedir</shrunk>
|
|
fit
|
Generate grid of overlapping
cells
For each cell,
- identify objects located
within cell and their templates
- “grow” the
cell to completely enclose all those templates
- mask out pixels with
negligible values in all templates
- simultaneously solve the
resulting equation
|
loresfile
loreserr
loresflag
templatedir
templatecat
|
tfitpars
tfitcat
tfitcell
tfitcovar
|
|
diags
|
Select a single best fit
value for each object
Construct model image from templates with best value applied
Construct residual image by subtracting model from LRI
Construct idmap image showing template id as
central pixel
|
tfitcat
templatedir
|
tfitcat<_best>
modelfile
<resid_>modelfile
<idmap_>modelfile
|
|
dance
|
Cross-correlate zones of
model image and LRI
Determine optimal shift in each zone
Create shifted transfer kernels for each zone
|
loresfile
modelfile
|
ddiagfile
dlogfile
psflookup
|
|
plotdance
|
Plot shifts computed by dance
|
ddiagfile
|
ddiagfile<.ps>
|
Other common modes
The TFIT pipeline software permits the user to specify any
or all of the processing stages in any order (although the user is responsible
for sensible ordering; the software doesn't check for this). Thus, it's easy to
customize the processing for a given situation. This section discusses some
common use modes.
Running from existing cutouts
The cutout stage assumes that each of the hires and lores images are contained in a
single file. For very high resolution data of large fields, however, this may
not be the case. In this case, each hires image should be processed separately,
through the cutout stage only, using a single input catalog so as to
avoid object duplication or name collisions. Then all cutouts can be combined
into a single directory, and subsequent processing can begin from the transform
stage.
If no hires image exists that precisely corresponds to the lores
image (for example, because the high resolution data is too big to fit into a
single mosaic image, so that it was processed in sections), the parameter file
should be modified so that the hiresfile
parameter points to a text file containing the relevant WCS information, i.e.
CRPIX1 0.5
CRPIX2 0.5
REL_SCALE 10
Similarly, one may wish to use one set of high resolution data with several
sets of low resolution data. There's no need to regenerate the cutouts each
time; just start with transform for each set of processing after the
first.
Second-pass processing
If astrometric
problems such as distortion are present, the registration dance will detect nonnegligible shifts, and create a set of compensating
transfer kernels to be used. A second pass through the pipeline can then be
run, starting with the convolve stage in multikernel
mode, placing the templates into the same directory to avoid reconvolving any objects that don't need to have a shift
applied. The transform stage can be skipped since the catalog does not
change, and processing can then proceed to the cull, fit, and diags stages.
Because the dance stage generates a new set of transfer kernels, it should not be run as part of a second pass.
Note that a separate task parameter file should be prepared for second pass
processing, with careful attention to any filename changes necessary to avoid
overwriting first-pass products.
Restarting from a specific stage
Sometimes due to errors, it will be necessary to restart the
pipeline from a specific stage. In order to do this,
- copy the existing task
parameter file
- edit the copy to remove the
earlier stages
- restart
the pipeline with the new parameter file.
You may also want to modify any
filenames for partially-completed stages. At present it is necessary to
manually clean up the results of a partially completed stage.
Appendix: Glossary
"best" file: The
final TFIT catalog file, containing only one measurement per object. The best
measurement is determined to be the measurement from a cell in which the object
was located closest to the cell center.
cell: a rectangular zone on the lores image over with the fit stage performs a simultaneous
fit for all templates contained therein
collage: a FITS file containing a model image produced by adding
templates (usually scaled by the best fit fluxes) in the correct locations in
an empty array. This diagnostic product provides a visualization of what TFIT
determined was the best fit for all objects. Also known as model
image.
convolution kernel: a FITS file
containing an image which, when convolved with the hires image, would produce
the lores image. Also known as transfer
kernel.
convolve: a TFIT processing stage that
generates templates from cutouts.
covariance index:
max(abs(covar[ij]/covar[jj])), where covar is the covariance matrix for
the cell. The covariance index provides an estimate, for a given
object, of how strongly affected its photometric measurement was by its
neighbors, compared to the intrinsic uncertainty in the measurement. It
depends on nearest neighbor distance but also on relative brightness:
faint objects are influenced by bright neighbors more than bright
objects are influenced by faint objects.
cull: a TFIT processing stage that compares the
bounds of each template to the bounds of the lores
image array. Templates that fall entirely outside the lores
image are culled: removed from the catalog and placed in a separate
subdirectory. Templates that fall partially outside the lores
image are clipped.
cutout (noun): a FITS file
containing a single object cut out from the hires image, with the segmentation
map applied to exclude contributions for other objects.
cutout (verb): a TFIT processing
stage that produces cutouts from the hires image.
dance: a TFIT processing stage that uses IRAF
tasks to measure subpixel misregistration
between the collage and the lores image, and produce
a set of compensating convolution kernels. Also known as registration
dance.
diags: a TFIT
processing stage that produces the "best" catalog, the collage, and
the residual image.
first pass: the normal mode of TFIT
pipeline processing, from cutout through dance (or plotdance)
stages. So named because the dance stage frequently indicates
that a second pass is necessary to compensate for misregistration.
fit: a TFIT processing stage that fits the
templates to the lores image in overlapping cells.
hires image or HRI: the "truth" image that will be used for a priori
information in measuring photometry on a lower resolution image
hires template or hr template: a
FITS file produced by convolving a cutout with the convolution kernel,
and saved at the pixel resolution of the HRI. Usually normalized.
lores image or LRI: the target image on which
you are attempting to measure photometry
plotdance: a TFIT processing stage that
generates a diagnostic plot from the results of the dance stage
residual image: a FITS file containing an image produced by subtracting
the collage from the lores image.
second pass: TFIT processing through the
convolve, cull, fit, and diags stages, using a set of
convolution kernels to compensate for misregistration.
stage: a processing step of the TFIT pipeline
template: a FITS file produced by convolving a cutout with the
convolution kernel, and then binning down if necessary (in a flux-conserving manner) to match the pixel resolution of the LRI. Usually normalized.
transfer kernel: see convolution
kernel
transform: a TFIT processing stage that transforms the hires catalog to
the reference frame of the lores image.
utility script:
a script that is not part of the TFIT pipeline, but can be run
independently to perform some useful operation on the TFIT data
products.
Appendix: Parameter file
An annotated version of the
task parameter file is included here for convenience:
#Params relating to the hi-res image set
hiresfile hires.fits
hirescat hires.cat
hiresseg hires_seg.fits
# These are input parameters for the cutout stage. The
# hiresfile is the image from which the cutouts will
be made,
# using the catalog and segmentation information in the other
# two files.
hicatcolnames None
#hicatcolnames NUMBER, X_IMAGE, Y_IMAGE,
XMIN_IMAGE, YMIN_IMAGE, XMAX_IMAGE, YMAX_IMAGE, BACKGROUND, FLUX_ISO
#This is the default set of column names from the input catalog,
#used if the value of "hicatcolnames" is
None.
#If you need to specify a different set of names, be
sure that
#the order corresponds to the default set. All names must be
#specified on the same line. Alternatively, you can provide
#a filename as the value here, with the actual column names
#in the file, all on the same line.
#..................................................................
#..................................................................
#Params relating the two images
relscale 10
#This is the relative pixel scale between the two
images.
#For example if the hires images has a pixel scale of
0.1
#arcsec/pixel, and the lores
image has a pixel scale of 0.5
#arcsec/pixel, the value of relscale
should be 5.
#..................................................................
#..................................................................
#Params relating to the lo-res
image set
loresfile lores.fits
bgconstant 0
#TFIT can subtract a global background constant from the
#lores image before fitting. We do not recommend
this; instead
#it is recommended that you background-correct the lores
#image as perfectly as possible before fitting.
loreserr lores_rms.fits
errtype
rms
rmsconstant 1
#TFIT is designed to work with an RMS map as the error map,
#but it will also accept a weight map, or a single constant
#value of the RMS from which an RMS map is generated. The
#errtype specifies which kind of error image is
provided.
#For best results, use a source-weighted RMS map, to
prevent
#the bright objects from dominating the fit.
loresflag
lores_flag.fits
maxflag
64
#The flag map is used at several points in the pipeline to
#avoid constructing templates, and fitting pixels, that contain
#bad data. At present this is implemented as a maximum value:
#templates containing at least one pixel with a flag map value
#greater than maxflag will not be created. Eventually
we may
#change this to work as a mask.
#..................................................................
#..................................................................
#Params relating to the transfer kernel
multikernels false
psffile
ch1_irac.fits
psflookup
ch1_dancecard.txt
posframe lores
#The transfer kernel must be a FITS image on the same
#pixel scale as the high res image. It should contain
#a centered, normalized image T such that
# hires PSF convolved with T = lores PSF.
#TFIT supports the use of multiple kernels to accommodate
#a spatially varying PSF. To use this option, set the
#multikernels value to true, and provide a psflookup file
#that divides the lores image into rectangular zones,
#specified as pixel ranges, and provides a transfer kernel
#filename for each zone. Any objects which fall in a zone
#not included in the lookup file will use the transfer kernel
#specified as psffile.
#
#The posframe parameter determines whether the pixel ranges
#specified in the psflookup files are in lores or hires pixels,
#and, accordingly, which catalog file will be used as an imput
#to this stage:
# - if posframe = lores, the lrfcat will be used. This is
# the normal mode for pass 2 processing.
# - if posframe = hires, the cutoutcat will be used. This is the
# mode used when using multiple kernels in the first pass run.
#..................................................................
#..................................................................
#Params relating to the cutout stage
cutoutcat
hrc_trim.txt
cutoutdir cutout
normalize true
#cutoutcat should be the filename containing only
those columns
#from the hirescat used by TFIT.
#cutoutdir is the directory containing the cutouts.
#These are output parameters if you start from the cutout stage;
#they are input parameters if you start from the convolve stage.
#The normalize parameter determines whether the cutouts will be
#normalized or not; it is normally set to true, so that the
#final output catalog will contain fluxes, rather than colors.
#..................................................................
#..................................................................
#Params relating to the convolve stage
templatedir templates
save_hr True
hr_templatedir hr_templates
#templatedir is the directory containing the templates.
#It is an output parameter for the convolve stage,
#and an input parameter for all subsequent stages.
#
#The next two parameters control the optional production of
#hr templates: ie, templates that have been convolved with the
#kernel, but not binned down to the LR pixel resolution.
#If these images are created, they may (soon) be used by the
#compl_convolve stage, to apply a shift computed by the dance
#rather than re-convolving. They may also be used to construct
#an "hr collage" as a diagnostic product for use in constructing
#a better kernel.
#..................................................................
#..................................................................
#Params relating to the transform stage
lrfcat
hrc_trim.txt_lrf
#This is the "cutoutcat"
file, transformed into the
#pixel reference frame of the lores image. It is an
output
#parameter for the transform stage, and an input parameter
#for the cull stage.
#..................................................................
#..................................................................
#Params relating to the cull stage
templatecat templates.cat
#This is the "lrfcat" file, modified to
exclude or modify objects
#that were culled because their templates fell entirely or
#partially outside the bounds of the lores image. It
is an
#output parameter for the cull stage, and an input parameter
#for all subsequent stages.
#..................................................................
#..................................................................
#Params relating to the fitting stage
#The fitting stage has the most complex set of
parameters.
#...First filenames
tfitpars tpipe_tfit.param
tfitcat
lores_tfit.cat_pass1
tfitcell
lores_tfit.cell_pass1
tfitcovar
lores_tfit.covar_pass1
#These are all output parameters. The "tfitpars" file
#specifies the name of the special parameter file for the fitting
#stage that will be generated from the parameters in this file.
#The others are filenames for the output catalog,
cell, and
#covariance files, respectively.
#...Then control parameters
cell_xdim 100
cell_ydim 100
cell_overlap 15
#These are the most important control parameters.
#cell_xdim and cell_ydim
specify the original size of the cell,
#in lores pixels. It should be large enough to
contain 2-3 lores
#PSFs. The cell will grow to avoid chopping off
partial templates,
#so this is a minimum size.
#The cell_overlap is also
specified in lores pixels.
#..including many that don't change too often
dithercell true
cellmask
true
maskfloor 1e-9
fitbackground false
writecovar true
#These are control parameters.
# dithercell=true
dithers the pattern of overlapping cells over
#
the lores image, to avoid corner problems.
# cellmask=true
uses a mask to exclude pixels from the fit which
#
do not contain a value of at least "maskfloor"
in
#
at least one template. This is necessary because
#
of the cell growth.
# maskfloor=1e9
should be set to a value at which it's okay to clip
#
the templates.
# fitbackground=false
does not fit the background in each cell. This
#
is necessary because of the cell growth.
# writecovar=true
writes the covariance information out to the covar
#
file.
#..................................................................
#..................................................................
#Params pertaining to diagnostics
modelfile lores_collage.fits
#This is an output parameter. It will contain the
collage made by
#multiplying each template by its best flux and dropping it into the
#right place. An additional diagnostic file will be created: it will
#contain the difference image (loresimage - modelfile). Its filename
#will be created by prepending "resid_" to the modelfile.
#..................................................................
#..................................................................
#Params pertaining to the dance
dzonesize 100
maxshift
1.0
ddiagfile
ddiags.txt
dlogfile
dlog.txt
#The first two are control parameters.
# dzonesize specifies the
size of the rectangular zones over which a
# shift will be calculated. It should be comparable to the size over
# which the misregistration is roughly constant; but,
it must be
# large enough to contain enough objects to provide a good
signal
# to the crosscorrelation.
#
# maxshift specifies the maximum size of the x/y
shift, in lores
# pixels, which is considered to be valid. Any shift larger than this
# is considered spurious and dropped from the final results, and
# replaced by an interpolated value from the surrounding zones.
#
# ddiagfile is an output parameter for the dance stage, and an input
# parameter for the plotdance stage.
# dlogfile is an output
parameter; it simply contains the output from
# the crosscorrelation process.
#..................................................................
#..................................................................
#
#Param defining the pipeline
#Typical firstpass run
order cutout, convolve, transform, cull, fit, diags, dance, plotdance
#The typical firstpass run contains all the valid
TFIT stages.
#There are several possible paths through the TFIT
pipeline, depending
#on the state of the data. The pipeline software does not enforce
#any choices about the makeup or ordering of these stages; the user
#must be familiar enough with the pipeline to choose a path that makes
#sense.
#Other possible paths include:
#order convolve, cull, fit, diags
#This is a typical secondpass run, using the multikernel option to
#regenerate templates using spatially varying transfer kernels
#generated by the dance, and omitting the transform stage because it's
#already been performed.
#order convolve, transform,
cull, fit, diags, dance, plotdance
#This is a typical GOODS ACS-to-lores firstpass run, using previously
#generated cutouts.
Appendix: Errors & Warnings
===========================================================================
Symptom: The TFIT log file shows the following:
Uhoh, TFIT failed: exit status 32512
Probable cause: The libcfitsio.so file
cannot be located
Diagnostic procedure:
Examine the environment variable LD_LIBRARY_PATH (or equivalent on other
platforms) and determine whether it contains a path to the cfitsio
library.
Remedy:
Update the LD_LIBRARY_PATH appropriately and restart from the fit stage.
===========================================================================
Symptom: The TFIT log file shows the following:
Reading error map ...
Program aborting...
Probable cause: The specified error map file is missing.
Diagnostic procedure:
Examine the parameter file for the file name & path associated with keyword
loreserr. Determine whether this file exists.
Remedy:
Place the correct file in the directory; or correct the filename in the task
parameter file.
==========================================================================
Symptom: The following error messages are generated during the convolve
stage:
Original cutout crpix1/2: -5988.5 -6409.5
Modified newwcs crpix1/2: -576.5 -619.5
Illegal value None
omitting h[cunit1] for this object
Illegal value None
omitting h[cunit2] for this object
card is too long, comment is truncated.
Cause: cunit1 and cunit2 keywords are missing from FITS headers for the
cutouts.
Remedy: None needed - this is harmless, because we don't use those
keywords. They are read in for completeness.
==========================================================================
Symptom: The TFIT program runs to completion, but the output catalog is
empty. Log file indicates that all objects were discarded from every cell. The
diagnostic stage ends with an error like this one:
ValueError: No catalog entries were found in isaac9ks_tfit.cat_pass1. Task aborting.
Probable cause: The templatecat
specified in the parameter file is incorrect.
Diagnostic procedure: Check to make sure that the templatecat
file exists. Plot the xy positions of the objects and
make sure they are in the lores image's pixel
reference frame.
Remedy: Correct the templatecat, and
rerun starting from the fit stage.
===========================================================================
Symptom: Residual map shows areas with significantly-too-faint fits.
Possible cause: Multiple occurrences of some objects in the _culled
catalog fed to TFIT.
Diagnostic procedure:
Identify an object id in the affected region by measuring its (x,y) in DS9, then looking for an object with the same (x,y) in the tfitcat_best
file.
Search for rows starting with that object id in the templatecat
file; there should be only one row per object id. (IE, object ids must be
unique.)
Remedy:
Sort the templatecat file with the unique
option to ensure only one entry per object id, and restart from the fit
stage.
Appendix: Installation Guide
Software
requirements & supported platforms
Operating
system limitations:
In principle, the TFIT package should run on any flavor of Unix that also supports IRAF. However, it has only been
tested only on SunOS 5.8 and Redhat Linux (RHEL 3 and
4).
System
requirements:
Python - version 2.4 or greater
pyFITS - version 1.1 or greater
numpy - version 1.0 or greater
convolve - version 2.0 or greater
matplotlib - version 0.84 or greater (recommended but
not required)
PyRAF - version 1.1 or greater
STSDAS - version 3.2 or greater
some version of make
some c++ compiler (gcc recommended)
CFITSIO
Where to get
the other packages:
Installation
Procedure
TFIT now
uses the Python distutils package to simplify
software installations.
1. Download the gzipped tarfile and unpack it in some convenient location. This is
the source directory, and is not the same as the installation directory.
2. Make sure that the LD_LIBRARY_PATH includes the location of the libcfitsio.so library; and the LHEASOFT environment
variable is set to the parent directory. This will happen automatically if you
have the LHEASOFT or HEASOFT6_64 packages installed on your system.
3. Check to make sure there exists a src/Macrodefs.<OSname> file for your OS,
and that its definitions are appropriate for your system. Create and/or modify
if necessary. (If you need to create a new Macrodefs
file, please send it to us so we can include it in future distributions.)
4. python setup.py install
[--local=/some/local/place/]
This command builds the C++ program, pre-compiles the python
code, and places all scripts and executables either in the standard places, or
(if the --local option is used) in the specified directory tree.
(Note: C++ and its libraries are notoriously finicky. You may need to edit the Macrodefs or Makefile for your
particular setup. At present, even a clean build generates a number of
warnings.)
5. If you use the --local option, you must ensure the following variables are set in order to correctly run TFIT:
PATH to include
/some/local/place/bin/
PYTHONPATH to
include /some/local/place
6. To verify that the package has been built and installed correctly, you may
download and unpack the test set provided, and follow the instructions for
testing.