NAME

mscombine -- Combine NICMOS, STIS, ACS or WFC3 images.

USAGE

mscombine input output

DESCRIPTION

This task is a simple CL script that enables running task `gcombine' on NICMOS, STIS, ACS and WFC3 images. Most of its parameters are just passed to the underlying `gcombine' without modification. The following paragraphs present issues specific to `mscombine'; the user should refer to the `gcombine' task for everything else, including algorithm details.

This task works by "expanding" each IMSET in a multiextension file into its component HDUs. Each extracted HDU goes into a temporary GEIS-format file. These temporary files are deleted upon successful task completion. Since `gcombine' can only handle GEIS files, make sure your `imtype' environment variable is set to ".hhh" before running this task.

After expanding the IMSET, each of its components go into a specific input list in `gcombine'. The SCI HDU goes into the `gcombine.input' parameter and the ERR HDU goes into the `gcombine.errmap' parameter. The DQ HDU is first logically ANDed pixel-by-pixel with a Bolean mask created from the bit specifications in the `dqbits' pset, and the result is fed into the `gcombine.masks' parameter. In this way it is possible to do selective pixel masking (which is not possible with `gcombine' by itself).

`gcombine' can be run in such a way that it scales the input images by their exposure times. The exposure time is usually set into a header keyword, but in the case of NICMOS and WFC3/IR IMSETs the exposure time information may be not present in the SCI HDU but in the TIME HDU instead. If all pixels in a given IMSET have the same exposure time, this exposure time value can be used by `mscombine' for scaling purposes. This is done by setting the `expname' parameter to "PIXVALUE". At this time `mscombine' does not support the case in which there are pixel-to-pixel variability in exposure time within a given IMSET.

After `gcombine' finishes, its output is re-assembled back into a multiextension FITS file. The `gcombine.output' file goes into the SCI HDU of the current output IMSET, and the `gcombine.out_err' file goes into the ERR HDU of the current output IMSET.

The output DQ HDU contains any input DQ bits that survived both the masking operation described above, and the rejection/masking/threshold algorithms internal to `gcombine'. Notice that with default values for the bit flags in the `dqbits' pset, the output DQ HDU will be always zeroed. This is because the default is to filter out pixels with *any* DQ flag set, thus only pixels with no DQ flags set will survive. To modify this behavior, turn off in the `dqbits' pset (set to "no") the specific DQ bits that must be propagated into the output.

The output TIME HDU in NICMOS and WFC3/IR IMSETs is computed by adding the individual TIME HDUs from all the corresponding IMSETs in the input files, and dividing the result pixel-by-pixel by the number of images used to build each pixel. This number is the total number of SCI HDUs input to `gcombine' minus the number of SCI HDUs rejected by either masking or the `gcombine' rejection algorithms. Before the addition, any pixels that were discarded by masking and/or rejection algorithms have their corresponding TIME values zeroed. The goal of this procedure is to scale the exposure time of each pixel in such a way as to make it consistent with the corresponding output SCI pixel, which is either an average or a median of the input pixels. The procedure will generate "correct" exposure times only when `gcombine' is run with scale="none".

When any form of scaling is used, the TIME HDU generated by `mscombine' may be incorrect. A complete solution of this problem must await the full rewrite of `gcombine', in which the correct scaling will be applyed on a pixel-by-pixel basis.

The output SAMP HDU in NICMOS and WFC3/IR IMSETs is generated by adding the individual SAMP HDUs from all the corresponding IMSETs in the input files. Before the addition, any pixels that were discarded by masking and/or by the gcombine rejection algorithms have their corresponding SAMP values zeroed. Thus the final output SAMP HDU in each IMSET will contain the actual total number of samples used to build each output pixel.

This task will work also with MULTIACCUM files, by looping over the EXTVER numbers in each file in the input list. All files in the input list must have the same number of IMSETs, and their EXTVER numbers should span the range from 1 to NEXTEND/5 (NICMOS and WFC3/IR) or NEXTEND/3. The HDUs can be physically in any order in the file though. The combination will be done among IMSETS with the same EXTVER number. The resulting output file will have the same number of IMSETS of each input file.

The parameters of this task comprise a subset of the `gcombine' parameters.

No log file output is provided in this version; all log output goes into STDOUT. If needed, redirect it into a file by using the ">" operator.

PARAMETERS

input [string]

List of input files to combine. They can be either ACCUM or MULTIACCUM NICMOS, STIS, ACS and WFC3 files, but the number of IMSETS, the dimensionality and the size of all HDUs in the input images must be consistent with each other.

output [string]

Output combined file. If a proper extension (".fits" or ".fit") is not supplied, the task will append ".fits" to the file name.

(dqbits = "") [pset]

Pset that stores the bit masks for rejecting pixels with specific anomalies.

(nsmod_e = yes) [bool]

Use noise models to compute output "error" image when the number of input images is less than 10 ?

(reject = "none")

[Allowed values: none | minmax | ccdclip | ccdcrrej | rsigclip | rsigcrrej | avsigclip | avsigcrrej| errclip | errcrrej]
Type of rejection operation performed on the pixels remaining after masking and thresholding. The algorithms are described in the DESCRIPTION section of the `gcombine' help page. The rejection choices are:

      none - No rejection
    minmax - Reject the nlow and nhigh pixels
   ccdclip - Reject pixels using CCD noise models
  ccdcrrej - Reject only positively deviant pixels using noise models
  rsigclip - Reject pixels using a robust sigma clipping algorithm
 rsigcrrej - Reject only positively deviant pixels using robust sigmas 
 avsigclip - Reject pixels using an averaged sigma clipping algorithm
avsigcrrej - Reject only positively deviant pixels using avsigclip
   errclip - Reject pixels using sigma based on input error images
  errcrrej - Reject only positively deviant pixels using error images

(combine = "average") [string]

[Allowed values: average | median]
Type of combining operation performed on the final set of pixels (after masking, thresholding, and rejection). The choices are "average" or "median".

(weight = "none") [string]

[Allowed values: none | pixelwise | mode | median | mean | exposure | @<file> | !<keyword>]
Type of weighting scheme used when combine = "average". The choices are shown by the allowed values above. It is important to note that allowed weighting schemes, except for "none", can be divided into two major categories: one is the "pixelwise" weighting and all the rest belong to the uniform weighting. While the weights in the uniform weighting scheme are constant for all pixels of each input IMSET, the reciprocal of the variance at a given pixel is used as the weight for that pixel in the pixel-wise ("pixelwise") weighting scheme. In the case of uniform weighting, the weights can be the "mode", "median", or "mean" of the specified statistics section, the "exposure" time, values given in a "file", or values in a user-specified header "keyword". When specified in a file the weights must be one per line in the order of the input IMSETs.

(nsmod_w = yes) [bool]

Use the noise model to compute the variances from mode, median or mean for uniform weights ?

(blank = 0.) [real]

Output value to be used when all points are considered as bad at a given pixel of the combined IMSET.

(scale = "none") [string]

[Allowed values: none | mode | median | mean | exposure | @<file> | !<keyword>]
Scaling factor to bring the input IMSETs to a common exposure level. The choices are "none", dividing by the "mode", "median", or "mean" of the specified statistics section, dividing by the "exposure" time in the header, dividing by the values in a specified "file", or dividing by values in a user-specified header "keyword". When specified in a file the scales must be one per line in the order of the input IMSETs. Notice that for NICMOS and WFC3/IR files there is no way of scaling by the TIME HDU (whatever the meaning of this operation).

(zero = "none") [string]

[Allowed values: none | mode | median | mean | @<file> | !<keyword>]
Zero flux offset to bring the input IMSETs to a common zero level. The choices are "none", shift by the "mode", "median", or "mean" of the specified statistics section, shift by values given in a "file", or shift by values in a user-specified header "keyword". When specified in a file the zero values must be one per line in the order of the input IMSETs.

(statsec = "") [string]

Section to use in computing image statistics for scaling and weighting. If a null section is given then the entire input array is sampled.

(expname = "PIXVALUE") [string]

Header keyword to be used with the exposure scaling and weighting options. Also, if an exposure keyword is specified, an "effective exposure time" will be computed and that keyword will be added to the output IMSET. The effective exposure time is defined as an average of the scaled input exposure values. The default value "PIXVALUE" instructs the task to look into the TIME HDU of NICMOS and WFC3/IR IMSETs. This will work only if the exposure time is the same for all pixels in the given IMSET.

Algorithm Parameters

(lthreshold = INDEF, hthreshold = INDEF) [real]

Low and high thresholds (floors and ceilings) to be applied to the input pixels. This is done before any scaling, rejection, and combining. If they are specified as INDEF, the thresholding will not be applied.

(nlow = 1, nhigh = 1) [integer]

The number of low and high pixels to be rejected by the "minmax" algorithm. These numbers are converted to fractions of the total number of input IMSETs so that if no rejections by masking and thresholding have taken place, the specified number of low and high pixels will be rejected, while if pixels have been rejected by masking and thresholding, then the fraction of the remaining pixels, truncated to an integer, will be used to do this operation of the "minmax" rejection.

(nkeep = 1) [integer]

The minimum number of pixels to retain or the maximum number to reject when using the clipping algorithms (ccdclip, crreject, sigclip, avsigclip, or errclip). When given as a positive value, this is the minimum number to keep. When given as a negative value, the absolute value is the maximum number to reject. The latter is in addition to pixels rejected by masking and thresholding.

(mclip = yes) [bool]

Use the median as the estimate for the true intensity rather than the average with high and low values excluded in the "ccdclip", "ccdcrrej", "rsigclip", "rsigcrrej", "avsigclip", "avsigcrrej", "errclip", and "errcrrej" algorithms?

(lsigma = 3., hsigma = 3.) [real]

Low and high sigma clipping factors for the "ccdclip", "ccdcrrej", "rsigclip", "rsigcrrej", "avsigclip", "avsigcrrej", "errclip", and "errcrrej" algorithms. After the algorithm estimates a "sigma", it will compare the deviation of a pixel value about the average or median with the products of (lsigma * sigma) and of (hsigma * sigma) to reject outliers on both ends, except that the "lsigma" is ignored for the various CR rejection algorithms.

(rdnoise = "0.0", gain = "1.0", snoise = "0.0") [string]

CCD readout noise in electrons, gain in electrons/DN, and sensitivity noise as a fraction. These parameters are needed when (1) the noise model is used to compute the output ERR HDU (nsmod_e = yes); (2) the "ccdclip" and "ccdcrrej" rejection algorithms are chosen; (3) the weights are computed based on the noise model (nsmod_w = yes). The values may be either numeric as shown above, or a SCI HDU (or PHU if inherit=yes) header keyword.

(tempdir = "tmp$") [file]

Directory for temporary files.

EXAMPLES

LIMITATIONS

o

Because of limitations on maximum number of files imposed by `gcombine', this task can process a maximum of 12 files in its input list.

o

No "sum" combination yet (available only in wfpc.combine).

o

In NICMOS and WFC3/IR files the resulting TIME HDU will be properly computed only if "scale=none"

o

The output file will have all its HDUs in non-compressed format, even if pixel values are constant.

o

Only STDOUT printout is supported (no log file).

BUGS

o

This task creates a large number of temporary files in the "tempdir" directory. If by any reason it aborts prematurely, these files will not be deleted and may consume a significant amount of disk space.

o

Because of the large number of temporary files, the task can be very time-consuming, in particular if the "tempdir" directory does not reside in the local machine.

o

The task may fail to produce the expected results in the error, DQ and other extensions in the output file, when the number of input files is exactly 10 and depending on particular combinations of input parameters. This is actually a design bug in underlying task gcombine.

REFERENCES

This task was written by I. Busko.

SEE ALSO

gcombine