NAME

pfos_dispfit -- Fit POA-FOS dispersion relation to wavelength standard lamp measurements and obtain coefficients for the POA-recalibrated Faint Object Spectrograph (FOS) ccs6 wavelength reference file.

USAGE

pfos_dispfit input fittab output iterations relax

DESCRIPTION

The pfos_dispfit task uses a table of measured wavelength standard lamp positions (pixel position and associated catalogue wavelength) to fit the POA-FOS dispersion relation. The fitting is done using non-linear least squares techniques (Newton-Raphson Method). Optimized values for up to 10 of these dispersion coefficients can replace the values in the POA-FOS ccs6 wavelength reference file (used by the POA-FOS calibration pipeline to calculate the wavelength scale in the .c0 file).

Fitting can be performed in two directions: (1) using wavelength as independent variable and keeping x pixel positions as the dependent variable, or (2) vice versa of using x pixel positions as the independent variable and wavelength as the dependent variable. Due to uncertainties in both the measured positions of lines and the catalogued wavelengths, it is always advisable to use both fitting directions for obtaining final (averaged) values for coefficients obtained with the non-linear least squares method.

Typically, pixel positions are measured by fitting Gaussian profiles to the spectral lines in graphs generated by IRAF tools like splot. Example catalogues of useful lines for the different FOS modes are available in the poa_disp$ directory, along with sample files to run this tool (see examples section below).

The pfos_dispfit task requires the user to specify: the number of iterations to be performed in the fit, the relaxation factor for the fit (the fraction of the corrections to be used while iterating), the number of times intermediate fit results should be printed to the screen, the direction of the fit, whether or not the user wants to select only certain input x/wave values and whether the user wants to recalculate the dependent variable at the end of the iterations using the updated fit results. Please read the explanation to the parameters below to better understand the scope of the tool.

The user can perform the fitting starting either from an existing table of coefficients or by setting user-defined parameters which hold the initial guesses of the parameters for the fit. As it is advisable to perform a moderate number of iterations with a modest value for the relaxation factor, the user may choose to perform only a limited number of subsequent iterations and produce a "safety copy" of the parameters and results before continuing fitting.

For more information on the POA FOS dispersion relations as well on the algorithm used in this fitting tool, please see the POA FOS Instrument Science Report POA/FOS-2001-04. The report can be found off the POA web pages:

http://www.stecf.org/poa/stecf_bib.html

[A side note regarding x pixel/diode positions]

Hence, when a table of measured line positions is generated using cursor input from a plotted spectrum (which, for example is a 4 times oversampled observation), a line centered in the 10th DATA array location will have a center xorg=10.0. Therefore, before fitting coefficients, these "x" positions need to be converted to a new "x" using the equation "x = (xorg-1)/NXSTEP". For this example we have "x = (xorg-1)/4" or x = 9/4 = 2.25.

[end of note]

If information is needed on how to create an STSDAS table from an ascii list of pixel positions, the ttools package will have all the necessary instructions. The task tcreate will create the table in the required format. Sample linelist tables for the FOS BLUE gratings have been made available along with their associated fit tables. These data and fit values are the current solutions to the FOS BLUE dispersion coefficients in the ccs6 reference file. Here is a list of the data and fit tables:

  poa_disp$linelistH13.fits & fitH13.fits  [H13 grating]
  poa_disp$linelistH19.fits & fitH19.fits  [H19 grating]
  poa_disp$linelistH27.fits & fitH27.fits  [H27 grating]
  poa_disp$linelistH40.fits & fitH40.fits  [H40 grating]
  poa_disp$linelistH57.fits & fitH57.fits  [H57 grating]
  poa_disp$linelistL15.fits & fitL15.fits  [L15 grating]
  poa_disp$linelistL65.fits & fitL65.fits  [L65 grating]

Also, please see the poa_preproc_fos, poa_calfos and processfos help files for more details on the POA FOS re-calibration processing steps.

PARAMETERS

input [file]

The name of input table with x-position and wavelength information. The names of these columns are specified using the x_col and wave_col parameters below. This table can have any number of additional columns, including a select column which can be specified using the sel_col parameter below. The x column must be of type real and the wavelength column of type double. The selection column can be of type real or integer. Rows are allowed to have INDEF values; only pairs of non-INDEF x and wavelength values will be read in as valid data points for fitting. The table can be FITS or STSDAS format. If the dependent variable should be recalculated using the updated coefficients, the newcalc_col parameter has to be set to the name of a column that will hold the new values of the dependent. This column will either be created and appended to the input table, or, if already present it will be overwritten. Please beware of this feature in order not to overwrite essential informatiuon in the original input file. Also, please see the note in the description section regarding pixel/diode positions when creating the x_col values in the input file.

fittab [file]

Optional input coefficient table with previous or initial fit information. The fit table should be in FITS format. This parameter can be left blank when using the parameter-driven input scheme (below). A sample starting table can be copied using the poa_disp$dispfit_blank.fits file as a template. This table contains the names of the 10 parameters for the fit (PARAMETER column), the initial guess values for the fit (INITIAL_GUESS column), a switch indicating whether a parameter is open or fixed for fitting (MODIFIABLE column), the final resultant value (FIT_VALUE) and the error associated with that fit value (FIT_ERROR). The fit table's INITIAL_GUESS or FIT_VALUE numbers can be used as the starting values of the fit; setting the iterations parameter to a negative number will signal the use of FIT_VALUE as the starting (re-fitting) values; positive iteration settings signal the INITIAL_GUESS values to be used as the starting numbers to the fitting. If the parameter use_params=no, the fittab must be provided by the user. The output parameter cannot be the same name as fittab since there is no clobbering allowed for this file.

output [file]

Name of output results fit table (FITS format). The output file name is required to run this tool; the name should include the .fits suffix. If fittab was specified, output will be a copy of the fittab with new fitting results. This table contains the names of the 10 parameters for the fit (PARAMETER column), the initial guess values which were used in the fit (INITIAL_GUESS column), the switch which indicates whether a parameter was open of fixed for fitting (MODIFIABLE column), the final resultant value (FIT_VALUE) and the error associated with that fit value (FIT_ERROR). The header of the output file contains six keywords which are related to the fitting:

  NUM_DATA  "Number of data points used"
  NUM_ITER  "Number of iterations performed"
  DEP_NAM   "Column name of dependent variable"
  IDEP_NAM  "Column name of independent variable"
  RELAXFAC  "Relaxation factor used in current fit"
  CHI_SQ    "Chi Sq in current fit"

To view the output file header, one can use the tables.ttools.tprint command, setting the prparam+ to print the keywords. For example:

	cl> tprint output.fits prparam+

When using the re-fitting option (negative value of iterations), the NUM_ITER keyword is updated to include the cumulative number of iterations performed, not the current number asked for in the fit.

iterations [integer]

Number of iterations to perform for the fit. Positive and negative values are allowed; negative values are not allowed when using use_params=yes. A positive iteration request indicates to use the INITIAL_GUESS column from fittab or the user defined initial values parameters as the starting fit values; a negative iteration request means to re-fit, using the previous results (from the FIT_VALUE value column in fittab). The current maximum number of allowed iterations is 1.E+08. If more iterations are needed, one can always continue with more iterations based on previous results, by specifying the negative number of iterations needed.

relax [real]

Relaxation factor for the fit. The absolute value will be used to calculate the fractional correction used per iteration. For very simple situations (such as when only one coefficient is allowed to vary, while all the others are fixed) a fraction of one can be used to quickly arrive at a minimal least squares solution. Otherwise, when more than one coefficient is allowed to vary and the set of coefficients is not yet extremely close to the final minimum for chi squared, it is important to keep the relaxation factor small enough to not let the corrections overshoot the fit. An optimal relaxation value occurs if the decrement per iteration is on the order of 10**(-N), where N is the number of free coefficients. For example, if one has 2 coeffs "OPEN", then set relax such that decrement is about 0.01 (1%) in the first few iterations; the decrement will shrink as one approaches the final solution, and so relax needs to be adjusted to larger values in consecutive runs.

A negative value for relax in addition forces printing the correlation matrix of coefficients at the end of iterations.

(use_params = no) [boolean]

Option to use the parameters below to set the initial values of the fit. If this parameter is set to yes, then the input fittab is ignored; fitting parameters will be read in from the user defined parameters below.

(print = 50) [integer]

Print results every N iterations to the STDOUT terminal. If print is greater than the minimum number of iterations, then only the last iteration is printed.

(fit_dir = "wave2x") [string]

The fitting direction can be wave2x or x2wave. If using wave2x, the wavelength is used as the independent variable as input, and x positions are calculated as the dependant variable in the fit. The opposite holds true when using the x2wave direction.

(x_col = "X") [string]

Name of column with x position in the input file. This parameter is case sensitive.

(wave_col = "WAVE") [string]

Name of column with wavelength in the input file. This parameter is case sensitive.

(sel_col = "") [string]

Optional name of selection column in the input file. This parameter is case sensitive. When left blank, no selection is performed on the input x and wavelength positions. If non-blank, the selection column will be opened in the input file; the column must be of type real of integer; values of 1 indicate to select the x/wave combination, while 0 indicates to pass the value. Please note that even if a select column has the value of 1, while either x or wave is INDEF, this input will not be selected.

(newcalc_col = "") [string]

If the user wants to recalculate the dependant variable column using the resultant coefficients, they can set this parameter to the name of the new column. A non-blank value will trigger the creation of the new column in the input file using the fit results. If left blank, no extra column will be calculated or added. If the requested new column is already in the input file, the program will overwrite the values of that column. Please beware of this feature; it is advised to use new column names instead of overwriting something in the file.

Following are the 10 coefficients and their open/fixed modifiers.

(a = 1.) [real]

When using user_params=yes, this will be the initial value for parameter a in the fit.

(a_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of a to be adjusted or not in the fit, respectively.

(b = 1.) [real]

When using user_params=yes, this will be the initial value for parameter b in the fit.

(b_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of b to be adjusted or not in the fit, respectively.

(c = 1.) [real]

When using user_params=yes, this will be the initial value for parameter c in the fit.

(c_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of c to be adjusted or not in the fit, respectively.

(d = 1.) [real]

When using user_params=yes, this will be the initial value for parameter d in the fit.

(d_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of d to be adjusted or not in the fit, respectively.

(e = 1.) [real]

When using user_params=yes, this will be the initial value for parameter e in the fit.

(e_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of e to be adjusted or not in the fit, respectively.

(o = 1.) [real]

When using user_params=yes, this will be the initial value for parameter o in the fit.

(o_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of o to be adjusted or not in the fit, respectively.

(p = 1.) [real]

When using user_params=yes, this will be the initial value for parameter p in the fit.

(p_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of p to be adjusted or not in the fit, respectively.

(q = 1.) [real]

When using user_params=yes, this will be the initial value for parameter q in the fit.

(q_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of q to be adjusted or not in the fit, respectively.

(r = 1.) [real]

When using user_params=yes, this will be the initial value for parameter r in the fit.

(r_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of r to be adjusted or not in the fit, respectively.

(s = 1.) [real]

When using user_params=yes, this will be the initial value for parameter s in the fit.

(s_mod = "FIXED") [string]

When using user_params=yes, "OPEN" or "FIXED" allows the value of s to be adjusted or not in the fit, respectively.

EXAMPLES

1. Run the pfos_dispfit task on a FOS BLUE (grating=H40) linelist file "sampleH40.fits" which contains x and wavelength positions. The sample fittab is called sampleH40_fit.fits; it has all but one of the coefficients already derived; the 5th coefficient is "OPEN" for fitting. The example will perform 500 iterations, based on the INITIAL_GUESS values in the fittab; it will print every 100 values; it will use a relaxation factor of 0.01; the fitting direction will be from wavelength to x position; no selection will be done on the input values; no new output column will be calculated.

  st> copy poa_disp$sampleH40* .
  st> pfos_dispfit sampleH40.fits sampleH40_fit.fits outfit.fits \
      500 0.01 use_params- print=100 fit_dir=wave2x x_col=X  \
      wave_col=WAVE sel_col="" newcalc_col=""

2. Continuing the example above, one can use the results of the previous fit as input to many more iteartions by using the FIT_VALUE column from the previous output file. Therefore, using the input and output of the above, we have the same input file, the output of the previous fit will now be the fittab of the current fit; in order to continue fitting, we set the iteration to a negative number of iterations; we will iterate 5000 times; using a relaxation factor of 0.1; we will select the input data based on the selection column "ss" in the input file; we will also create a new column of x positions called "NEW_X".

  st> pfos_dispfit sampleH40.fits outfit.fits outfit2.fits  \
      5000 0.1 use_params- print=1000 fit_dir=wave2x x_col=X  \
      wave_col=WAVE sel_col=ss newcalc_col=NEW_X

NOTES

Once the dispersion coefficients have been calculated they need to be inserted into the ccs6 POA FOS reference file for the appropriate grating/aperture/channel combination. This can be done using tedit or a number of table manipulation stsdas tasks.

BUGS

REFERENCES

References to the Post Operational Archive system for HST data can be found on:

      "http://www.stecf.org/poa"

HELP

For assistance using this particular task, please contact ecf-poa@eso.org, or stdesk@eso.org (see http://www.stecf.org/poa).

For assistance using this or any other tasks, please contact help@stsci.edu or call the help desk at 410-338-1082.

SEE ALSO

poa_preproc_fos, poa_calfos, processfos, imheader, tcreate, tprint

Type "help poa_fos opt=sys" for a higher-level explanation of the poa_fos package and the process of re-calibrating a FOS data set.

Type "help poa_preproc_fos" for an explanation of the task which runs the FOS data pre-processor.

Type "help poa_calfos" for an explanation of the poa_calfos task for re-calibrating FOS data.