parthitv -- Remove particle events from HSP data files.
parthitv infile outfile outtable scheme dthresh athresh sigmas
The term "particle event" is a general term for any observed spike in the data. This task removes particle events by assigning a user-specified number to a data point's corresponding pixel in its output mask file. The user can choose from two methods to remove such events: (1) the statistical method will flag any observed measurement higher than a user-specified multiple of standard deviations above the mean value of the whole data set, and (2) the threshold method flags any observed measurement higher than a user-specified threshold level.
The statistical method is done in an iterative fashion. After the first round of particle event removal take place, remaining data points are rechecked for points above the statistical criterion of the new data set. The process goes on until no new points are removed.
This task also converts count to count rate (for digital data), averages the input data, and produces an average count rate (or analog reading) and its mean error. The average count rate (or analog reading) will be written to the output table as a new row. Output observation epoch is the modified Julian date of the first packet time.
Only single-group GEIS files can be used as input to this task.
- infile [string]
- Input file template. The input files must be GEIS file(s), i.e., they must have
the extensions .d0h, .d1h, .d2h, or .d3h.
The extension .d0h is assumed to be the digital star data file, .d1h the
digital sky data, .d2h the analog star data, and .d3h the analog sky data;
if any of the files is missing, the corresponding mean value is set to zero in
the output table. This task also looks for data quality files corresponding to
each input file in the same directory. The extensions for the data quality
mask files are .q0h,
.q1h, .q2h, and .q3h, with the digit in the extension matching that of
the input GEIS data file. If any input mask file is missing, the task
will assume all data in the corresponding input data file are valid.
The following keywords are needed from the science file header:
'ROOTNAME' ID of the observation set (char*10). 'DATA_TYP' Data type (char*7). 'SAMPTIME' Sample time of the observation (real). The following group parameters are required to calculate the observation epoch: 'FPKTTIME' Time of the first packet, i.e., the starting time of the exposure (char*24).
- outfile [string]
- Output template for updated data quality mask file(s). This parameter can be (1) a null, (2) a directory name, or (3) a file template. If it is null, no output mask files are generated. If it is a directory name, the output mask files will reside in the specified directory and will have the same file names as the corresponding input files except that the extensions are .n*h instead of .d*h.
- outtable [file name]
- Name of the output table to which the results of this task will be
table contains the following columns:
'ROOTNAME' ID of the observation set (char*10). 'SCHEME' Method of removal, as passed from the 'scheme' parameter (char*12). 'DOBJ' Mean digital count rate of the object after the particle event removal (real). 'DSKY' Mean digital count rate of the sky after the particle event removal (real). 'AOBJ' Mean analog digital number (DN) of the object after the particle event removal (real). 'ASKY' Mean analog digital number (DN) of the sky after the particle event removal (real). 'DOBJ_ERR' Mean error of 'DOBJ' (real). 'DSKY_ERR' Mean error of 'DSKY' (real). 'AOBJ_ERR' Mean error of 'AOBJ' (real). 'ASKY_ERR' Mean error of 'ASKY' (real). 'DTHRESH' If 'scheme = threshold', then this value is equal to the digital threshold level, as passed from the 'dthresh' parameter; if 'scheme = statistical', then this value is equal to the statistical criterion, as passed from the 'sigmas' parameter (real). 'ATHRESH' If 'scheme = threshold', then this value is equal to the analog threshold level, as passed from the 'athresh' parameter; if 'scheme = statistical', this will be the statistical criterion, as passed from the 'sigmas' parameter (real). 'OBJ_SAMPTIME' Pixel sample time of the object, in seconds (real). 'SKY_SAMPTIME' Pixel sample time of the sky, in seconds (real). 'DOBJ_REMOVED' Number of points been removed by this task for the digital object file (int). 'DSKY_REMOVED' Number of points been removed by this task for the digital sky file (int). 'AOBJ_REMOVED' Number of points been removed by this task for the analog object file (int). 'ASKY_REMOVED' Number of points been removed by this task for the analog sky file (int). 'EPOCH' Epoch of the first packet time in modified Julian date (double).
- scheme [string, allowed values = statistical | threshold]
- Method used for particle events removal.
- dthresh [real]
- Threshold level to be used to remove particle events in digital data (counts). Data points higher than the threshold are removed; points equal to or lower than the threshold are kept.
- athresh [real]
- Threshold level to be used to remove particle events in analog data (DN). Data points higher than the threshold are removed; points to or lower than the threshold are kept.
- sigmas = 3. [real, min = 0.]
- Number of standard deviations above the mean value. Points with measurements higher than this level are flagged as particle events when the statistical method is used (i.e., number of sigmas). If lowflag=yes, points lower than this number of standard deviations below the mean value are also removed.
- (lowflag = no) [bool]
- When using the statistical method, this task can also remove points LOWER than the statistical threshold by setting this flag to "yes".
- (hitval = 1.) [real]
- Numerical value to be assigned to the mask file pixels whose corresponding data points are flagged as particle events.
1. Remove particle events from all data files in the current directory. Write the result to the output table output. The output mask files will be in the same directory. Use the statistical method to remove points 3 sigmas higher than the mean value of all valid points. Pixels corresponding to particle events are assigned the value of 1 in the output mask file.
hs> parthitv "*.d*h" "." "output" "statistical" sigmas=3 \ >>> lowflag=no hitval=1