SIGNIFICANCE_OF_RESULT · PARAMETERS · EXAMPLES · TIME_REQUIREMENTS
BUGS · SEE_ALSO
fft -- Perform Fast Fourier Transform of time series
fft source_file background_file fft_file column_name bins bin_length
A Fast Fourier Transform (FFT ) is performed on the specified column of the input source_file. A disk based FFT algorithm is used, so that the size of the FFT data set will not be affected by memory limitations. Currently all gaps in the data are filled with zeros. It is possible to analyze the window function for the data by requesting the exp column as input (see parameter column_name).
Two forms of input are possible, either the tabular output of the LTCURV task, root_ltc.tab, or a time-sorted QPOE file, root_sti.qp. The default input type is a _ltc.tab file. If the number of bins is large the preferred input is the _sti.qp file, which avoids using disk space on a table file with most elements=0.
Three files are output:
1) The root_fft.tab file contains all Fourier coefficients and should not be generated for a long transform. This is done by setting the parameter fftcoeff=no. 2) The parameter power_thresh is set so that only the largest coefficients are printed to the root_pwr.tab file. The default setting lists only "significant" normalized powers. 3) The file root_ftp.tab gives the distribution of Fourier powers.
Simple gap-filling algorithms have been provided for coherent transforms. Either linear interpolation between gap end-points, or gap filling with the mean value (see parameter gap_fillmode).
A summed FFT can be calculated by inputting a list of files for the sum, using the standard IRAF input list mechanism. This list can take two forms, a list consisting of a single QPOE file with different time filter extensions, or a list of _ltc.tab files generated from different time intervals of a single QPOE file with the LTCURV task. The TIMFILTER task will display the valid time intervals for a QPOE file and the user may choose which intervals are appropriate.
The summed FFT method requires that the bin_length and number of bins be the same for all input components and any components not matching will be excluded. When using the summed FFT, information about the FFT coefficients for the individual data intervals are not retained. The rfreq and ifreq columns in the output _fft.tab file are not valid, and only the power and npower should be considered as resulting from the sum. Examples 5 and 6 illustrate how to generate a summed FFT.
If the events are random, the average value of the fft power density, "power", is (no. of events)/(no. of bins), or (no. of events)/2*(no. of frequencies). The average value of the normalized density, "npower", is 1.
The probability of npower at a given frequency having a value > p is exp(-p). The probability of npower at all of the N frequencies having a value < p is :
[1 - exp(-p)]**N, or approximately [1 - N(exp(-p))] for large p.This is the "confidence" that a power density, npower=p is not random. Thus, in a 2048 bin transform (which yields values for npower at 1024 frequencies), the value of npower at a "confidence", C, of 90% is given by :
1 - C = 0.1 = 1024[exp(-p)], and p = 9.23.Values of npower at several significance levels are given in the *_pwr.tab file header.
- source_file = prompt = Input source file
(table or time-sorted QPOE file)
The input time-sorted QPOE file or a time series table file. Any time series in table format will do. In particular the output of the LTCURV module is designed for this and is the default input. For a summed FFT, input a list using the @ delimiter.
- background_file = "none" prompt = Input Background Timing File
(time-sorted background QPOE file)
Optional input in time ordered QPOE format. The file is created by running TIMSORT on a specified background region. The file may be specified by supplying the file name or by giving the root and the "_bti.qp" extension will be appended by the task.
- fft_file = "." prompt = root name for output file
The root filename for the output files. There will be three files:
<root>_fft.tab <root>_ftp.tab <root>_pwr.tabAn empty string will cause the root from the input source filename to be used and automatically append the appropriate extension. A "." will do the same without using the full pathname from the input file.
<root>_fft.tab (columns: freq, rfreq, ifreq, power, npower)
This file contains the output Fourier transform with a column for the real and the imaginary parts. It also contains a column for the power density distribution. Where
power_density(i) = | fourier_transform(i) |
<root>_ftp.tab (columns: logel, phist, npower)
This file contains a histogram of the power density distribution where
phist - The number of fft bins with this normalized power logel - The log of the number of elements with the power npower - The normalized power corresponding to the bin center
<root>_pwr.tab (columns: bin, freq, power, npower)
This file contains the bins from the resultant Fourier transform that have a normalized power density distribution exceeding the normalized power associated with a 50% confidence level or the level specified by the user with power_threshold. The normalized power density cut-off values for confidence levels 80%, 90%, 95%, and 99.5% are also given in the header of the _pwr.tab file.
- column_name = "src" prompt = Input table column heading
The label of the desired column from the input source table file. By choosing different column headings, the FFT can be performed on different input data. If the source_file is of the form _ltc.tab, a TIMPRINT of that table file will display all the possible choices, i.e. src, bkgd, ctrt, net, exp, etc. If the source_file is a timsorted QPOE file, the allowed column headings are : src, ctrt, exp, bkgd, and net.
- bins = prompt = Number of bins
The number of bins to perform the FFT on. If 0 is input the task will prompt for bin_length. The number of bins should be a power of 2. If it is not, the task will choose the closest power of 2 to the input value.
- bin_length = 1. prompt = Bin length in seconds
The length of the bin for the FFT. If the total number of bins is not a power of 2, the task will choose the closest power of 2 to the input value.
- (display = 1) [int]
A range of debug output from 0 for minimum to 5 for maximum:
0 -- no display 1 -- display of input records and fft records 2 -- display of history record and confidence levels
- (clobber = no) [boolean]
If yes, then an existing copy of the requested output file will be OVERWRITTEN. If no, an existing copy of the requested output file will cause a FATAL ERROR.
- (fftcoeff = yes) [boolean]
If yes then the _fft.tab file will be generated giving all the Fourier coefficients. However, for very long transforms this file can become unweildly. Therefore, setting fftcoeff=no will eliminate this file and leave the user with just the _pwr.tab and _ftp.tab summary files.
- (expthresh = 50.) [real]
The percentage of total exposure required for each bin to pass the exposure filter. For example, if expthresh is 0.5 then only bins whose exposure time is >= 0.5 * BINLEN will be used. An expthresh of 0.0 allows all bins to be used.
- (gap_fillmode = 0) [int]
Currently three modes are possible:
0 -- no gap filling 1 -- linear interpolation between ends of gaps 2 -- filling gap with the constant value
- (gap_fillconstant ) [int]
For gap_fillmode = 2, this specifies the constant value to be used as the filler value.
- (get_gintvs = yes) [boolean]
This specifies that the good time intervals can be obtained from the input file header. This can be set to no when dealing with foreign data for which there are no GTI records in the QPOE file. In some instances, it is useful to set get_gintvs=no for it can reduce processing time considerably. This is helpful when using a very large data set or a large number of GTI's. However, inclusion of gaps in the data set can effect precision greatly, and the user should consider them when determining the best approach to the FFT.
- (power_threshold = -1.0) [real]
When this value is negative, the threshold for which the normalized power values are saved is calculated statistically and set at 50% confidence. This value can be set to a specific positive value that will represent the normalized power threshold value. All values of normalized power above this value will be saved in the _pwr.tab file. The header will list the threshold and significance values in both raw and normalized power units.
- (histogram_bin_size = 0.5) [real]
This specifies the size of the normalized power bins for creating the _ftp.tab file.
- (histogram_no_bins = ) [int]
This specifies the total number of bins to use for the normalized power histogram when creating the _ftp.tab file.
The range of values in the normalized power histogram are dependent on both of the above parameters :
range = histogram_bin_size * histogram_no_bins
- (srcarea = ) [real]
This specifies the area of the region used to extract counts in the input table. Normally, this would be provided in the table header by the xtiming.ltcurv task. It is used for labeling only.
- (binlen = ) [real]
This specifies the bin length used to create the input table. Normally, this would be provided in the table header by the xtiming.ltcurv task. It is used for labeling only.
- (beg_time = ) [real]
This specifies the starting time for the binned data in the input table. Normally, this would be provided in the table header by the xtiming.ltcurv task. It is used for labeling only.
- (end_time = ) [real]
This specifies the stop time for the binned data in the input table. Normally, this would be provided in the table header by the xtiming.ltcurv task. It is used for labeling only.
type 'help timfilter', 'help timsort', and 'help ltcurv' for information concerning input files.1. Perform FFT analysis on light curve output snr3_ltc.tab.
xt> fft Input source file (table or time-sorted qpoe file) : snr3_ltc root name for output file [root_fft(ftp,pwr).tab]: . Input table column heading (src): Looking for TABLE input file: snr3_ltc.tab FFT done with 4096 bins! fft: snr3_ltc.tab src ./snr3_fft.tab Creating raw fft output file: ./snr3_fft.tab Creating histogram output file: ./snr3_ftp.tab Creating fft power output file: ./snr3_pwr.tab2. Using command line format, perform FFT analysis on light curve output snr3_ltc.tab using the ctrt column.
xt> fft "snr3_ltc" "none" "snr3ctrt" "ctrt" Looking for TABLE input file: snr3_ltc.tab FFT done with 4096 bins! fft: snr3_ltc.tab ctrt snr3ctrt_fft.tab Creating raw fft output file: snr3ctrt_fft.tab Creating histogram output file: snr3ctrt_ftp.tab Creating fft power output file: snr3ctrt_pwr.tab3. Perform the FFT analysis on the window function of snr_ltc.tab. Get more verbose output.
xt> fft.display=2 xt> fft Input source file (table or time-sorted qpoe file) : snr_ltc root name for output file [root_fft(ftp,pwr).tab] : snrexp Looking for TABLE input file: snr_ltc.tab Input table column heading (src): exp fft: fft analysis of file snr_ltc.tab, data column: exp, exp thresh: 50.00 Expect to read 100 bins from ltcurv file: snr_ltc.tab Area of photon collection is 1789.0000000000025437 Actually read 100 bins from ltcurv file: snr_ltc.tab FFT done with 128 bins! fft: snr_ltc.tab exp snrexp_fft.tab conf: 0.9949999999999999 crit power density: 326.7184413278046317 conf: 0.9500000000000000 crit power density: 246.4981106773664887 conf: 0.9000000000000000 crit power density: 221.6787553574638192 conf: 0.8000000000000000 crit power density: 195.8206341636832092 conf: 0.5000000000000000 crit power density: 156.8425544942991134 Creating raw fft output file: snrexp_fft.tab Creating histogram output file: snrexp_ftp.tab Creating fft power output file: snrexp_pwr.tab xt> fft.display=14. Perform FFT analysis on the net counts from source file snr_sti.qp and background file snr_bti.qp using a time filter for the first GTI. Do not generate a table of fft coefficients.
xt> fft.fftcoeff=no xt> fft Input source file (table or time-sorted qpoe file) : snr_sti.qp[time=(79486997.99:79488826.47)] root name for output file [root_fft(ftp,pwr).tab]: snr1net Input background qpoe file (none):snr_bti.qp[time=(79486997.99:79488826.47)] Input table column heading (src): net Number of bins ( 0 for binsize ) (0:) (0): 8000 FFT done with 8192 bins! fft: snr_sti.qp[time=(79486997.99:79488826.47)] NET snr1net_fft.tab Creating histogram output file: snr1net_ftp.tab Creating fft power output file: snr1net_pwr.tab xt> fft.fftcoeff=yes5. Perform a summed FFT on source data from GTI#1 and GTI#3 of snr_sti.qp using input list sumdfft.lis. Both data intervals are 1024 seconds long.
xt> ty sumdfft.lis snr_sti.qp[time=(79487397.0:79488421.0)] snr_sti.qp[time=(79492987.0:79494011.0)] xt> fft Input source file (table or time-sorted qpoe file) : @sumdfft.lis root name for output file [root_fft(ftp,pwr).tab]: sumd Input background qpoe file (none): Input table column heading (src): Number of bins ( 0 for binsize ) (0:) (0): 2048 FFT done with 2048 bins! FFT done with 2048 bins! fft: snr_sti.qp[time=(79492987.0:79494011.0)] SRC sumd_fft.tab Creating raw fft output file: sumd_fft.tab Creating histogram output file: sumd_ftp.tab Creating fft power output file: sumd_pwr.tab6. Perform a summed FFT on light curve ouput. Use input list ltcfft.lis. Note that both _ltc.tab files have the same number of bins. The input files are made in the following way :
xt> ltcurv "snr_sti.qp[time=(79487397.0:79488421.0)]" "none" "reg1" 0 0.5 Creating ltcurv file: reg1_ltc.tab xt> ltcurv "snr_sti.qp[time=(79492987.0:79494011.0)]" "none" "reg2" 0 0.5 Creating ltcurv file: reg2_ltc.tab xt> ty ltcfft.lis reg1_ltc.tab reg2_ltc.tab xt> fft Input source file (table or time-sorted qpoe file): @ltcfft.lis root name for output file [root_fft(ftp,pwr).tab] (.): ltcv Input background qpoe file (none): Input table column heading (src): FFT done with 2048 bins! FFT done with 2048 bins! fft: reg2_ltc.tab src ltcv_fft.tab Creating raw fft output file: ltcv_fft.tab Creating histogram output file: ltcv_ftp.tab Creating fft power output file: ltcv_pwr.tabNote : to analyze the "net" of source counts and background counts with the summed FFT method, one must make light curves as in example 6 using background information, and input a list of _ltc.tab files into fft, chosing "net" as column heading. Currently there is not method of inputting a list of _sti.qp and corresponding _bti.qp files into the FFT.
The time will depend on the number of bins for the FFT.
When inputting a list of source files for a summed FFT, there is NO support for a corresponding list of background files. When running FFT, temporary files are written to the current working directory. Space must be allowed for this.
LTCURV - for more info on the input data
TIMPRINT - for more info on displaying the output data
FFTPLOT, FTPPLOT - for more info on plotting the output data