Scanpi User's Guide

Overview of Scanpi

The Scan Processing and Integration tool (Scanpi, sometimes referred to as ADDSCAN for historical reasons) performs 1-dimensional (in-scan) scan averaging of the IRAS raw survey data. The IRAS raw scan data, aka the Calibrated Reconstructed Detector Data (CRDD) are stored in files merged with the IRAS pointing (boresight) information. This collection of IRAS survey data is known as the Level 1 Archive, and it is stored in compressed format on an archive server maintained at IRSA.

Scanpi averages the IRAS scans with various weighting schemes. It is used to obtain the fluxes of extended, confused or faint sources, and to estimate true local upper limits. The sensitivity gain is comparable to that obtained in the IRAS Faint Source Survey -- about a factor of 2-5 over the IRAS Point Source Catalog (PSC), depending on the local noise and number of scans crossing the target position. Scanpi is also useful for diagnosing source extent.

The "classic" Scanpi has been upgraded and released (version 6.0) to the public in September 2007. This User's Guide and associated documentation have been updated to reflect the changes. A list of the differences between the classic and new Scanpi are also documented. Parameter settings for the new Scanpi are described below; default parameter settings for the classic Scanpi are listed in red. Scanpi is also available using a Program Interface, where the user can access Scanpi at IRSA through HTTP program interfaces, rather than a web form.

Input Parameters

The start-up page allows users to enter "Single Location" coordinate/object name or a list of sources or positions via "Upload Table of Sources".

Most of IRSA's services allow users to enter single objects or coordinates into a web form, or they can upload a file containing multiple objects and coordinates. The latter method is called "Table Upload," for which IRSA accepts four ASCII table formats:

• IPAC ASCII Column-Aligned (Download template)
• Comma-Separated Values (CSV) or Comma-Delimited (Download template)
• Tab-Delimited (Download template)
• A Simple List of Objects (such as astronomical source names) (Download template)

There are two ways to ensure your tables will work with IRSA's services:

1. Run your tables through IRSA's Table Reformat and Validation tool to identify possible errors.
2. Read and follow the Using Tables With IRSA Services help document to learn and troubleshoot table formats, including:
   • Detailed descriptions of the four acceptable table formats
   • A description of how sexagesimal and Galactic coordinates are handled by IRSA services
   • Best practices for creating tables
   • Troubleshooting solutions for common error messages

The front page also allows the users to change processing parameters. Below is a list of all the parameters; the default values are listed for both the new (in black) and classic (in red) version of Scanpi.

Fitting Range Parameters for 4 bands are described below. All parameters are +/- of the source center. A visualization of all the fitting ranges is available.

Processing

At the start of Scanpi processing, all scans passing within approximately the "Maximum Allowed Distance From Detector Center" are extracted from the IRAS orbit files. This allows the processing to narrow down the scans which were "close" to the source, from millions to a few hundred. "Maximum Allowed Distance From Detector Center" is set to 2.2 arcmin [for classic and new Scanpi]. The precision of the orbit files is about 0.5 arcmin, and therefore this first Scanpi parameter is just a coarse check for scans within the vicinity of the source. More precise calculations are performed once this subset of scans is read and scans within 1.7 arcminutes [the same default as classic Scanpi] (user parameter setting called "Cross Scan Distance") of the target position are definitively identified. Then the data are interpolated using cubic splines and resampled at 10 samples per arcminute in all bands (the detectors were actually under-sampled at about 4, 4, 2, and 1 samples per arcminute at 12, 25, 60 and 100 microns). The resampling is primarily to facilitate alignment of the scans.

A baseline is fit to each individual scan. The residual noise away from the source is computed using the region given by "Source Exclusion Range" (defaulting to +/-2, +/-2 +/-4 and +/-6 arcminutes [classic Scanpi: +/-2, +/-2 +/-2.5 and +/-4 arcminutes] at 12,25,60,100 microns). A point source template is fit to each scan.

The scans are averaged four ways: a mean with noisy detectors half-weighted (aka scan #999), a straight mean (aka scan #1001), a median (aka scan #1002) and a noise-weighted mean (aka scan #1003). These averaged scans are analyzed to find the peak, determine if a believable signal exists, find the zero-crossings, compute the integrated fluxes, fit a point-source template, determine point-source template flux, and compute the full width at quarter and half maxima.

The default output consists of plots of the non-baseline-subtracted averaged/coadd scans with any successful template fits indicated. [classic Scanpi: coaddition scan #999 was plotted with the baseline, while the other three scans were plotted with the baseline subtracted] Also available in the results set are scan track plots illustrating the geometry of the included scans. (All scans are averaged as if they were parallel. Scans which are nearly opposite in direction compared to the north-most scan are reversed before averaging.) A table of widths and flux estimators is also printed. The flux estimators include the integral between the zero crossings, the integral between fixed points, an estimate based on the peak flux, and the flux from the point source template as fit.

Output

Default Scanpi processing will produce, for each of the four IRAS wavelength bands, coaddition plot overlays and coadd summary tables of the primary fitting parameters. This page is called "Results Overview". From the Result Overview page, the user can click on "View, Select and Rerun", in order to view the single scan data, profile plots, the coadds, and scan tracks, or to deselect scans or reset fitting parameters to rerun the processing. This page is called "Result Details".

In the "Result Details" page, for default processing, there will be one Summary Table per band; one entry for each individual scan and 4 averaged scans (averaged scan tabular information is presented in blue table cells, both on the "Results Overview" and "Result Details" pages.)

Each summary table contains the following columns:

Profile plots are made for each individual scan and for the four averaged/coadd scans. Selecting "Show Plots of Intermediate Steps" will create an additional web page of all the plots of each major Scanpi processing step. The plots are labeled by the type of fit being displayed, the band number and either the individual scan name or the coadd name/number. Individual scan names are characterized in the Scanpi file description README; in brief, the names take this format:

b#_sop_obs_det_suffix
b#_###_###_##_suffix

"b" followed by the band number, followed by the sop (Satellite Operating Plan) number, followed by the observation number, followed by the detector number, followed by a suffix which is unique for every processing step.

In the Scan Profile Plots, the horizontal axis is the distance from the target position in the in-scan direction in arcminutes; the vertical axis is flux, indicated in either Jy or W/m**2. [classic Scanpi: Note that the scans may be flipped (+/- of zero on the x-axis) compared to the results of classic Scanpi.]

In the individual Scan Profile Plots the datapoints are in black and the template fit is in red. The flux and template fit have been background subtracted; columns plotted from the data in the 'Table' file are "in_scan" vs "flux" and "TMPFIT".

In the Coadd Scan Profile Plots the datapoints (including the background flux) are in black, background fit (before subtraction) is in red and the template fit (including the background level) is in cyan. The flux and template fit have not been background subtracted; columns plotted from the data in the 'Table' file are "in_scan" vs "flux_wbg", "YFIT" and "TMPFIT_wbg".

[classic Scanpi: The plot for SCAN 999 showed the interpolated and resampled data with the baseline left in and the baseline fit indicated. The other three averaged scans (SCANs 1001, 1002 and 1003) have the baseline fit subtracted but indicate the signal range and the background range. All 4 types of plots (and the individual scans, if requested) show the template fit, if a template was successfully fit to the data.]

A link is provided to the individual scan data files as well as to the coadd data files; the files are in IPAC Table Format. The individual scan data files contain more header keywords, which are carried over from the IRAS Level 1 data files; most of these header keys are not relevant for coadded scans and are therefore dropped, except the band number, coadd scan number and sigma of the background. The tables have the following columns:

• in_scan - distance from the source center in arcminutes
• flux - flux in Jy (background subtracted)
• flux_wbg - flux in Jy (non-background subtracted flux)
• YFIT - flux in Jy of the fit of the background level
• TMPFIT - integrated flux of source as determined by the fit to template, after background subtraction
• TMPFIT_wbg - same as TMPFIT without background subtraction.

The fluxes presented in the Summary Results Tables for columns "peak", "fnu_t", "fnu_z" and "amp" have had corrections applied as follows:

• Units Conversion to Jy: Fluxes were converted from W/m^2 (Level 1) to Jy. The equivalent band passes are (13.48, 5.16, 2.58 and 1.0) x 10^12 Hz.
• Point Source Correction: The raw fluxes (listed in the data files) were multiplied by the AC to DC correction factor. The IRAS detectors had a dwell-time dependent responsivity, such that the calibration for very extended sources is different from the calibration for point sources. The raw fluxes are DC calibrated (i.e. for very extended sources) and must be corrected for point sources. The correction factors used are listed at the top of the Scanpi tabular output and are about 1.308, 1.277, 1.060 and 0.98 at 12, 25, 60 and 100 microns.
• Detector Widths Correction: In addition, the integrated fluxes ("fnu_z" and "fnu_t") were normalized by the integrated point source response which is 0.82, 0.844, 1.437, and 3.234 at 12, 25, 60 and 100 microns.

Scan Track Plots

Scan tracks (showing the paths of the detector centroids) are plotted for all the data. The user must click on the "Scan Tracks" link(s) to request an on-the-fly plot of the scan tracks. The background reference image is either ISSA (bands 1, 2, 3, and 4) or DSS (DSS2 R band), depending on the user selection. Please note that the ISSA images are in EPOCH B1950 while DSS are in EQUINOX 2000. Scan tracks on ISSA backgrounds take approximately 3 seconds to generate since the FITS images are housed at IRSA. The scan tracks on DSS backgrounds take over 30 seconds to generate since the FITS images must be requested from the Multimission Archive at STScI (MAST).

The scan tracks plots are in the usual astronomical convention, with North up and East to the left. Each scan is labeled by scan number at its start in red. Scans selected (for coaddition) are in cyan; scans deselected (from coaddition in a rerun) are in orange; the sky grid is in blue; the user input source location is given by magenta cross-hairs next to the word "Source".

[classic Scanpi: Scan tracks are plotted using RA offset from source center and DEC offset from source center, without a background image.]

Plots of Intermediate Steps

If the user selects "Show Plots of Intermediate Steps" on the Scanpi front page, the "Results Overview" page will have a link to "Plots of Intermediate Steps" at the top of the page. Clicking on that link opens a page of Summary Step-by-Step listing of plots, starting from the raw scan data to the very last step of source template fitting. Clicking on any of the plots brings up a larger copy of that plot. The scans are in the same order as on the "Results Details" web page, with single scans at the start of the table, and the coadd scans at the end, one table per band requested.

The following table described each of the columns:

Reruning Scanpi

The "Result Details" web page allows the user to view the results as well as rerun the coaddition (for any of the bands), using any number of the individual scans, by selecting the ones relevant and deselecting the ones not needed. The ranges for the source and background fitting can also be adjusted when rerunning Scanpi.

Deselecting all the scans for one band is interpreted as a request not to perform coaddition; rerunning with that condition will not produce any of the coadd scan results. Deselecting any of the bands using the "Rereun band #" checkbox, will exclude results for that band from the summary pages in the subsequent run; you can re-select "Rerun band #" in the subsequent run in order to bring back the processing for that band if needed. Note that deselecting any of the bands on the original search form, using "Process IRAS Band", excludes that band from all results and subsequent reruns.

Once done with the selection/deselection of scans, click on the "Rerun Scanpi" button at the bottom. The form can also be reset to the previous settings should one need to go back (using the "Reset Form" button at the bottom.) Lastly, the "Start Over" button will return the user to the Scanpi front page which has the search form.

Table Upload - Multiple Results & Bulk Download

When using the Table Upload feature of running Scanpi on multiple sources, the fitted results are presented in tabular form. The summary table contains the same information in the user table, plus position information for each source, as well as a summary of Scanpi results. The columns in this table are as follows: the first two columns are the "ra" and "dec" (or "lon"/"lat") positions of each source (decimal degrees), then there is a copy of the columns in the user input table, followed by a "ResultLink" which points to a web page of Scanpi results for one source; the next column is "SuccessStatus" where 'YES' indicates that the processing was successful and 'NO' when there was an error; the next column is the user requested "Band", followed by the Scanpi fit results "sigma", "snr", "peak", "fnu_t", "miss", "amp" and "corr_coeff".

The user can click on each "ResultLink" to view the Scanpi summary page for each requested source. Alternatively, the user can usea script created by IRSA, for bulk download of all or a select-set of results. The script uses Wget to download all the Scanpi results per source. You only need to download the script to your machine. The scripts contain one download command per input source. You can edit the script to comment out any sources which you do not want to download. The commands are structured to perform the download recursively for each source (directory of data; named using the position of each source). Once you are finished running the download script, you will see one directory for every source you requested; in those directories is a README file which describes all results files and subdirectories.

Interpretation of Results

[classic Scanpi: The entire section below pertains to the classic version of Scanpi and has not been updated with analysis using the new version. The section should be in red, but has been left in black since these issues are still relevant to the new version of Scanpi. Note that the new version uses a different calculation for "sigma" (see above); therefore some of the sigma cutoff values listed below may not apply. ]

Scanpi provides several flux estimators. While it is clear that integrated flux estimators are better for extended sources, it is NOT the case that the template amplitude is always the best choice for point source fluxes. Some rules of thumb for choosing flux estimators are as follows:

For very strong point sources (> 20 Jy), the peak amplitude is a good estimator. In this case the template amplitude is affected by hysteresis, which will put "wings" on the source, which may elevate or tilt the template relative to the large-scale baseline.

For moderate point sources (1 to 20 Jy) the best estimators are the template amplitude, peak amplitude, or fnu(t). fnu(t) is obtained by integration within the signal range specified by the user. These three estimators should be in fairly good agreement for moderately strong point sources. Under some circumstances the template fit may be corrupted by background emission or noise, rendering it less valuable as a flux estimator.

For weak point sources (less than 1 Jy at 12, 25, and 60 microns and less than 2 Jy at 100 microns, SNR of a few) the best estimator is the template amplitude. Note that the cubic spline fitting can cause some "ringing" which tends to enhance weak point sources pushing their amplitude up by as much as 20% and their half-maximum width down. This is one of the main reasons for preferring template amplitude for weak sources.

For bright extended sources, fluxes are best estimated by fnu(Z), the integrated signal between the zero-crossings. However, if the SNR is less than about 10, it becomes difficult to identify reliable zero crossings, and fnu(t) may be preferable.

To decide if a source is extended (in-scan) one of two tests can be used: the integrated flux is substantially larger than the peak flux by several SIGMA, or the width of the signal is significantly larger than the expected width for a point source (Fullmer, L. and Helou, 1987). The second test is best carried out for the 25% width rather than the 50% width if the SNR is sufficiently high (>20).

There is no general rule as to which of the averaging methods produces the best flux estimates. The noise-weighted mean (1003) should minimize the noise, but is more vulnerable to miscalibration. What is most important is that the results from 1001, 1002, and 1003 should agree within about 1 SIGMA or so. If they disagree by more than this the individual scans should be examined more closely for peculiarities. Beyond that, the median is probably the most consistently "good" estimator of the three, chiefly because of the non-Gaussian nature of noise in the IRAS data.

References and Additional Materials

Beichman, C.A., Neugebauer, G., Habing, H.J., Clegg, P.E., and Chester, T.J. (1988) "IRAS Catalogs and Atlases: Explanatory Supplement". (Washington, DC: GPO)

Fullmer, L. and Helou, G., (1987) "Statistical Characterization of Scanpi. Chapter I. Point Source Widths at 12microns", IPAC memo.

Helou, G. and Kahn, I., (1986) "All About Scanpi", IPAC memo.

Helou, G. et al, (1988), Ap. J. Suppl., 68,151

Kopan, E. and Helou, G., (1988) "Calibration of the CDC Production Version of Scanpi", IPAC memo.