catalogs -- describe the catalog configuration file
A catalog is a large set of tabular data records from which smaller tabular data sets can be extracted by issuing a catalog server query. Catalogs may be installed locally or accessed remotely. Installing a catalog involves creating a record in the catalog configuration file which specifies the catalog network address, the catalog query format, and the catalog query output format. In the following sections the configuration file is described in the context of accessing astrometric catalogs.
A record in the catalog configuration file specifies the network address, the query format, and the output format of each supported catalog server. Catalog server records names have the form "catalog@site", e.g. "usno2@noao". Adding support for a new catalog server or responding to changes in the behavior of an already supported server requires either adding a new record to the configuration file or changing an existing record in the configuration file. It does not require changing the the catalog access code.
The catalog server network address tells the catalog access code where and how to connect to the network. Each network address has the syntax "domain:port:address:flags" e.g. "inet:80:www.noao.edu:text".
The query format specifies the form of the query server string, and the names, default values, units, and format of the query parameters. A set of query parameter names are reserved for accessing astrometric catalogs including "ra", "dec", "radius", "hwidth", "width", "rawidth", "decwidth", rahwidth, and dechwidth. The names of these parameters are not part of the catalog access API. Other types of catalogs may have different reserved query parameter names. The user replaces the default query values with user query parameter values before making the query.
The server query output format specifies the form of the expected server output: including the data stream type, the record size, and the name, location, size, data type, units and format of each field in the record. A set of standard field names is reserved for accessing the output of astrometric catalog servers including "id", "ra", "dec", and "mag[1-n]". These standard field names are not part of the catalog access API. Other catalog types may have a different set of standard field names in the future.
The following two examples illustrate typical catalog configuration file records. Note that both records can be used to access the same catalog data. The first example accesses the catalog as simple text, the latter as blocked text.
Example 1: Accessing the catalog as simple text. begin susno2@noao address inet:80:www.noao.edu:text query GET /cgi-bin/usno/usnoextract?search=yes&ra=%-s&dec=%-s&width=%-s HTTP/1.0 \n\n nquery 4 ra 00:00:00.00 hours %0.2h dec 00:00:00.0 degrees %0.1h radius 5.0 minutes %0.1f qsystem J2000.0 INDEF %s type stext hskip 10 tskip 6 recsize 0 triml 0 trimr 4 nheader 1 csystem J2000.0 nfields 4 ra 1 0 d hours %12.3h dec 2 0 d degrees %12.2h mag1 3 0 r INDEF %4.1f mag2 4 0 r INDEF %4.1f
Example 2: Accessing the catalog as blocked text. begin busno2@noao address inet:80:www.noao.edu:text query GET /cgi-bin/usno/usnoextract?search=yes&ra=%-s&dec=%-s&width=%-s HTTP/1.0 \n\n nquery 4 ra 00:00:00.00 hours %0.2h dec 00:00:00.0 degrees %0.1h radius 5.0 minutes %0.1f qsystem J2000.0 INDEF %s type btext hskip 10 tskip 6 recsize 44 triml 0 trimr 4 nheader 1 csystem J2000.0 nfields 4 ra 1 13 d hours %12.3h dec 14 14 d degrees %12.2h mag1 28 6 r INDEF %4.1f mag2 34 6 r INDEF %4.1f
The beginning of a new catalog record is indicated by a line of the form "begin catname" where catname is a unique name of the form "catalog@site" . If there is more than one record with the same name, the last record is the one that is read. The same catalog server may be accessed in more than one way by creating multiple records. For example if the catalog server supports an optional magnitude selection feature, then in one record this feature can be enabled and in another it can be disabled.
The address , query and nquery keywords are required and define the network address, query command, format and query parameters for the catalog.
The address keyword "domain", "port", and "flags" fields are almost always "inet", "80", and "text" respectively, so in most cases the only address keyword field that has to be filled in is the address field :www.noao.edu" in this case.
The query keyword defines the query command whose form is server dependent. The query parameter values are encoded via the %-s formatting strings. The calling program must encode the user query parameter values into a set a strings which then replace the -%s format statement in the query string.
The number of query parameters is defined by the nquery keyword. The number of query parameters must be greater than or equal to the number of "-%s" strings in the query keyword value. The name, default value, units, and format of each query parameter are listed below the nquery keyword, one query parameter description per line. The query parameters should be defined in the catalog configuration file in the same order that they appear in the query keyword value. Alert readers will notice that in the examples above the number of query parameters is 4 but there are only 3 "%-s" strings in the query keyword value. In these examples the qsystem query parameter, which defines the coordinate system of the ra and dec query parameter values is fixed at J2000. For some servers this parameter may be a true query parameter, i.e. the server may accept coordinates in B1950 or J2000 or some other coordinate system.
For astrometric catalogs the reserved query parameter names "ra", "dec", and "qysystem" should be used to define the extraction region center and its coordinate system, and one or more of "radius", "width", "xwidth", and "ywidth" should be used to define the extraction region size. The units of "ra" should be "hours", "degrees", or "radians", the units of dec should be "degrees" or "radians", and units of the size query parameter should be "degrees" or "minutes". The qsystem parameter value may be any one of the supported celestial coordinate systems. The most common qsystem values are "icrs", "J2000", or "B1950". The query parameter formats are used to convert numerical values supplied by the calling program to string values that can be passed to the query string. It should be emphasized that the reserved query parameter names and units are conventions that are adopted to simplify writing the configuration file and astrometric applications. They are not part of the catalog access API itself.
The type keyword defines the form of the query output. The current options are "stext" for simple text and "btext" for blocked text. Simple text contains newline delimited records with whitespace delimited fields. Blocked text contains newline delimited records and fixed position and size fields. If the type keyword is missing "stext" is assumed.
The recsize keyword is the length of the record in characters including the newline character. If the record size is variable recsize should be set t0 0. If undefined the recsize keyword defaults to 0 for variable record size.
The hskip , tskip , ltrim , and trim define the number of leading and and trailing records in the query output to sky, and the number of leading and trailing characters in each record to trim, i.e. replace with blanks. If absent these keywords default to zero.
The nheader keyword defines the number of header keywords. Header keyword values are global quantities which apply to the catalog server output as a whole. There may be 0 or many header keywords.
For most astrometry catalog the most important and often only header keyword is csystem which defines the coordinate system of the query output coordinates, i.e. if csystem is "J2000" then the coordinates of objects extracted from the catalog are in "J2000".
The nfields keyword defines the number of fields in the query output records. The name, offset, size, datatype, units, and format of each field follow, one field description per line. For simple text files the offset is field or column number and the size is 0 meaning undefined. For blocked text files the offset is the 1-indexed position of the first character (which may be blank) in the field and size is the field size in characters where the field size includes trailing blanks. Using a blocked text description may be required for dealing with fields containing embedded blanks. The type defines the preferred data type of a field. In the examples above the ra and dec are assigned double precision data types. This means that for precision reasons the calling program should but is not required to read these quantities into double precision variables. The units information is used to perform any required coordinate conversions, and the format information is used in cases where the calling program must decode a field, perform some numerical operation on it, and reencode it with the original precision.
For astrometric catalogs the reserved standard field names "id", "ra", "dec", "mag#" etc should be used to define the standard field names. The current standard field name list is id , ra , dec , era, edec , pmra , pmdec , epmra , epmdec , catsystem , equinox , epoch , px , rv , epx , erv , mag , color , emag , ecolor , xp , yp , xc , yc , exc , eyc , imag , and eimag . At a minimum an astrometric catalog must contain the "ra" and "dec" fields. The units of the ra field must be "hours", "degrees", or "radians" and the units of the "dec" field must be "degrees" or "radians". The other standard fields are optional and define quantities like: proper motions in ra and dec, the coordinate system, equinox, and epoch of observation, parallax, radial velocity, magnitude and color information, and predicted image pixel coordinates. The definitions and default units for all these quantities are defined more fully in the help for the astrometry package. It should be emphasized that the reserved field names and units names are conventions that are adopted to simplify writing the configuration file and astrometric applications. They are not part of the catalog access API itself.