vo.table API documentation

vo.table

This file contains a contains the high-level functions to read a VOTable file.

vo.table.parse(source, columns=None, invalid='exception', pedantic=True, support_hexadecimal=False, chunk_size=1024)

Parses a VOTABLE xml file (or file-like object), and returns a VOTable object, with a nested list of Resource instances and Table instances.

source may be a filename or a readable file-like object.

If the columns parameter is specified, it should be a list of field names to include in the output. The default is to include all fields.

The invalid parameter may be one of the following values:

• ‘exception’: throw an exception when an invalid value is encountered (default)
• ‘mask’: mask out invalid values

When pedantic is True, raise an error when the file violates the spec, otherwise issue a warning. Warnings may be controlled using the standard Python mechanisms. See the warnings module in the Python standard library for more information.

When support_hexadecimal is True, integer fields may contain hexadecimal numbers. When False, this part of the spec is unsupported for efficiency reasons.

chunk_size is the number of rows to read before converting to an array. Higher numbers are likely to be faster, but will consume more memory.

vo.table.parse_single_table(source, **kwargs)

Parses a VOTABLE xml file (or file-like object), assuming it only contains a single TABLE element, and returns a Table instance.

See parse() for a description of the keyword arguments.

vo.tree

This file defines the nodes that make up the VOTABLE XML tree.

class vo.tree.CooSys(ID=None, equinox=None, epoch=None, system=None, id=None, config={})

Bases: vo.tree._SimpleElement

A class representing the COOSYS element, which defines a coordinate system.

The keyword arguments correspond to setting members of the same name, documented below.

ID

[required] The XML ID of the COOSYS element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

epoch

Specifies the epoch of the positions. It must be a string specifying an astronomical year.

equinox

A parameter required to fix the equatorial or ecliptic systems (as e.g. “J2000” as the default “eq_FK5” or “B1950” as the default “eq_FK4”).

system

Specifies the type of coordinate system. Valid choices are:

‘eq_FK4’, ‘eq_FK5’, ‘ICRS’, ‘ecl_FK4’, ‘ecl_FK5’, ‘galactic’, ‘supergalactic’, ‘xy’, ‘barycentric’, or ‘geo_app’

class vo.tree.Element

Bases: object

A base class for all classes that represent XML elements in the VOTABLE file.

class vo.tree.Field(votable, ID=None, name=None, datatype=None, arraysize=None, ucd=None, unit=None, width=None, precision=None, utype=None, ref=None, type=None, id=None, config={})

Bases: vo.tree._SimpleElement

A class that represents the FIELD element, which describes the datatype of a particular column of data.

The keyword arguments correspond to setting members of the same name, documented below.

If ID is provided, it is used for the column name in the resulting recarray of the table. If no ID is provided, name is used instead. If neither is provided, an exception will be raised.

ID

The XML ID of the FIELD element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

arraysize

Specifies the size of the multidimensional array if this FIELD contains more than a single value.

See multidimensional arrays.

datatype

[required] The datatype of the column. Valid values (as defined by the spec) are:

‘boolean’, ‘bit’, ‘unsignedByte’, ‘short’, ‘int’, ‘long’, ‘char’, ‘unicodeChar’, ‘float’, ‘double’, ‘floatComplex’, or ‘doubleComplex’

Many VOTABLE files in the wild use ‘string’ instead of ‘char’, so that is also a valid option, though ‘string’ will always be converted to ‘char’ when writing the file back out.

links

A list of Link instances used to reference more details about the meaning of the FIELD. This is purely informational and is not used by the vo package.

name

An optional name for the FIELD.

parse(iterator, config)

precision

Along with width, defines the numerical accuracy associated with the data. These values are used to limit the precision when writing floating point values back to the XML file. Otherwise, it is purely informational – the Numpy recarray containing the data itself does not use this information.

ref

On FIELD elements, ref is used only for informational purposes, for example to refer to a COOSYS element.

to_xml(write, indent)

type

The type attribute on FIELD elements is reserved for future extensions.

ucd

The unified content descriptor for the FIELD.

unit

A string specifying the units for the FIELD.

utype

The usage-specific or unique type of the FIELD.

values

A Values instance (or None) defining the domain of the column.

width

Along with precision, defines the numerical accuracy associated with the data. These values are used to limit the precision when writing floating point values back to the XML file. Otherwise, it is purely informational – the Numpy recarray containing the data itself does not use this information.

class vo.tree.FieldRef(table, ref)

Bases: vo.tree._SimpleElement

A class representing the FIELDref element, which is used inside of GROUP elements to refer to FIELD elements defined elsewhere.

get_ref()

Lookup the Field instance that this FieldRef references.

ref

The ID of the FIELD that this FIELDref references.

class vo.tree.Group(table, ID=None, name=None, ref=None, ucd=None, utype=None, id=None, config={})

Bases: vo.tree.Element

Stores information about the grouping of FIELD and PARAM elements.

This information is currently ignored by the vo package—that is the columns in the recarray are always flat—but the grouping information is stored so that it can be written out again to the XML file.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the GROUP element. May be None or a string conforming to XML ID syntax.

description

An optional string describing the GROUP. Corresponds to the DESCRIPTION element.

entries

A list of members of the GROUP. This list may only contain objects of type Param, Group, ParamRef and FieldRef.

iter_fields_and_params()

Recursively iterate over all Param elements in this Group.

iter_groups()

Recursively iterate over all sub-Group instances in this Group.

name

An optional name for the grouping.

parse(iterator, config)

ref

Currently ignored, as it’s not clear from the spec how this is meant to work.

to_xml(write, indent)

ucd

The unified content descriptor for the GROUP.

utype

The usage-specific or unique type of the GROUP.

class vo.tree.Info(ID=None, name=None, value=None, id=None, config={})

Bases: vo.tree._SimpleElement

A class for storing INFO elements, which contain arbitrary key-value pairs for extensions to the standard.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the INFO element. Used for cross-referencing.

name

[required] The key of the key-value pair.

value

The value of the key-value pair. (Always stored as a string or unicode string).

class vo.tree.Link(ID=None, title=None, value=None, href=None, action=None, id=None, config={}, **kwargs)

Bases: vo.tree._SimpleElement

A class for storing LINK elements, which are used to reference external documents and servers through a URI.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the LINK element.

content_role

Defines the MIME role of the referenced object. Must be one of:

None, ‘query’, ‘hints’, ‘doc’ or ‘location’

content_type

Defines the MIME content type of the referenced object.

href

A URI to an arbitrary protocol. The vo package only supports http and anonymous ftp.

class vo.tree.Param(votable, ID=None, name=None, value=None, datatype=None, arraysize=None, ucd=None, unit=None, width=None, precision=None, utype=None, type=None, id=None, config={})

Bases: vo.tree.Field

A class to represent the PARAM element, which are constant-valued columns in the data.

Param objects are a subclass of Field, and have all of its methods and members. Additionally, it defines value.

to_xml(write, indent)

value

[required] The constant value of the parameter. Its type is determined by the datatype member.

class vo.tree.ParamRef(table, ref)

Bases: vo.tree.Element

A class representing the PARAMref element, which is used inside of GROUP elements to refer to PARAM elements defined elsewhere.

The keyword arguments correspond to setting members of the same name, documented below.

It contains the following publicly-accessible members:

ref: An XML ID refering to a <PARAM> element.

get_ref()

Lookup the Param instance that this PARAMref references.

ref

The ID of the PARAM that this PARAMref references.

class vo.tree.Resource(name=None, ID=None, utype=None, type='results', id=None, config={}, **kwargs)

Bases: vo.tree.Element

A class to store the information in a RESOURCE element. Each resource may contain zero-or-more TABLE elements and zero-or-more nested RESOURCE elements.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the RESOURCE element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

coordinate_systems

A list of coordinate system definitions (COOSYS elements) for the RESOURCE. Must contain only CooSys objects.

description

An optional string describing the RESOURCE. Corresponds to the DESCRIPTION element.

extra_attributes

A dictionary of string keys to string values containing any extra attributes of the RESOURCE element that are not defined in the specification. (The specification explicitly allows for extra attributes here, but nowhere else.)

infos

A list of informational parameters (key-value pairs) for the resource. Must only contain Info objects.

iter_coosys()

Recursively iterates over all the COOSYS elements in the resource and nested resources.

iter_fields_and_params()

Recursively iterates over all FIELD and PARAM elements in the resource, its tables and nested resources.

iter_tables()

Recursively iterates over all tables in the resource and nested resources.

links

A list of links (pointers to other documents or servers through a URI) for the resource. Must contain only Link objects.

name

An optional name for the RESOURCE.

params

A list of parameters (constant-valued columns) for the resource. Must contain only Param objects.

parse(votable, iterator, config)

resources

A list of nested resources inside this resource. Must contain only Resource objects.

tables

A list of tables in the resource. Must contain only Table objects.

to_xml(write, indent)

type

[required] The type of the resource. Must be either:

• ‘results’: This resource contains actual result values (default)
• ‘meta’: This resource contains only datatype descriptions (FIELD elements), but no actual data.

utype

The usage-specific or unique type of the FIELD.

class vo.tree.Table(votable, ID=None, name=None, ref=None, ucd=None, utype=None, nrows=None, id=None, config={})

Bases: vo.tree.Element

A class to store a TABLE element, which optionally contains data.

It contains the following publicly-accessible members, all of which are mutable:

array: A Numpy recarray of the data itself, where each row is a row of votable data, and columns are named and typed based on the <FIELD> elements of the table.

mask: A Numpy recarray of only boolean values, set to True wherever a value is undefined.

If the Table contains no data, (for example, its enclosing Resource has type == ‘meta’) array and mask will be zero-length arrays.

Note

In a future version of the vo package, the array and mask elements will likely be combined into a single Numpy masked record array. However, there are a number of deficiencies the current implementation of Numpy that prevent this.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the TABLE element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

create_arrays(nrows=0, config={})

Create new arrays to hold the data based on the current set of fields, and store them in the array and mask member variables. Any data in existing arrays will be lost.

nrows, if provided, is the number of rows to allocate.

description

An optional string describing the TABLE. Corresponds to the DESCRIPTION element.

fields

A list of Field objects describing the types of each of the data columns.

format

[required] The serialization format of the table. Must be one of:

‘tabledata’ (TABLEDATA), ‘binary’ (BINARY), ‘fits’ (FITS_).

Note that the ‘fits’ format, since it requires an external file, can not be written out. Any file read in with ‘fits’ format will be read out, by default, in ‘tabledata’ format.

get_field_by_id(ref, before=None)

Looks up a FIELD or PARAM element by the given ID.

get_field_by_id_or_name(ref, before=None)

Looks up a FIELD or PARAM element by the given ID or name.

get_group_by_id(ref, before=None)

Looks up a GROUP element by the given ID. Used by the group’s “ref” attribute

groups

A list of Group objects describing how the columns and parameters are grouped. Currently this information is only kept around for round-tripping and informational purposes.

iter_fields_and_params()

Recursively iterate over all FIELD and PARAM elements in the TABLE.

iter_groups()

Recursively iterate over all GROUP elements in the TABLE.

links

A list of Link objects (pointers to other documents or servers through a URI) for the table.

name

An optional name for the table.

nrows

[immutable] The number of rows in the table, as specified in the XML file.

params

A list of parameters (constant-valued columns) for the table. Must contain only Param objects.

parse(iterator, config)

ref

Refer to another TABLE, previously defined, by the ref ID for all metadata (FIELD, PARAM etc.) information.

to_xml(write, indent)

ucd

The unified content descriptor for the TABLE.

class vo.tree.VOTableFile(ID=None, id=None, config={})

Bases: vo.tree.Element

A class to represent the top-level VOTABLE element.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the VOTABLE element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

coordinate_systems

A list of coordinate system descriptions for the file. Must contain only CooSys objects.

description

An optional string describing the VOTABLE. Corresponds to the DESCRIPTION element.

get_coosys_by_id(ref, before=None)

Looks up a COOSYS element by the given ID.

get_field_by_id(ref, before=None)

Looks up a FIELD element by the given ID. Used by the field’s “ref” attribute.

get_field_by_id_or_name(ref, before=None)

Looks up a FIELD element by the given ID or name.

get_first_table()

Often, you know there is only one table in the file, and that’s all you need. This method returns that first table.

get_group_by_id(ref, before=None)

Looks up a GROUP element by the given ID. Used by the group’s “ref” attribute

get_table_by_id(ref, before=None)

Looks up a TABLE element by the given ID. Used by the table “ref” attribute.

get_values_by_id(ref, before=None)

Looks up a VALUES element by the given ID. Used by the values “ref” attribute.

infos

A list of informational parameters (key-value pairs) for the entire file. Must only contain Info objects.

iter_coosys()

Recursively iterate over all COOSYS elements in the VOTABLE file.

iter_fields_and_params()

Recursively iterate over all FIELD and PARAM elements in the VOTABLE file.

iter_groups()

Recursively iterate over all GROUP elements in the VOTABLE file.

iter_tables()

Iterates over all tables in the VOTable file in a “flat” way, ignoring the nesting of resources etc.

iter_values()

Recursively iterate over all VALUES elements in the VOTABLE file.

params

A list of parameters (constant-valued columns) that apply to the entire file. Must contain only Param objects.

parse(iterator, config)

resources

A list of resources, in the order they appear in the file. Must only contain Resource objects.

set_all_tables_format(format)

Set the output storage format of all tables in the file.

to_xml(fd, indent=0)

Write to an XML file.

fd may be a path to a file, or a Python file-like object.

version

The version of the VOTable specification that the file uses. Always “1.1”, since the vo package is designed to only output version 1.1 files.

class vo.tree.Values(votable, field, ID=None, null=None, ref=None, type='legal', id=None, config={})

Bases: vo.tree.Element

A class to represent the VALUES element, used within FIELD and PARAM elements to define the domain of values.

The keyword arguments correspond to setting members of the same name, documented below.

ID

The XML ID of the VALUES element, used for cross-referencing. May be None or a string conforming to XML ID syntax.

max

The maximum value of the domain. See max_inclusive.

max_inclusive

When True, the domain includes the maximum value.

min

The minimum value of the domain. See min_inclusive.

min_inclusive

When True, the domain includes the minimum value.

null

For integral datatypes, null is used to define the value used for missing values. Since the TABLEDATA datatype simply excludes missing values altogether, this attribute is ignored and merely stored for round-tripping support. This attribute should be used for the BINARY format, but is not, currently.

options

A list of string key-value tuples defining other OPTION elements for the domain. All options are ignored – they are stored for round-tripping purposes only.

parse(iterator, config)

ref

Refer to another VALUES element by ID, defined previously in the document, for MIN/MAX/OPTION information.

to_xml(write, indent)

type

[required] Defines the applicability of the domain defined by this VALUES element. Must be one of the following strings:

• ‘legal’: The domain of this column applies in general to this datatype. (default)
• ‘actual’: The domain of this column applies only to the data enclosed in the parent table.

vo.tree.check_astroyear(year, config)

Raises a VOTableSpecError if year is not a valid astronomical year as defined by the VOTABLE standard.

vo.tree.check_string(string, attr_name, config)

Raises a VOTableSpecError if string is not a string or Unicode string.

vo.tree.check_ucd(ucd, config)

Raises a VOTableSpecError if ucd is not a valid unified content descriptor string as defined by the VOTABLE standard.

vo.tree.resolve_id(ID, id, config)

vo.tree.uniqify_names(fields)

Make sure that all names and titles in a list of fields are unique, by appending numbers if necessary.

vo.converters

This module handles the conversion of various VOTABLE datatypes to/from TABLEDATA and BINARY formats.

class vo.converters.Array(field, config={})

Bases: vo.converters.Converter

Handles both fixed and variable-lengths arrays

class vo.converters.ArrayVarArray(field, base, arraysize, config={})

Bases: vo.converters.VarArray

Handles an array of variable-length arrays, i.e. where arraysize ends in ‘*’.

parse(value)

class vo.converters.Bit(field, config={})

Bases: vo.converters.Converter

Handles the bit datatype.

array_type

alias of BitArray

binoutput(value)

binparse(read)

output(value)

parse(value)

vararray_type

alias of ScalarVarArray

class vo.converters.BitArray(field, base, arraysize, config={})

Bases: vo.converters.NumericArray

Handles an array of bits.

binoutput(value)

binparse(read)

vararray_type

alias of ArrayVarArray

class vo.converters.Boolean(field, config={})

Bases: vo.converters.Converter

Handles the boolean datatype.

array_type

alias of BooleanArray

binoutput(value)

binparse(read)

binparse_value(value)

output(value)

parse(value)

vararray_type

alias of VarArray

class vo.converters.BooleanArray(field, base, arraysize, config={})

Bases: vo.converters.NumericArray

Handles an array of boolean values.

binoutput(value)

binparse(read)

vararray_type

alias of VarArray

class vo.converters.Char(field, config={})

Bases: vo.converters.Converter

Handles the char datatype. (7-bit unsigned characters)

static output(value)

class vo.converters.Complex(field, config={})

Bases: vo.converters.FloatingPoint, vo.converters.Array

The base class for complex numbers.

array_type

alias of ComplexArray

output(value)

parse(value)

parse_parts(parts)

vararray_type

alias of ComplexVarArray

class vo.converters.ComplexArray(field, base, arraysize, config={})

Bases: vo.converters.NumericArray

Handles a fixed-size array of complex numbers.

parse(value)

parse_parts(parts)

vararray_type

alias of ComplexArrayVarArray

class vo.converters.ComplexArrayVarArray(field, base, arraysize, config={})

Bases: vo.converters.VarArray

Handles an array of variable-length arrays of complex numbers.

parse(value)

class vo.converters.ComplexVarArray(field, base, arraysize, config={})

Bases: vo.converters.VarArray

Handles a variable-length array of complex numbers.

parse(value)

class vo.converters.Converter(field, config={})

Bases: object

The base class for all converters. Each subclass handles converting a specific VOTABLE data type to/from the TABLEDATA and BINARY on-disk representations.

binoutput(value)

Convert the object value in the native in-memory datatype to a string of bytes suitable for serialization in the BINARY format.

binparse(read)

Reads some number of bytes from the BINARY format representation by calling the function read, and returns the native in-memory object representation for the datatype handled by self.

output(value)

Convert the object value in the native in-memory datatype to a string suitable for serializing in the TABLEDATA format.

parse(value)

Convert the string value from the TABLEDATA format into an object with the correct native in-memory datatype.

class vo.converters.Double(field, config={})

Bases: vo.converters.FloatingPoint

Handles the double datatype. Double-precision IEEE floating-point.

class vo.converters.DoubleComplex(field, config={})

Bases: vo.converters.Complex

Handle doubleComplex datatype. Pair of double-precision IEEE floating-point numbers.

class vo.converters.Float(field, config={})

Bases: vo.converters.FloatingPoint

Handles the float datatype. Single-precision IEEE floating-point.

class vo.converters.FloatComplex(field, config={})

Bases: vo.converters.Complex

Handle floatComplex datatype. Pair of single-precision IEEE floating-point numbers.

class vo.converters.FloatingPoint(field, config={})

Bases: vo.converters.Numeric

The base class for floating-point datatypes.

output(value)

parse

alias of float

class vo.converters.Int(field, config={})

Bases: vo.converters.Integer

Handles the int datatype. Signed 32-bit integer.

class vo.converters.Integer(field, config={})

Bases: vo.converters.Numeric

The base class for all the integral datatypes.

output

alias of str

class vo.converters.Long(field, config={})

Bases: vo.converters.Integer

Handles the long datatype. Signed 64-bit integer.

class vo.converters.Numeric(field, config={})

Bases: vo.converters.Converter

The base class for all numeric data types.

array_type

alias of NumericArray

static binoutput(value)

binparse(read)

vararray_type

alias of ScalarVarArray

class vo.converters.NumericArray(field, base, arraysize, config={})

Bases: vo.converters.Array

binoutput(value)

binparse(read)

output(value)

parse(value)

parse_parts(parts)

vararray_type

alias of ArrayVarArray

class vo.converters.ScalarVarArray(field, base, arraysize, config={})

Bases: vo.converters.VarArray

parse(value)

class vo.converters.Short(field, config={})

Bases: vo.converters.Integer

Handles the short datatype. Signed 16-bit integer.

class vo.converters.UnicodeChar(field, config={})

Bases: vo.converters.Converter

Handles the unicodeChar data type. UTF-16-BE.

static output(value)

static parse(value)

class vo.converters.UnsignedByte(field, config={})

Bases: vo.converters.Integer

Handles the unsignedByte datatype. Unsigned 8-bit integer.

class vo.converters.VarArray(field, base, arraysize, config={})

Bases: vo.converters.Array

Handles variable lengths arrays (i.e. where arraysize is ‘*’).

binoutput(value)

binparse(read)

output(value)

vo.converters.array_splitter

A regex to handle splitting values on either whitespace or commas.

SPEC: Usage of commas is not actually allowed by the spec, but many files in the wild use them.

vo.converters.get_converter(field, config)

Given a Field object field, return an appropriate converter class to handle the specified datatype.

vo.voexceptions

Exceptions and warnings used by the vo package.

exception vo.voexceptions.UnimplementedWarning

Bases: exceptions.SyntaxWarning

Warning emitted when the a feature of VOTABLE is not implemented.

exception vo.voexceptions.VOTableSpecError

Bases: exceptions.ValueError

Error raised when the input XML file violates the spec and there is no good workaround.

exception vo.voexceptions.VOTableSpecWarning

Bases: exceptions.SyntaxWarning

Warning emitted when the input XML file violates the spec, but there is an obvious workaround.

vo.voexceptions.warn_or_raise(message, config)

Warn or raise an exception, depending on the pedantic setting.

vo.util

Various utilities and cookbook-like things.

class vo.util.HomogeneousList(types, values=[])

Bases: list

A subclass of list that contains only elements of a given type or types.

append(x)

insert(i, x)

vo.util.convert_to_write_function(fd)

Returns a function suitable for streaming output.

fd may be:

• a function that writes to a stream, in which case it is returned verbatim.
• a file path, in which case it is opened, and the write() method on the file object is returned.
• an object with a write() method, in which case that method is returned.

vo.util.safeeval(source)

parse a dictionary from a file or file-like object. Only strings and numbers are permitted as primitives and the only containers allowed are lists and dictionaries. This is to ensure that use of this file cannot be used as a security breach.

>>> print safeeval("{'test':'123','hehe':['hooray',0x10]}")
{'test': '123', 'hehe': ['hooray', 16]}

vo.xmlutil

Various XML-related utilities

vo.xmlutil.check_anyuri(uri, config={})

Raises a ValueError if uri is not a valid URI as defined in RFC 2396.

vo.xmlutil.check_id(ID, name='ID', config={})

Raises a VOTableSpecError if ID is not a valid XML ID. name is the name of the attribute being checked (used only for error messages).

vo.xmlutil.check_mime_content_type(content_type, config={})

Raises a ValueError if content_type is not a valid MIME content type (syntactically at least), as defined by RFC 2045.

vo.xmlutil.check_token(token, attr_name, config={})

Raises a ValueError if token is not a valid XML token, as defined by XML Schema Part 2.

vo.xmlutil.fix_id(ID, config={})

Given an arbitrary string, create one that can be used as an xml id. This is rather simplistic at the moment, since it just replaces non-valid characters with underscores.

vo.xmlutil.get_xml_iterator(source)

Returns an iterator over the elements of an XML file. See fast_iterparse for more information.

vo.xmlutil.obj_to_attrs(obj, attrs)

Converts an object with a bunch of attributes into an XML attribute list (as a string).

obj is any Python object

attrs is a sequence of attribute names to pull from the object

If any of the attributes is None, it will not appear in the XML.

vo.xmlutil.write_text_element(write, element_name, content, indent)

Writes out an XML element, wrapping the text as necessary.

write: a function that writes to a stream

element_name: The tag name of the element

content: the text of the element

indent: The number of spaces to indent

vo.xmlutil.xml_escape(data)

Escape &, <, >, “, ‘, in a string of data.