Some Example Serializations of the Quantity ======================================= FOREWORD ======================================= The purpose of this document is to show some valid XML serializations for an IVOA quantity. Before presenting the examples, a few points should be made, as the XML serialization/examples can only go so far to describe what is intended, or may obscure important points/issues. 1. There are 3 types of quantity, "basic", "core" and "standard", and in that order are extensions of the prior type node structure. Extension is via accretion of XML components (whether child nodes or attributes). Thus, as components are optional/have sensible defaults, you should be able to "upcast" a more primitive type of quantity into a higher one. For example, the serialization of a coreQuantity describes both a coreQuantity, and a standard quantity (with some implicit defaults). Conversely, IF a "advanced" level quantity is fairly simple (such as a core quantity that held only one scalar number) then its representation in XML will look exactly like its parent realization, the basic quantity. The components of these types of quantity are given elsewhere, but summarized here, are: Basic Quantity [qid, name, description ] UCD CoordSystem Units DataType Value Accuracy Core Quantity [qid, name, description, size] UCD CoordSystem Units DataType Value | Values | Members AltValues Accuracy Std Quantity [qid, name, description, size ] UCD CoordSystem Units AxesList DataType Value | Values | Members AltValues Accuracy All quantities additionally have an XML id "qid" as well as name and description attributes. A special node, "refQuantity" can be used to reference any quantity (by id) at another point in an XML document. This mechanism is intended to allow to compression of an XML document where one or more quantities are repeated. The multi-dimensional quantities "Core" and "Standard" have a required attribute of "size" which indicates how many _values_ are contained within the quantity. 2. XML serializations of quantities may have the following valid XML node names (excepting extensions the user may make to the schema) with the corresponding XML schema types: XML Node Name XML Schema Complex Types allowed ------------- -------------------------------- basicQuantity basicQuantityType coreQuantity coreQuantityType stdQuantity stdQuantityType quantity stdQuantityType A user should be able to create XML schemata from these types. The new nodes may have limits placed on the quantities such that units, UCD, etc are limited to a preferred settings. For example, the "velocity" element constructed from a "stdQuantityType" may be limited to having units of type "cm/sec^-1" and a UCD for "velocity". The schema would give this "extended" quantity the node name "velocity" in order to identify it in the XML document. In all other aspects, it behaves as an unrestricted quantity. Examples on how to do this in XML schemata will be given in another document. 3. Within a type of quantity, there are a number of simplifying defaults which, for the largest fraction of data, will serve to compress the XML which need be passed; some possible simplifications are noted below. Thus, when some attributes are not specified, they are assumed to fall to some default. For example, if dataType isn't specified, it is assumed to be "a scalar string"; if "units" are not specified, they are assumed to be "unitless". To show a *possible* choice, the following examples *could* be considered equivalent: =and= = or even = (an XML note for newbies: the CDATA section needed to preserve spacing) where the string " my data" are two items of string data " my" and "data". In this case, we must assume that the length, since unspecified (no dataType node), may be gleaned from the "size" on the quantity and the spaces in the CDATA (where, if size is unspecified or 1, it's assumed the spaces *are* part of the string, and if size > 1 then the spaces between the non-space characters are NOT part of the value). 4. Units are represented as strings. There are a variety of standards out in the community right now which we dont want to get embroiled in. Thus, for the time being all examples just have filler "units" elements and the actual strings shouldn't be taken so seriously. 5. Where possible, the structure of the XML serialization should help control how users express their data so that it's "scientific"; BUT this is a NOT requirement/priority on the serialization. It becomes hard to impossible to formulate a serialization which is both richly expressive, has re-usable components AND is scientifically policing at the same time. Some police work, of course, can and should be done, but we should not lose sight of the fact that having a sufficiently rich serialization which can be transmitted and understood in less than the age of the universe is the priority here. 6. We allow 2 prescriptions for serialization of single scalar string meta-data in the model: either as PCDATA within a child node or as an attribute on the parent node. Why both? The first type is so that the user can control whitespace as well as allow for some re-use of the quantity model. The second case because when white-spacing isn't an issue, attributes allow for the user to have a '1-line' serialization which is more compact in form than using child nodes. Thus, given the advantages of each, we allow the user to have either (but *not both* on the *same* node). This means that the following are "equivalent" prescriptions: John Q. User ... Meta-data in the model which certainly "qualifies" for this dual treatment includes "name", "description", and "UCD". As "size" is an integer attribute, it does NOT have this mechanism. Furthermore, you are not allowed to use both prescriptions within the same element, e.g. Joe ... is clearly illegal by this rule. 7. Populating the data-cube (or matrix) of the quantity from the serialized "values" sections is assumed to be done in an implicit ordering. The rule is that the order of the axes in the serialization indicate the "fast" to "slow" axes. For example then, the following 2 dimensional matrix: A B C D E F is described by the vertical/horizontal i,j axes which have respective sizes "2" and "3". The serialization of this matrix and its data would be then: 0 1 2 0 1 A B C D E F where "j" (horizontal) axis is the 'fast' axis. Thus, the locations in the matrix are populated in [i,j] order of [0,0], [0,1], [0,2], [1,0], [1,1] and [1,2]. By reversing the order of the axes, we would get the data into the matrix as: A C E B D F Because data are stored as strings in PCDATA section, we are assuming that whitespace is unimportant, and as is the XML spec, any multiple whitespace will be collapsed into single whitespaces at parse time. For the purposes of the "implicit" rule, all whitespace indicates a delimiter between values. This rule presents an obvious problem of how to encapsulate multi- dimensional string data that has spaces/tabs/etc embedded within the value. Clearly, in order to treat this case, we must resort to some fixed width or alternative delimiter symbol rule for parsing the values, and this is not currently given. As an alternative delimited parsing rule, a good choice, for example, is to separate fields within the values via some XML tags, e.g. a 3 field string could be: whitespace doesn't matter here Or in this field either Clearly, CDATA sections may be used between these tags to preserve whitespace within fields, and the API will have to allow this control. It is also clear that this is not the most compact/efficient means of transporting (or parsing!) the values, so some sort of fixed width parsing solution is also desirable as an alternative, especially for passing along large datasets. Serializations presented here will need to have additional, explicit IO information in order to have the ability to wrap outside files (whether ASCII or binary) or to do fixed with parsing. We leave this for (near) future work. 8. Equivalence principle between single and multi-dimensional quantities: It may be desirable to create a serialization where something built in the multi-dimensional way would have "equivalence" in the atomic representation. For example, the matrix: A B C D which has a 2-d velocity, e.g. V(x,y), would look like in the multi-dimensional representation : cm/sec^-1 velocity:ucd 10.1 12.1 10.2 14.2 A B C D is really a compact description of 4 atomic (aka Basic) quantities, each with its own meta-data which are also atomic in nature, e.g. 10.1 10.2 cm/sec^-1 A 12.1 10.2 cm/sec^-1 B 10.1 14.2 cm/sec^-1 C 12.1 14.2 cm/sec^-1 D So what has happened in order to make the constituent atomic quantities is that the parent matrix quantity has been broken apart with axes quantities turned into meta-data in each atomic quantity. In principle, by analyzing the handful of atomic quantities, they could be reassembled into the matrix quantity again, if that is desired, but this is a relatively difficult task to do in general, and it is not required that parser of the serialization be able to do this "reverse" composition task. 9. Referencing mechanism. We will need to consider id/idref mechanisms for referencing one quantity within a document by another. In particular, the use of the reference mechanism in order to "compress" repeated information by quantities is highly desirable. ======================================= EXAMPLES ======================================= With these working assumptions, and with the agreements on the interface, and theoretical structure of the quantities, the following use-cases will be serialized to show examples of how we might do things. Summary of use-cases: 1. A single scalar value 2. A single vector value with 2 components 3. A single tuple value 4. A single object as data of a quantity 5. A list of string (scalar) values 6. A 2-d quantity 7. A 3-d quantity 8. A 2-d quantity described with alternative axes 9. A single scalar value with alternative value frame 10. A 2-d quantity of scalar values with alternative value frame 11. A 2-d quantity of vector values 12. A 2-d tabular quantity 1. Single Scalar Value This can, of course be handled by the basic quantity and above. To give an idea of compression/defaults the following examples are equivalent: Author name of Author John Q. Public =or= John Q. Public =or= name of Author John Q. Public 2. A single vector value with 2 components Depending on what we want to allow in DataType for basicQuantity, single vectors may be handled by all quantities. In this example, the "dataType" is "vector" which is a node which requires one or more "value frame" components to describe the vector. In the following example the velocity vector "10.34x, 12.81y" is described. Note: one issue with this approach: the parent quantity units can be different/in conflict with those of the child value frames. The solution is to put in the schema a rule that the units node doesn't appear when the dataType is of sub-type "vector". 10.34 1281. 3. A single tuple value Here, we have chosen to express the tuple as the fairly common celestial sky quantity. When information of the quantity is a "tuple" (association of one quantity to one or more other quantities), then the "values" + "dataType" nodes are replaced with a "members" node. This will prevent people from expressing tuples as obj1 + obj2 + number. Thus, the tuple prescription looks like: _whatever_appropriate_here _whatever_appropriate_here In order to make this example clear, let's show the referenced Ra/Dec quantities, so ra_ucd_in_here COORD_SYSTEM STUFF HERE hr:arcmin:arcsec hr arcmin arcsec 20 30 49.23 de_ucd_in_here COORD_SYSTEM STUFF HERE deg:min:sec deg min sec -89 30 21.22 4. A single object as the data of a quantity In this example, the member is any old object you like (rather than a quantity). This may be out of spec for the quantity just now, but nevertheless it's interesting to consider. (Note: as this example is conceived, you would need to extend the quantity parser in order to load the object serialization). XML binding may also be used to, at least, create a working "data" container (in other words, it will be an object with accessor methods cooresponding to the information presented, but no other methods that the class may have). If that extension isn't present, then based on the mode the quantity parser is running in, an warning or error might be thrown (or the aforementioned simple XML binding could be used).