Extensible 3D (X3D) encodings
Part 3: Compressed binary encoding
4 Concepts
This clause describes key concepts in this part of ISO/IEC 19776. This includes how the X3D constructs defined in ISO/IEC 19775-1 are encoded as a compressed binary file.
Table 4.1 lists the topics is this clause.
Table 4.1 — Topics in this clause
The following conventions are used throughout this part of ISO/IEC 19776.
Italics are used for event and field names and are also used when new terms are introduced and equation variables are referenced.
A fixed-space
font is used for URL addresses and source code
examples. ISO/IEC 19776 Classic VRML encoding (see
ISO/IEC 19776-2) examples appear in bold,
fixed-space
font.
Node type names are appropriately capitalized (e.g., “The Billboard node is a grouping node...”). However, the concept of the node is often referred to in lower case in order to refer to the semantics of the node, not the node itself (e.g., “To rotate the billboard...”).
The form "0xhh" expresses a byte as a hexadecimal number representing the bit configuration for that byte.
Throughout this part of ISO/IEC 19776, references are denoted using the “x.[ABCD]” notation, where "x" denotes which clause or annex the reference is described in and “[ABCD]” is an abbreviation of the reference title. An exception to this convention is that International Standards are referenced by the number of the standard.
EXAMPLE 2.[ABCD] refers to a reference described in Clause 2 and C.[ABCD] refers to a reference described in Annex C.
This X3D encoding provides a compact transmission format that minimizes delivery size and maximizes parsing speed while following the precepts of XML (see 2.[XML]). Privacy and ownership protections are optionally provided using XML security mechanisms.
There is an essential one-to-one match between the nodes and fields of the abstract X3D scene graph and corresponding constructs in the Compressed binary encoding. This section describes the design patterns that govern the correspondence.
The following compression steps (as depicted in Figure 4.1) are performed as part of the Compressed binary encoding. The actual processing order and methods of accomplishment may vary between applications. An implementation may skip any step so long as the resultant compressed binary file is the same as if these steps were followed.
The following decompression steps (as depicted in Figure 4.2) are performed as part of the Compressed binary encoding. The actual processing order and methods of accomplishment may vary between applications. An implementation may skip any step so long as the resultant compressed binary file is the same as if these steps were followed.
A registry of node compression methods is maintained under, and using the established procedures of, the ISO International Registration Authority for Items1. A file conforming to this part of ISO/IEC 19776 includes only compressors that have been standardized or have been approved for inclusion in the registry. See Annex B Node compressor registry for more information on registered node compressors.
ISO/IEC 24824-1 specifies the technique used to encode a binary stream representing an X3D scene graph. This technique has the name of Fast Infoset. Conceptually, an XML-encoded canonical X3D document (possibly further modified by node compressors) is passed to a Fast Infoset processor to write the binary stream.
1Contact information for the ISO-designated Registration Authority for Items registered under the ISO/IEC 9973 procedures is available at the ISO Maintenance Agencies and Registration Authorities web site: http://www.iso.ch/iso/standards_development/maintenance_agencies.htm.
An X3D file is structured as specified in ISO/IEC 19775-1.
Conceptually, the X3D scene input to the Fast Infoset encoder is an XML-encoded document with certain restrictions. X3D canonical form eliminates file ambiguities that have no impact on the 3D content but which otherwise would negatively impact security issues, compression or parsing performance.
X3D canonical form is based on Canonical XML (see 2.[XML-Canonicalization]) which specifically allows modification to the default XML canonicalization rules. This provides the ability to establish equivalence between differently formatted (but functionally identical) XML documents. This capability is required for the application of XML Encryption (see 2.[XML-Encryption]) or XML Signature (see 2.[XML-Signature]) syntax and processing techniques.
The following X3D canonicalization restrictions are applied to an X3D scene (or scene fragment) prior to encryption, signature or compression:
EXAMPLE 1
<NavigationInfo type='"WALK" "EXAMINE" "ANY"'/>
′
character entity."
character entity (and escaped by a leading backslash
"\" character).EXAMPLE 1
<Text string=' "\"Hello, quotation marks\"" "Line 2
has no quotation marks" '/>
displays the following
two lines:
“Hello, quotation marks”
Line 2 has no quotation marks
A default or substitute DTD as specified in Annex A of
ISO/IEC 19776-1
is included following the
<?xml version="1.0" encoding="UTF-8"?>
header in canonical form.
NOTE 1 The default DTD is not included in the final Compressed binary encoding, only substitute DTD values are compressed.
Default or substitute
X3D schema (see Annex B of ISO/IEC 19776-1) attributes are included in the root
<X3D>
element.
NOTE 2 The default X3D Schema attributes are not included in the final Compressed binary encoding, only substitute X3D schema attribute values are compressed.
EXAMPLE 2
2.004e3
(equal to 2004.0).
DEF
, USE
and (non-default)
containerField
shall appear before other attributes, which then follow in alphabetic order.
This ordering typically provides higher parsing performance during subsequent decoding.
This step supersedes the XML canonicalization
rule that all attribute values are provided in alphabetic order.
All MFNode content for a child field shall be provided in a contiguous block with no intermixed containerField usage.
EXAMPLE 3 The following code exhibits X3D not in this form:
<Collision> <Shape containerField="children" /> <Shape containerField="proxy" /> <Shape containerField="children" /> </Collision>
The canonical form with proper child-element grouping is:
<Collision> <Shape containerField="proxy" /> <Shape containerField="children" /> <Shape containerField="children" /> </Collision>
EXAMPLE 4 The construct:
<Group DEF="someDEF" class="someClass"></Group>
is converted to:
<Group DEF="someDEF" class="someClass"/>
EXAMPLE 1 <?xml encoding='finf'?>'11100000000000000000000000000000000'
EXAMPLE 2 <?xml version='1.0' encoding='finf' standalone='yes'> '11100000000000000000000000000000000'
Other XML header elements like the DTD or schema definition are encoded using standard Fast Infoset mechanisms.Two additional fields are appended to the X3D tag to denote Compressed binary version information. These attributes are:
<X3D binaryVersion="3.2"
serializerVersion="Fast Infoset 2004"
>
</X3D>SFString
SFString
[initializeOnly]
[initializeOnly]
All Prototype-related elements are treated in a manner consistent with the representations of the X3D XML encoding. Thus, Prototype instances are treated as PrototypeInstance constructions in XML.
EXAMPLE
<ProtoDeclare name="ExampleSensor"> <ProtoInterface> <field accessType="initializeOnly" name="enabled" type="SFBool" value="true"/> </ProtoInterface> <ProtoBody> <TouchSensor description="touch to activate"> <IS> <connect nodeField="enabled" protoField="enabled"/> </IS> </TouchSensor> </ProtoBody> </ProtoDeclare> <ProtoInstance name="ExampleSensor"> <fieldValue name="enabled" value="false"/> </ProtoInstance>
The following "native-tag" representation is not legal and shall not be used within the Compressed binary encoding:
<ExampleSensor enabled="false"/>
Node compressors are responsible for compressing a node and all its children. Typically, this compressor has domain-specific knowledge about how to compress a node's data.
EXAMPLE Geometric compression techniques may be used to store an IndexedFaceSet more compactly.
The compressor shall restore the exact same scenegraph structure as was compressed, including any ProtoInstance usage. In addition, no reordering of data is allowed; i.e., vertex values in a Coordinate node point array shall not change ordinal position.
A compressor shall only be assigned to handle a single node and all its children. Other constructs, like Prototypes and Routes, may not be compressed using node compressors. Furthermore, nodes that implement the X3DScriptNode or X3DProgrammableShaderObject shall not be assigned a node compressor. A node compressor can be used to compress the child nodes of these node types.
Node compressors shall not be nested. A provided node compressor shall handle all of the children nodes for that node type. A compressor may leave a node or field unaltered in the compressed stream.
A node compressor transforms a scenegraph fragment using the following steps:
EXAMPLE The following illustrates compressing an IndexedFaceSet
An IndexedFaceSet containing a MetadataString and a Coordinate node are to be compressed using a geometric compression algorithm (encoding 1). The input scene graph looks like this:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue">
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
The geometric encoder compresses and encodes the coordIndex and point fields into the payload section. The resulting form sent to Fast Infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value=" the compressed and encoded coordIndex and point fields ">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="rate" value="12:1"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="1.0 2.0 3.0 4.0 5.0 6.0">
</IndexedFaceSet>
Then the final form parsed by a decoder engine of X3D files encoded in the Compressed binary encoding is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue"/>
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
An optional MetadataString may be specified before the encoding MetadataInteger name using the keyword "preserved". If the corresponding "preserved" keyword is specified, the decompressed form shall retain all ".x3db" MetadataSet data including the payload in addition to the uncompressed values in the original node. The user's metadata is retained but will not be returned to its original location.
EXAMPLE The following example illustrates compressing an IndexedFaceSet with preserved parameters
An IndexedFaceSet containing a MetadataString and a Coordinate node is to be compressed using a geometric-compression algorithm (encoding 1). The input scene graph looks like this:
<IndexedFaceSet coordIndex="...">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="preserved" value="true">
<MetadataInteger name="encoding" value="1"/>
<MetadataSet name="Encoder Metadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The compressed X3D canonical form sent to Fast Infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value="the compressed and encoded coordIndex and point fields">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The final form parsed by the X3D engine is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value="myValue"/>
<MetadataInteger name="encoding" value="1">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="lossy" value="true"/>
<MetadataString name="coordinateBits" value="15"/>
<MetadataString name="normalBits" value="6"/>
<MetadataString name="colorBits" value="9"/>
<MetadataString name="textureBits" value="9"/>
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="1.0 2.0 3.0 4.0 5.0 6.0"/>
</IndexedFaceSet>
An X3D Compressed binary processor compressing a node that already contains a MetadataSet with the name ".x3db" and reference "Web3D" shall generate the payload block using the specified encoder and Encoder Metadata parameters. Any unknown or incorrect values may be ignored for compression purposes.
Further type-specific field compression is specified in 5 Encoding of fields. Table A.2.5 specifies the indices to use for each algorithm.
This part of ISO/IEC 19776 uses Fast Infoset to serialize and compress an X3D document. It uses several techniques that reduce the size of an X3D document and that increase the speed of creating and processing such documents. These techniques are based on the use of vocabulary tables that allow small (typically) integer values (vocabulary table indexes) to be used instead of character strings.
EXAMPLE 1 Character strings that form the names of nodes or fields in an X3D document.
The tables that apply to this part of ISO/IEC 19776 are specified in Annex A Fast Infoset tables.
A further optimization uses the encoding algorithm vocabulary table. This table identifies specialized encodings that can be employed for commonly occurring strings, again with a number of built-in algorithms.
EXAMPLE 2 If there is a string that looks like the decimal representation of an integer in the range -32768 to 32767, that string can be encoded by identifying that this vocabulary table is being used, giving the vocabulary table index, and then encoding the integer as a two-octet signed integer. Floating-point numbers and arrays of such numbers can be supported in the same way.
This part of ISO/IEC 19776 defines several unique encoders for X3D that utilize type-aware knowledge to further compress data. Details are specified in 5 Encoding of fields.
Use of XML Signature and XML Encryption is documented within the context of examples in Annex C Examples.
This part of ISO/IEC 19776 uses Fast Infoset to deserialize and uncompress an X3D document. The tables that apply to this part of ISO/IEC 19776 are specified in Annex A Fast Infoset tables.
The encoding algorithm vocabulary table is used to reference built-in algorithms to decompress field values.
This part of ISO/IEC 19776 defines several unique encoders for X3D that utilize type-aware knowledge to further compress data. Details are specified in 5 Encoding of fields.
Type-specific field decompression is handled by the registered field encoders specified in 5 Encoding of fields.
Type-specific field decompression is handled by the registered field encoders specified in 5 Encoding of fields.
Node compressors are responsible for decompressing a node and all its children. Typically, this decompressor has domain-specific knowledge about how to decompress a node's data.
The decompressor shall restore the exact same scenegraph structure as was compressed, including any ProtoInstance usage. In addition, no reordering of data is allowed; e.g., vertex values in a Coordinate node point array shall not change ordinal position.
A decompressor shall only be assigned to handle a single node and all its children. Other constructs, like Prototypes and Routes, may not be compressed using node compressors. Furthermore, nodes that implement the X3DScriptNode or X3DProgrammableShaderObject shall not be assigned a node compressor. A node decompressor can be used to decompress the child nodes of these node types.
Node decompressors shall not be nested. A provided node decompressor shall handle all of the children nodes for that node type.
A node decompressor transforms a compressed scenegraph fragment using the following steps:
Restore the exact original scenegraph structure using the payload data and the unprocessed nodes and fields.
Create a MetadataSet named ".x3db" with a default reference of "Web3D". This value may be omitted to further reduce size.
Move user-specified metadata (if any) to become the first child of the created MetadataSet node. This step is ignored if no user-specified metadata is provided.
Add a MetadataInteger entry for the encoding named "encoding" with an integral value attribute. This shall be an encoder specified in Annex B Node compressors. It is suggested that the value for the reference field of the MetadataInteger node be a URI, pointing to further information for that encoder.
Add a MetadataInteger entry for the resulting compressed data named "payload".
Remove fields from the original scenegraph handled by the encoder. Leave all remaining scenegraph structure intact.
Example The following illustrates decompressing an IndexedFaceSet:
An IndexedFaceSet node containing a MetadataString node and a Coordinate node are to be compressed using a geometric compression algorithm (encoding 1). The input scenegraph from FastInfoSet looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value="the compressed and
encoded coordIndex and point fields as a binary payload block">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="MyEncoderMetadata">
<MetadataString name="rate"
value="12:1">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD">
</IndexedFaceSet>
The .x3db MetadataSet is removed and the user's metadata placed back in its original position. Then the final form parsed by a decoder engine of X3D files encoded in the Compressed binary encoding is:
<IndexedFaceSet coordIndex="0, 1, 2, 3, 4, 5, -1">
<MetadataString name="myData" value="myValue">
<Coordinate DEF="COORD" point="1.0 2.0 3.0, 4.0 5.0 6.0">
</IndexedFaceSet>
If the preserved keyword was used when encoding, the .x3db MetadataSet node will be retained in the decompressed stream. The user's metadata will be retained but will not be returned to its original position.
EXAMPLE The decompressed X3D form returned from Fast Infoset looks like this:
<IndexedFaceSet>
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataInteger name="payload" value="the compressed and
encoded coordIndex and point fields as a binary payload block">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="Binary Params">
<MetadataString name="lossy"
value="true">
<MetadataString name="coordinateBits"
value="15">
<MetadataString name="normalBits"
value="6">
<MetadataString name="colorBits"
value="9">
<MetadataString name="textureBits"
value="9">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
After the geometry decompressor converts the payload into X3D, the final form parsed by the X3D engine is:
<IndexedFaceSet coordIndex="...">
<MetadataSet name=".x3db" reference="Web3D">
<MetadataString name="myData" value ="myValue">
<MetadataInteger name="encoding" value="1">
<MetadataSet name="Encoder Metadata">
<MetadataSet name="Binary Params">
<MetadataString name="lossy"
value="true">
<MetadataString name="coordinateBits"
value="15">
<MetadataString name="normalBits"
value="6">
<MetadataString name="colorBits"
value="9">
<MetadataString name="textureBits"
value="9">
</MetadataSet>
</MetadataSet>
</MetadataSet>
<Coordinate DEF="COORD" point="...">
</IndexedFaceSet>
The original user metadata is retained but now resides as a child of the MetadataSet with the name ".x3db".
An optional final step is gzip compression of the entire compressed scene as specified in 2.[RFC1952]. This step supports global compression, while the other compression techniques described in this part of ISO/IEC 19776 are localized to individual chunks of data.
NOTE gzip compression of compressed binary encoded scenes may have a negative impact on parser processing performance during decompression.
Compressed binary encoded files shall use the file extension “.x3db”. The MIME-type associated with that binary compressed X3D encoded file is “model/x3d+binary”.
Compressed binary encoded files that have been “gzipped” shall use a file extension of “.x3db.gz”. The MIME type is “model/x3d+binary”. The content-encoding value is “gzip”.
The concept of MIME types is defined in 2.[RFC2077].