[x3d-public] second round of X3D JSON conversion support using X3dToJson.xslt

Don Brutzman brutzman at nps.edu
Mon Mar 9 01:33:30 PDT 2015


I finally got around to reading Douglas Crockfords book "JavaScript: The Good Parts" from
O'Reilly and Associates. http://shop.oreilly.com/product/9780596517748.do

It really extracts a solid core of Javascript and shows why many awful & bad parts of the language lead to severe common misconceptions. Now having a better understanding of the pathologies, for me it is now hard to imagine ever programming effectively in Javascript without it.

So I took another pass at JSON encodings for X3D.  Summary of modifications:
- strict compliance with JSLint http://www.jslint.com
- achieve unique keys by keeping each nested X3D node as a unique object
- Elements, comments and collected attributes are each JSONObjects
- keep child node arrays together as JSONArray of JSONObjects via shared containerField field name

The updated pattern now passes JSLint.

Still need to fix handling of header elements, shouldn't be hard.

Still need to compare to other patterns produced so far.  Feel's like it is getting closer to success, catching up to multiple other exemplars and hopefully capturing best practices.  We'll see how useful the completed version might be.

Example excerpt and updated documentation follow.  Comments welcome.

[
     { "#comment":"Example scene to illustrate X3D nodes and fields (XML elements and attributes)" },
     { "Group":
       {
         "@bboxCenter":[0, 0, 0 ],
         "@bboxSize":[-1, -1, -1 ],
         "-children":
           [
             { "Viewpoint":
               {
                 "@DEF":"ViewUpClose",
                 "@centerOfRotation":[0, -1, 0 ],
                 "@description":"Hello world!",
                 "@position":[0, -1, 7 ],
                 "@fieldOfView":0.7854,
                 "@jump":true,
                 "@orientation":[0, 0, 1, 0 ],
                 "@retainUserOffsets":false
               }
             },
             { "Transform":
               {
                 "@rotation":[0, 1, 0, 3 ],
                 "@center":[0, 0, 0 ],
                 "@scale":[1, 1, 1 ],
                 "@scaleOrientation":[0, 0, 1, 0 ],
                 "@translation":[0, 0, 0 ],
                 "@bboxCenter":[0, 0, 0 ],
                 "@bboxSize":[-1, -1, -1 ],
                 "-children":
                   [
                     { "Shape":
                       {
                         "@bboxCenter":[0, 0, 0 ],
                         "@bboxSize":[-1, -1, -1 ],
                         "-geometry":
                           [
                             { "Sphere":
                               {
                                 "@radius":1,
                                 "@solid":true
                               }
                             }
                           ],
                         "-appearance":
                           [
                             { "Appearance":
                               {
                                 "-material":
                                   [
                                     { "Material":
                                       {
                                         "@DEF":"MaterialLightBlue",
                                         "@diffuseColor":[0.1, 0.5, 1 ],
                                         "@ambientIntensity":0.2,
                                         "@emissiveColor":[0, 0, 0 ],
                                         "@shininess":0.2,
                                         "@specularColor":[0, 0, 0 ],
                                         "@transparency":0
                                       }
                                     }
                                   ],
                                 "-texture":
                                   [
                                     { "ImageTexture":
                                       {
                                         "@DEF":"ImageCloudlessEarth",
                                         "@url":["earth-topo.png", "earth-topo.jpg", "earth-topo-small.gif", "http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png", "http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg", "http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif" ],
                                         "@repeatS":true,
                                         "@repeatT":true
                                       }
                                     }
                                   ]
                               }
                             }
                           ]
                       }
                     }
                   ]
               }
             },
             { "Transform":
               {
                 "@translation":[0, -2, 0 ],
                 "@center":[0, 0, 0 ],
                 "@rotation":[0, 0, 1, 0 ],
                 "@scale":[1, 1, 1 ],
                 "@scaleOrientation":[0, 0, 1, 0 ],
                 "@bboxCenter":[0, 0, 0 ],
                 "@bboxSize":[-1, -1, -1 ],
                 "-children":
                   [
                     { "Shape":
                       {
                         "@bboxCenter":[0, 0, 0 ],
                         "@bboxSize":[-1, -1, -1 ],
                         "-geometry":
                           [
                             { "Text":
                               {
                                 "@DEF":"TextMessage",
                                 "@string":["Hello", "world!" ],
                                 "@maxExtent":0.0,
                                 "@solid":false,
                                 "-fontStyle":
                                   [
                                     { "FontStyle":
                                       {
                                         "@justify":["MIDDLE", "MIDDLE" ],
                                         "@family":["SERIF" ],
                                         "@horizontal":true,
                                         "@leftToRight":true,
                                         "@size":1.0,
                                         "@spacing":1.0,
                                         "@style":"PLAIN",
                                         "@topToBottom":true
                                       }
                                     }
                                   ]
                               }
                             }
                           ],
                         "-appearance":
                           [
                             { "Appearance":
                               {
                                 "-material":
                                   [
                                     { "Material":
                                       {
                                         "@USE":"MaterialLightBlue",
                                         "@ambientIntensity":0.2,
                                         "@diffuseColor":[0.8, 0.8, 0.8 ],
                                         "@emissiveColor":[0, 0, 0 ],
                                         "@shininess":0.2,
                                         "@specularColor":[0, 0, 0 ],
                                         "@transparency":0
                                       }
                                     }
                                   ]
                               }
                             }
                           ]
                       }
                     }
                   ]
               }
             }
           ]
       }
     }
]



http://www.web3d.org/x3d/content/examples/HelloWorld.x3d

https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X3dToJson.xslt

=======================================================================
X3D JSON Design Considerations and X3dToJson.xslt Converter Status

References and resources:
http://www.web3d.org/wiki/index.php/X3D_JSON_Encoding

Status: work in progress.

1. Working features in this converter:
- XML elements (X3D nodes), XML attributes (X3D simple-type fields), comments.
- Square and squiggly brackets, commas.
- Escaping quotation marks.
- Handling of singleton numeric values.
- Handling of MFVec arrays (simple-array form).
   http://www.web3d.org/x3d-resources/content/examples/X3dResources.html#Examples
- Embedded support in X3D-Edit found under X3D Conversions menu list.

2. TODO and regression testing:
- Refactor handling of head, component, meta, unit and Scene elements.
- Conversion applied and results available in X3D Examples Archives.
- Redo suppression of default attribute values inserted by XML parser reading DOCTYPE
   http://stackoverflow.com/questions/11749319/how-can-i-ignore-the-doctype-declaration-with-xsl
   Saxon command-line option expand:off applied (but apparently doesn't always work)
   http://saxonica.com/documentation9.0/using-xsl/commandline.html
- Strictly differentiate typing of string/number/boolean values and arrays.
   Currently accomplished heuristically, perhaps that is sufficient. Typing can
   allow Javascript type promotion/demotion as needed and make parsing simpler.
- Handling special characters (UTF-16, Unicode) in strings.
- Preserve contained CDATA text (principal example: contained Script source code).
- Round-trip testing using a JSON-to-XML converter to ensure completeness and
   correctness.  Probably need to write this since XML-JSON encodings vary.
- Test capabilities using X3DOM parsing, rendering, serialization.
- Test capabilities using three.js, XML3D, webgl source code, and other
   javascript libraries is also important and welcome.
- Confirm proper handling and escaping of MFString array quoted values.

3. What X3D JSON specification will need to specify:
    http://www.web3d.org/specifications/X3dSpecificationRelationships.png

- Round-trip conversion capability as primary requirement to show that features
   in the X3D Abstract Specification can all be represented in an X3D JSON file.
- @attributeName prefix using @ character.
- "#comment" preservation of comments as an allowed option.
- "#CDATA"   preservation of character data as a requirement.
- Handling of values and arrays for numeric, boolean and string types.
- Use of "null" (if any).
- Handling of singleton numeric values, e.g.
	"transparency": 1,
   rather than
	"transparency": [1],
- Handling of nested vector values in MFVec arrays, for example
	"keyValue": [0, 0, 0, 1, 1, 1, 2, 2, 2],
         (simpler form is a better match for rendering usage)
   or else
	"keyValue": [ [0, 0, 0], [1, 1, 1], [2, 2, 2] ],
         (structured form is easier for programmers to manipulate)
   or perhaps either?
- Omitting default attribute values allowed?
- Omitting or including default containerField values?  These might be the
   basis for restructuring the JSON to insert explicit containerField values
   prior to each node definition.
- File extension (.js or .json?) and MIME type

4. Functional goals and design patterns:

Overview. JSON is a built-in JavaScript object. This works out of the box:
the parse method simply creates a valid JS object from a JSON string.
Once that is done, the resulting object tree still has to be traversed
in order to attach the renderable presentation and behavior if the scene graph.

The exact manner in which an Javascript engine chooses to interpret the JSON and
render the X3D scenegraph is an independent decision for each Javascript library.
The JSON representation is only meant to provide the scenegraph for appropriate
initialization in the 3D engine (and optionally export its state at a given
moment if needed).

Critical reference: Douglas Crockford book "JavaScript: The Good Parts" from
O'Reilly and Associates. http://shop.oreilly.com/product/9780596517748.do

Design patterns:
- Elements, comments and collected attributes are each mapped as JSONObjects.
- Child scene-graph nodes are mapped as JSONObject arrays with single key containerField.
- Non-scene-graph nodes (X3D, head, component, meta, unit, Scene) are simple JSONObjects.
- Mapping object members to field names ensures key uniqueness.
- Testable using jslint http://www.jslint.com
- Prefixes: @ for attributes, #comment for comments, - for containerField names.

5. Specific design goals and use cases include:
- X3D JSON is written in same javascript language as the rest of the application.
- it can be optimized with the rest of the application.
- it can be tested with the rest of the application.
- it can refer to and interact with non-3D modules (e.g. a third-party library
   for formatting Date timestamps).
- no eval() method needs to be applied.
- lightweight to parse.
- meaningful and low complexity because it's simpler to think with, easier to read,
   easier to diff between two states, and easier to synchronize between devices.
- easy to require(), to use the same package management as the rest of the
   javascript application code and assets.
- Easy for programmers to manipulate and adapt.
- Strive for reusability, avoid hard-wiring the object syntax to one exclusive
   API approach if possible (there are many "similar but different" scene graphs).
- Future ISO 19776-5 X3D JSON Encoding has to capture everything representable
   in a scene graph defined by the X3D Abstract Specification.  Full expressoin
   of all scene graph information is commonly referred to as "round trippable"
   property.  The round-trippable requirement is shared by all X3D encodings:
   XML .x3d, ClassicVRML .x3dv, Compressed Binary .x3db.
- TBD what is use-case requirement for strict or flexible typing of data values?

6. Examples

- HelloWorld.json and HelloWorld.scene.json
   http://www.web3d.org/x3d/content/examples/HelloWorld.x3d
   http://www.web3d.org/x3d/content/examples/HelloWorld.json
- All Web3D archive examples are automatically converted and published:
   http://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
- Need X3DOM examples with corresponding parsing of someScene.json
- Other example uses welcome.

7. Apparently not possible

- Embedded JSON comments are specifically disallowed by JSON specification
   and so round-trippable inclusion must be a custom feature of this encoding.
   http://www.quora.com/How-do-I-write-comments-inside-a-JSON-document
- No JSON-unique header. Can optionally use X3D/head/meta name=value pairs
   (if capturing full document) or comments in a scene-graph fragment
   or even an X3D Metadata node.  However for anything intended to be
   interoperable/reusable, consistency and repeatability is needed.
=======================================================================

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman



More information about the x3d-public mailing list