[x3d-public] second round of X3D JSON conversion support using X3dToJson.xslt

Roy Walmsley roy.walmsley at ntlworld.com
Mon Mar 9 02:46:54 PDT 2015


Don,

Great that you are making progress.

I have two questions, if I may. Apologies in advance if it simply my lack of
knowledge and understanding.

1)  Why does a container field need to have a different prefix to the other
attributes (i.e. "-" as opposed to "@")?
2) How is it with routes and prototypes, including 'IS' statements?

Roy

-----Original Message-----
From: x3d-public [mailto:x3d-public-bounces at web3d.org] On Behalf Of Don
Brutzman
Sent: 09 March 2015 08:34
To: X3D Graphics public mailing list
Subject: [x3d-public] second round of X3D JSON conversion support using
X3dToJson.xslt

I finally got around to reading Douglas Crockfords book "JavaScript: The
Good Parts" from O'Reilly and Associates.
http://shop.oreilly.com/product/9780596517748.do

It really extracts a solid core of Javascript and shows why many awful & bad
parts of the language lead to severe common misconceptions. Now having a
better understanding of the pathologies, for me it is now hard to imagine
ever programming effectively in Javascript without it.

So I took another pass at JSON encodings for X3D.  Summary of modifications:
- strict compliance with JSLint http://www.jslint.com
- achieve unique keys by keeping each nested X3D node as a unique object
- Elements, comments and collected attributes are each JSONObjects
- keep child node arrays together as JSONArray of JSONObjects via shared
containerField field name

The updated pattern now passes JSLint.

Still need to fix handling of header elements, shouldn't be hard.

Still need to compare to other patterns produced so far.  Feel's like it is
getting closer to success, catching up to multiple other exemplars and
hopefully capturing best practices.  We'll see how useful the completed
version might be.

Example excerpt and updated documentation follow.  Comments welcome.

[
     { "#comment":"Example scene to illustrate X3D nodes and fields (XML
elements and attributes)" },
     { "Group":
       {
         "@bboxCenter":[0, 0, 0 ],
         "@bboxSize":[-1, -1, -1 ],
         "-children":
           [
             { "Viewpoint":
               {
                 "@DEF":"ViewUpClose",
                 "@centerOfRotation":[0, -1, 0 ],
                 "@description":"Hello world!",
                 "@position":[0, -1, 7 ],
                 "@fieldOfView":0.7854,
                 "@jump":true,
                 "@orientation":[0, 0, 1, 0 ],
                 "@retainUserOffsets":false
               }
             },
             { "Transform":
               {
                 "@rotation":[0, 1, 0, 3 ],
                 "@center":[0, 0, 0 ],
                 "@scale":[1, 1, 1 ],
                 "@scaleOrientation":[0, 0, 1, 0 ],
                 "@translation":[0, 0, 0 ],
                 "@bboxCenter":[0, 0, 0 ],
                 "@bboxSize":[-1, -1, -1 ],
                 "-children":
                   [
                     { "Shape":
                       {
                         "@bboxCenter":[0, 0, 0 ],
                         "@bboxSize":[-1, -1, -1 ],
                         "-geometry":
                           [
                             { "Sphere":
                               {
                                 "@radius":1,
                                 "@solid":true
                               }
                             }
                           ],
                         "-appearance":
                           [
                             { "Appearance":
                               {
                                 "-material":
                                   [
                                     { "Material":
                                       {
                                         "@DEF":"MaterialLightBlue",
                                         "@diffuseColor":[0.1, 0.5, 1 ],
                                         "@ambientIntensity":0.2,
                                         "@emissiveColor":[0, 0, 0 ],
                                         "@shininess":0.2,
                                         "@specularColor":[0, 0, 0 ],
                                         "@transparency":0
                                       }
                                     }
                                   ],
                                 "-texture":
                                   [
                                     { "ImageTexture":
                                       {
                                         "@DEF":"ImageCloudlessEarth",
                                         "@url":["earth-topo.png",
"earth-topo.jpg", "earth-topo-small.gif",
"http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png",
"http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg",
"http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif" ],
                                         "@repeatS":true,
                                         "@repeatT":true
                                       }
                                     }
                                   ]
                               }
                             }
                           ]
                       }
                     }
                   ]
               }
             },
             { "Transform":
               {
                 "@translation":[0, -2, 0 ],
                 "@center":[0, 0, 0 ],
                 "@rotation":[0, 0, 1, 0 ],
                 "@scale":[1, 1, 1 ],
                 "@scaleOrientation":[0, 0, 1, 0 ],
                 "@bboxCenter":[0, 0, 0 ],
                 "@bboxSize":[-1, -1, -1 ],
                 "-children":
                   [
                     { "Shape":
                       {
                         "@bboxCenter":[0, 0, 0 ],
                         "@bboxSize":[-1, -1, -1 ],
                         "-geometry":
                           [
                             { "Text":
                               {
                                 "@DEF":"TextMessage",
                                 "@string":["Hello", "world!" ],
                                 "@maxExtent":0.0,
                                 "@solid":false,
                                 "-fontStyle":
                                   [
                                     { "FontStyle":
                                       {
                                         "@justify":["MIDDLE", "MIDDLE" ],
                                         "@family":["SERIF" ],
                                         "@horizontal":true,
                                         "@leftToRight":true,
                                         "@size":1.0,
                                         "@spacing":1.0,
                                         "@style":"PLAIN",
                                         "@topToBottom":true
                                       }
                                     }
                                   ]
                               }
                             }
                           ],
                         "-appearance":
                           [
                             { "Appearance":
                               {
                                 "-material":
                                   [
                                     { "Material":
                                       {
                                         "@USE":"MaterialLightBlue",
                                         "@ambientIntensity":0.2,
                                         "@diffuseColor":[0.8, 0.8, 0.8 ],
                                         "@emissiveColor":[0, 0, 0 ],
                                         "@shininess":0.2,
                                         "@specularColor":[0, 0, 0 ],
                                         "@transparency":0
                                       }
                                     }
                                   ]
                               }
                             }
                           ]
                       }
                     }
                   ]
               }
             }
           ]
       }
     }
]



http://www.web3d.org/x3d/content/examples/HelloWorld.x3d

https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X
3dToJson.xslt

=======================================================================
X3D JSON Design Considerations and X3dToJson.xslt Converter Status

References and resources:
http://www.web3d.org/wiki/index.php/X3D_JSON_Encoding

Status: work in progress.

1. Working features in this converter:
- XML elements (X3D nodes), XML attributes (X3D simple-type fields),
comments.
- Square and squiggly brackets, commas.
- Escaping quotation marks.
- Handling of singleton numeric values.
- Handling of MFVec arrays (simple-array form).
 
http://www.web3d.org/x3d-resources/content/examples/X3dResources.html#Exampl
es
- Embedded support in X3D-Edit found under X3D Conversions menu list.

2. TODO and regression testing:
- Refactor handling of head, component, meta, unit and Scene elements.
- Conversion applied and results available in X3D Examples Archives.
- Redo suppression of default attribute values inserted by XML parser
reading DOCTYPE
 
http://stackoverflow.com/questions/11749319/how-can-i-ignore-the-doctype-dec
laration-with-xsl
   Saxon command-line option expand:off applied (but apparently doesn't
always work)
   http://saxonica.com/documentation9.0/using-xsl/commandline.html
- Strictly differentiate typing of string/number/boolean values and arrays.
   Currently accomplished heuristically, perhaps that is sufficient. Typing
can
   allow Javascript type promotion/demotion as needed and make parsing
simpler.
- Handling special characters (UTF-16, Unicode) in strings.
- Preserve contained CDATA text (principal example: contained Script source
code).
- Round-trip testing using a JSON-to-XML converter to ensure completeness
and
   correctness.  Probably need to write this since XML-JSON encodings vary.
- Test capabilities using X3DOM parsing, rendering, serialization.
- Test capabilities using three.js, XML3D, webgl source code, and other
   javascript libraries is also important and welcome.
- Confirm proper handling and escaping of MFString array quoted values.

3. What X3D JSON specification will need to specify:
    http://www.web3d.org/specifications/X3dSpecificationRelationships.png

- Round-trip conversion capability as primary requirement to show that
features
   in the X3D Abstract Specification can all be represented in an X3D JSON
file.
- @attributeName prefix using @ character.
- "#comment" preservation of comments as an allowed option.
- "#CDATA"   preservation of character data as a requirement.
- Handling of values and arrays for numeric, boolean and string types.
- Use of "null" (if any).
- Handling of singleton numeric values, e.g.
	"transparency": 1,
   rather than
	"transparency": [1],
- Handling of nested vector values in MFVec arrays, for example
	"keyValue": [0, 0, 0, 1, 1, 1, 2, 2, 2],
         (simpler form is a better match for rendering usage)
   or else
	"keyValue": [ [0, 0, 0], [1, 1, 1], [2, 2, 2] ],
         (structured form is easier for programmers to manipulate)
   or perhaps either?
- Omitting default attribute values allowed?
- Omitting or including default containerField values?  These might be the
   basis for restructuring the JSON to insert explicit containerField values
   prior to each node definition.
- File extension (.js or .json?) and MIME type

4. Functional goals and design patterns:

Overview. JSON is a built-in JavaScript object. This works out of the box:
the parse method simply creates a valid JS object from a JSON string.
Once that is done, the resulting object tree still has to be traversed in
order to attach the renderable presentation and behavior if the scene graph.

The exact manner in which an Javascript engine chooses to interpret the JSON
and render the X3D scenegraph is an independent decision for each Javascript
library.
The JSON representation is only meant to provide the scenegraph for
appropriate initialization in the 3D engine (and optionally export its state
at a given moment if needed).

Critical reference: Douglas Crockford book "JavaScript: The Good Parts" from
O'Reilly and Associates. http://shop.oreilly.com/product/9780596517748.do

Design patterns:
- Elements, comments and collected attributes are each mapped as
JSONObjects.
- Child scene-graph nodes are mapped as JSONObject arrays with single key
containerField.
- Non-scene-graph nodes (X3D, head, component, meta, unit, Scene) are simple
JSONObjects.
- Mapping object members to field names ensures key uniqueness.
- Testable using jslint http://www.jslint.com
- Prefixes: @ for attributes, #comment for comments, - for containerField
names.

5. Specific design goals and use cases include:
- X3D JSON is written in same javascript language as the rest of the
application.
- it can be optimized with the rest of the application.
- it can be tested with the rest of the application.
- it can refer to and interact with non-3D modules (e.g. a third-party
library
   for formatting Date timestamps).
- no eval() method needs to be applied.
- lightweight to parse.
- meaningful and low complexity because it's simpler to think with, easier
to read,
   easier to diff between two states, and easier to synchronize between
devices.
- easy to require(), to use the same package management as the rest of the
   javascript application code and assets.
- Easy for programmers to manipulate and adapt.
- Strive for reusability, avoid hard-wiring the object syntax to one
exclusive
   API approach if possible (there are many "similar but different" scene
graphs).
- Future ISO 19776-5 X3D JSON Encoding has to capture everything
representable
   in a scene graph defined by the X3D Abstract Specification.  Full
expressoin
   of all scene graph information is commonly referred to as "round
trippable"
   property.  The round-trippable requirement is shared by all X3D
encodings:
   XML .x3d, ClassicVRML .x3dv, Compressed Binary .x3db.
- TBD what is use-case requirement for strict or flexible typing of data
values?

6. Examples

- HelloWorld.json and HelloWorld.scene.json
   http://www.web3d.org/x3d/content/examples/HelloWorld.x3d
   http://www.web3d.org/x3d/content/examples/HelloWorld.json
- All Web3D archive examples are automatically converted and published:
   http://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
- Need X3DOM examples with corresponding parsing of someScene.json
- Other example uses welcome.

7. Apparently not possible

- Embedded JSON comments are specifically disallowed by JSON specification
   and so round-trippable inclusion must be a custom feature of this
encoding.
   http://www.quora.com/How-do-I-write-comments-inside-a-JSON-document
- No JSON-unique header. Can optionally use X3D/head/meta name=value pairs
   (if capturing full document) or comments in a scene-graph fragment
   or even an X3D Metadata node.  However for anything intended to be
   interoperable/reusable, consistency and repeatability is needed.
=======================================================================

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman

_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org





More information about the x3d-public mailing list