[X3D-Public] [x3dom-developers] initial X3D JSON conversion support using X3dToJson.xslt
Don Brutzman
brutzman at nps.edu
Mon Oct 13 12:47:56 PDT 2014
Wow, what a great "deep dive" this examination thread has been, we are uncovering many important design issues.
Still the work continues, and more is needed on syntax especially. Here we go - hopefully even further.
On 10/13/2014 2:34 AM, Behr, Johannes wrote:
> Hi,
>
> +1 for Cecile's, Sons and Yvonnes proposal.
OK, first am assuming that the design principles are what is best resonating (rather than syntax which still needs some discussion).
Here they are captured from Cecile's message, with some additions. Clarifications and additions are welcome.
https://svn.code.sf.net/p/x3d/code/www.web3d.org/x3d/stylesheets/X3dToJson.xslt
=======================================================================
4. Functional goals and use cases
Overview. JSON is a built-in JavaScript object. This works out of the box:
the parse method simply creates a valid JS object from a JSON string.
The resulting object tree however still has to be traversed to attach
meaning and behavior.
Exactly the way that an engine chooses to render the scenegraph is up to it,
the JSON data is only meant to initialize the scenegraph in the 3D engine
(and optionally export its state at a given moment if needed).
Specific design goals and use cases include:
- it's the same scripting language as the rest of the application
- it can be optimized with the rest of the application
- it can be tested with the rest of the application
- it can refer to and interact with non-3D modules (e.g. a third-party library
for formatting Date timestamps)
- no eval()
- lightweight to parse
- meaningful and low complexity because it's simpler to think with, easier to read,
easier to diff between two states, and easier to synchronize between devices
- easy to require(), to use the same package management as the rest of the
application code and assets
plus
- Easy for programmers to manipulate and adapt
- Strive for reusability, avoid hard-wiring the object syntax to one exclusive
API approach if possible (there are many "similar but different" scene graphs)
- Future ISO 19776-5 X3D JSON Encoding has to capture everything in a scene graph
defined by the X3D Abstract Specification. Commonly referred to as
"round trippable" property shared among X3D encodings.
- TBD strict or flexible typing?
=======================================================================
meanwhile, so far have only seen one example, from Cecile, but it is too ambiguous to be consistently repeatable (for me at least). detailed point comparisons follow below.
not clear from the email if you are all agreeing to the same syntax/structure, you guys probably have a backstory of common understanding that is deeper than what has been exposed here.
so please bear with me if this is all obvious already - we need to look more closely at these issues.
> This is compact and reflects the internal X3D model and easy to parse.
>
>> +1 for Cecile's proposal with explicit containers.
I'm not yet seeing enough information to converge or diverge. Please help me understand the internal X3D model and easier parsing.
Apparent ambiguities include the following, comparing Cecile's example with the syntax of current X3dToJson.xslt output.
a. how is
"_type": "Shape",
different than
"Shape": [
The "_type" doesn't add any information per se.
b. how is
translation: _data.translation || [0, 0, 0],
different than
"@translation": [ 0, -2, 0 ],
Also wondering what the || (inclusive or) operator is doing there. It does not appear in the JSON specification. This appears to be a mix of JSON Data Interchange Format (ECMA-404) and javascript source.
We need to distinguish between custom Javascript object types and X3D JSON. Sure such decoration is welcome, but that is a programmer-driven choice to add, and not part of a re-usable interoperable JSON encoding.
c. What precisely are
"appearance": {
"material": {
Either they are field names relative the parent (e.g. containerField values), or else
lower-case node names (which happens a lot when using X3DOM in non-strict HTML).
In whichever case, then how are they different than
"Appearance": [
{
"Material": {
d. still not seeing an X3DOM example of JSON.parse() and .json scene (though have asked several times).
as before, if X3DOM has an internal X3D model with corresponding parsable JSON, that will be very helpful.
is anything online, hopefully showing loading and parsing?
if not, OK. even draft-y or different-ish examples will likely aid understanding.
i think that the current HelloWorld.json example seems perfectly parsable already, and it would be good to see how much close the result is to how X3DOM works.
>> Two remarks:
>> - I think there shouldn't be a default type. Otherwise an application needs to know these default values, which again makes it harder to implement tools.
Sorry, again the point is not quite clear, specifically are you talking about X3D fields or X3D nodes?
For field data, not requiring strict differentiation of strong typing seems pretty sensible. The current stylesheet output didn't need to check much, the type-differentiation heuristics for /boolean/number/number array/string/string array/ were simple. Also since DOM represents everything as a string, then it would seem likely that we could allow type promotion/demotion whenever necessary since Javascript does a good job at that. So yes, it does look like relaxing type requirements for non-node field data looks promising.
For nodes, this relates to the containerField approach. Seldom needed, but important when a node might have multiple node children with different purposes. For example, disambiguation allows Collision node to distinguish parent-child relationships between regular MFNode children and a SFNode proxy Shape. This possibility only occurs a handful of times (out of ~250 nodes).
For protos, am guessing that mapping of prototype declarations and instances will always be a customization job by an API or renderer. Clarity supporting conversion will be the key, not a particular syntactic form.
> Missing default types help to decrease the file size and is more easier to read for most user.
yes, the programmer doesn't necessarily have to pay as much attention when debugging to understand whether it is correct.
regarding compression, worth considering but secondary. incidentally gzip-compressed JSON is almost as small as EXI-compressed XML.
regarding readability, not considered a primary strength of JSON. again worth considering but not necessarily a decisive factor.
> Maybe we could mark the with/without default-types in the header.
hmmm, not sure there is a JSON header per se? seems intentionally absent.
We could add information to the X3D/head/meta name=value pairs, but that complicates parsing.
Comments in a scene-graph fragment or even an X3D Metadata node are also possible. However for anything intended to be interoperable/reusable, consistency and repeatability is needed.
Am guessing that many use cases will also want to use scene-graph subtree fragments rather than an entire document worth of header+scene information.
>> - MFFields should be encoded as simple arrays (not array of arrays). Typical multi-value fields such as normals, keyValues would be trnalsted to TypedArrays of GLBuffers anyway and for these operations arrays of array is only degrade the performance and have very little added value
>
> +1
sounds sensible... but Leonard just posted a different use case where programmers can more easily manipulate structured arrays.
maybe we should allow either form? javascript array manipulation can likely serialize a structured array into a linear list without too much difficulty. this might also be considered a form of type promotion/demotion.
"When you come to a fork in the road, take it." - Yogi Berra 8)
> Best regards
> Johannes
>
>>
>> Just my 2 cents,
>> Kristian
>>
>>
>>> Hi everyone,
>>>
>>>
>>>> JSON is a built-in JavaScript object. This works out of the box: the parse method simply creates a valid JS object from a JSON string.
>>>> The resulting object tree however still has to be traversed to attach meaning and behavior.
>>>
>>> Exactly: the way the engine chooses to render the scenegraph is up to it, the JSON data is only meant to initialize the scenegraph in the 3D engine (and optionally export its state at a given moment if needed).
>>>
>>>
>>> Which is actually why it would feel redundant to have prototypes inside JSON if you use it in for Javascript because in real life I'd use real JS modules if I need a custom object type with properties and methods (CommonJS, AMD, ES6 modules, typescript modules, factories or whatever the author prefers) because:
>>> - it's the same scripting language as the rest of the application
>>> - it can be optimized with the rest of the application
>>> - it can be tested with the rest of the application
>>> - it can refer to and interact with non-3D modules (e.g. a thirdparty library for formatting Date timestamps)
>>> - no eval()
>>>
>>>
>>> My goals for a scenegraph format are:
>>> - lightweight to parse
>>> - meaningful and low complexity because it's simpler to think with, easier to read, easier to diff between two states, and easier to sync between devices
>>> - easy to require(), to use the same package management as the rest of the application code and assets
p.s. Cecile - have captured/adapted these thoughtful points as first-draft "Functional goals and use cases" above - thank you.
>>> However I'm fine not having feature parity with XML if it keeps it lightweight and straightforward, so this is more a VRML/X3D-inspired format, not "real X3D".
>>>
>>>
>>> Example (the "_type" is omitted in some nodes because I also have a concept of "Default type" for some fields, e.g. how most of the time the value in "Shape.appearance.material" is a Material node):
>>>
>>> [
>>> {
>>> "_type": "Shape",
>>> "appearance": {
>>> "material": {
>>> "transparency": 0.5,
>>> "diffuseColor": [0.8, 1, 0.8]
>>> }
>>> },
>>> "geometry": {
>>> "_type": 'Box',
>>> "size": [2, 1, 2]
>>> }
>>> },
>>> {
>>> "_type": "MyCustomObjectType",
>>> "translation": [0, 0, 1],
>>> "scale": [2, 2, 2]
>>> }
>>> ]
>>>
>>> And an example (without messaging or engine-specific code) of one of many ways to create something similar to Protos:
>>>
>>> var MyCustomObjectType = function(data){
>>> 'use strict';
>>> var _data = data || {};
>>> var _json = {
>>> _type: 'Transform',
>>> translation: _data.translation || [0, 0, 0],
>>> scale: _data.scale || [1, 1, 1],
>>> children: [
>>> {
>>> _type: 'Shape',
>>> appearance: {
>>> material: {
>>> transparency: 0.5,
>>> diffuseColor: [0.8, 1, 0.8]
>>> }
>>> },
>>> geometry: {
>>> _type: 'Sphere',
>>> radius: 5
>>> }
>>> }
>>> ]
>>> };
>>> Object.defineProperty(
>>> this,
>>> 'json',
>>> {
>>> configurable: false,
>>> enumerable: true,
>>> writable: false,
>>> value: function(){
>>> return _json;
>>> }
>>> }
>>> );
>>> };
>>>
>>> MyCustomObjectType.prototype.modifySomething = function(){
>>> 'use strict';
>>> //
>>> //
>>> };
>>>
>>> module.exports = MyCustomObjectType;
>>>
>>>
>>> See you,
>>> Cecile
>>>
>>>
>>> _______________________________________________
>>> X3D-Public mailing list
>>>
>>> X3D-Public at web3d.org
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the X3D-Public
mailing list