[x3d-public] second round of X3D JSON conversion support using X3dToJson.xslt <-- read this, previous mail messed up.

[Herbert Stocker]

Please don't read the previous mail, it got headings messed up, argh.
This is the right, complete version:



Hi Don and all,

before jumping in on this, i've read all the discussions preceding this.
Took me about 3 hours. One can do this only on spare time.
But i was happy to see how people bring good arguments, factual and
respectful (of course).
And i would have argued into the same directions, too.

Though you say yourself that your current iteration of the encoding 
needs investigation,
> Still haven't compared it to interesting design alternatives 
> previously posted by Cecile, Yvonne, Kristian, John and others.  
> Continued exploration should help a "best of breed" emerge, the 
> problem space continues to shrink. 
i would like to mention a few things that come to mind.
Maybe just to come up with the same what you would.
*
Can we omit the prefixes '@' and '-'?*
i understand that there are some conventions in XSLT, XPATH, etc., but
do we need to*keep them**in JSON?**
* The big point of JSON is that you can pass a JSON string to JSON.parse()
and receive an *object tree of JavaScript objects*. With key value pairs.
In JavaScript the '@' or '-' characters *don't feel native* as key names.
To rephrase the above question, why have them there?

Moreover, they make the keys *not form valid variable names*, so one can't
use this syntax:
myMaterial.transparency= .5;
but must use *more noisy syntax*:
myMaterial*['@*transparency*']*= .5;
This is what Cécile has pointed out already today (now yesterday).

i do understand that some convertors need to have an indication whether
something is a field (attribute) or a child node, but is it really so?
if so, we can omit at least one of the two. I'd vote for the '@' as it 
appears more often.
*

Can we ***reduce nesting level?**
i feel there are more nesting levels than would be necessary.
Is there a limit in the JSON spec? (hope not).
Consider the meta tag in the header:

"head":[
          { "meta":
            {
              "@content":"HelloWorld.x3d",
              "@name":"title"
            }
          },
          { "meta":

There is a '[', and two '{'s to get from the 'head' field to the 
'@content' field.
That's *3 nesting levels*.
With Kristians / Céciles approach, there are only two braces to cross, a 
square
one and a curly one:

"head":[
          { "_type":   "meta",
            "content": "HelloWorld.x3d",
            "name":    "title"

          },
          { "_type": "meta",


*Why **{**"Transform": **{**fields-of-the-Transform **}****}**?*
That's now hard to explain.
The inner pair of curly braces (green ones) feels natural to me.
It envelopes the name-value pairs describing the Transform node in question.
That's what curly braces are for in JSON.

But the outer curlies (red ones) look to me like a workaround for 
something.
Like the workaround for the fact that the node names are not unique, but key
names must be unique.
This forms a JavaScript object - curly braces denote objects in JSON - 
that consists
of a single field, which is the node type and has the actual node 
description as the
value.
It is an *additional nesting level*, that does not exist in the data 
model of X3D.
i think that should be avoided.

Normally when you have things that have no unique name but their order 
is significant,
you put them into an array.
That array already exists in your encoding. It is the '[' and ']' 
surrounding the Transform
node. The only thing that should be eliminated is the red curlies.

Again, the "_type" approach of Cécile and Kristian solves this.

{"_type": "Transform", fields-of-the-Transform }


*C***an we remove square braces from **SF**Node fields?****
i see this in the Shape node:

"-geometry":[  {  "Sphere":{  fiels-of-the-Shape}  }  ]

with the transparency field, we agreed on having no '[' and ']'for *SF* 
fields.
Similarly, there is no need for them in *SF*Node fields.
i think the blue squares should be removed here.


"*@version" in the header should stay a string.*
Version strings look like numbers, but sometimes one wants to add 
letters. E.g. for indicating
beta versions or release candidates. Don't know how this applies to a 
spec though.
Besides that, the number 3.3, for example, is not exactly representable 
as a binary floating point
number, i guess. It may be represented internally as 3.2999999999999987 
or such.


These are my suggestions. Would be interesting if that JavaScript book 
you mentioned has
reasons against them.


best regards,

/Herbert Stocker/



On 09.03.2015 16:28, Don Brutzman wrote:
> On 3/9/2015 2:46 AM, Roy Walmsley wrote:
>> Don,
>>
>> Great that you are making progress.
>
> Thanks Roy.  It is definitely challenging but appears to be do-able.
>
> Douglas Crockford's "JavaScript: The Good Parts" is a fundamentally 
> important reference.
>
>> I have two questions, if I may. Apologies in advance if it simply my 
>> lack of
>> knowledge and understanding.
>
> All help is much appreciated, questions are really important. Thanks.
>
> btw if you find anyone with full knowledge and understanding, please 
> invite them over immediately!  8)
>
>> 1)  Why does a container field need to have a different prefix to the 
>> other
>> attributes (i.e. "-" as opposed to "@")?
>
> The @ symbol is commonly used in XML XPath, XSLT etc. to indicate the 
> name of an attribute.  Didn't want to overload that semantic usage 
> further.
>
> I applied the - symbol to prefix containerField labels for node arrays 
> to see how it looks and to possibly simplify processing. Without that 
> distinguishing symbol, a parser trying to reconstruct an X3D scene 
> graph from JSON might need deeper knowledge of how X3D works.  So it 
> is experimental.
>
> Organizing child arrays by using the field names of X3D nodes is an 
> asset we have that allows each node to be handled as a uniquely keyed 
> object - the "O" in JSON.  (This approach is not possible in other XML 
> to JSON patterns, where generally XML child elements don't have such a 
> unique key or label.)
>
> I suspect that the - symbol prefix might be able to go away, or be 
> further improved, when we get to using these a bit more.  Not clear yet.
>
> Meanwhile (after briefly sleeping on it) I've added the rest of what 
> is needed for converting the X3D header.  Non scene-graph nodes (X3D, 
> head, component, meta, unit, Scene) are simple JSON objects that are 
> now included.  Note that they have no containerField and in some cases 
> no attributes at all, so the design pattern here is simpler and a 
> little bit different than the rest of the scene graph.
>
> Completely converted HelloWorld.json attached.  Looks like a pair of 
> slightly different design patterns (first for header and then for 
> scene graph) are required and holding together OK so far.  Still 
> passes jslint in its strictest mode. http://www.jslint.com
>
> Incidentally the need for a slight hybrid approach like this design 
> may be part of why a single approach has been a bit elusive so far.  
> Still haven't compared it to interesting design alternatives 
> previously posted by Cecile, Yvonne, Kristian, John and others.  
> Continued exploration should help a "best of breed" emerge, the 
> problem space continues to shrink.
>
> Conversion stylesheet updates committed to
> https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X3dToJson.xslt 
>
>
> Extra: added some design comments on whether a special JSON header is 
> needed (probably not) or DOCTYPE information is needed (not).
> =======================================================================
> 7. Apparently not possible or not needed.
>
> - Embedded JSON comments are specifically disallowed by JSON 
> specification
>   and so round-trippable inclusion must be a custom feature of this 
> encoding.
> http://www.quora.com/How-do-I-write-comments-inside-a-JSON-document
> - No JSON-unique header. Can optionally use X3D/head/meta name=value 
> pairs
>   (if capturing full document) or comments in a scene-graph fragment
>   or even an X3D Metadata node.  However for anything intended to be
>   interoperable/reusable, consistency and repeatability is needed.
>   Leaving the output JSON as simple as possible avoids code problems and
>   provides programmers with the greatest possible flexibility. Further
>   analysis will show if a JSON header is needed, probably a non-problem.
> - DOCTYPE information is dropped.  This is commonplace for XSLT and does
>   not appear to be needed in the JSON encoding.  If actually needed by 
> someone,
>   such information can be reconstructed simply by using the X3D 
> version number.
> http://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#Validation 
>
> =======================================================================
>
>> 2) How is it with routes and prototypes, including 'IS' statements?
>>
>> Roy
>
> Great question, that is the next level. Script, field, CDATA source 
> blocks, ProtoDeclare, IS/connect, ExternProtoDeclare and fieldValue 
> might require some special handling in the stylesheet, though I 
> suspect that these patterns will be able to handle them just fine.  
> I'll need another session (and more strong coffee!) to confirm that 
> satisfactorily.
>
> Given that this hybrid approach is now different from other 
> XML-to-JSON converters, we'll probably also need to write a Javascript 
> program to parse and then write the JSON back out as XML .x3d in order 
> to confirm the lossless round-trip requirement.  JsonToX3d.js I suppose.
>
> If nothing else, all of this design variety illustrates the value of 
> finding an acceptable JSON encoding that can be reused consistently.
>
> all the best, Don
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20150310/f903dc2a/attachment.html>