[X3D-Public] [x3dom-developers] x3d to json experiments & links of interest

[Jung, Yvonne]

> Am 08.07.2014 um 17:22 schrieb "Behr, Johannes" <johannes.behr at igd.fraunhofer.de>:
> 
> Hi,
> 
>> some random comments:
>> - Agree to Cecile's comment about the special meaning of type and the common way underscoring these "meta-data"
>>> The typesystem allows to select the correct target fields for generic addChild() operations (e.g. performed by the XML-parser)
>> - Indeed. However, this is a practical approach, right? It's not speified and works because there are no ambiguities in the standard nodes.
> 
> Correct. But it's specified. Every SFNode/MFNode field definition includes the type of node which can be linked.
> e.g. the Shape Node: (http://www.web3d.org/files/specifications/19775-1/V3.2/Part01/components/shape.html#Shape)
> 
> Shape : X3DShapeNode {
>  SFNode  [in,out] appearance NULL     [X3DAppearanceNode]
>  SFNode  [in,out] geometry   NULL     [X3DGeometryNode]
>  SFNode  [in,out] metadata   NULL     [X3DMetadataObject]
>  SFVec3f []       bboxCenter 0 0 0    (-∞,∞)
>  SFVec3f []       bboxSize   -1 -1 -1 [0,∞) or −1 −1 −1
> }
> 
> The geometry-field can only link to X3DGeometryNode derived Nodes.
> A generic addChild() method (like in x3dom) just uses a simple type.isA() operation to find the right field. 
> 
>> But if I define a node like this:
>> 
>> Tree {
>>   SFNode left NULL [Tree]
>>   SFNode right NULL [Tree]
>> }
> 
> For ambiguities cases you have to use the otherwise optional "containerField" property. 

This is thinking XML...

> Not perfect but helps to simplify and hide the complexity from the user for most cases.
> 
> KISS for the most cases but open enough for exception.
> 
> But I'm personally open for both encodings. 
> 
> - Explicit/VRML like fields is closer to the actually data model. 
> - Implicit/XML like is more compact and simpler for beginner. 

No, for beginners it is simpler to get it wrong, e.g. they put an appearance under a geometry etc. VRML-style is more obvious, as the relationships are explicit. Also, parsing would more or less reduce to calling the native JSON.parse(str), the rest is init stuff.

Best, Yvonne 

> 
> Maybe something smart which supports both :)
> 
> Best regards
> Johannes
> 
>> 
>> Now I have to be explicit in the serialization
>> 
>> - I agree to Yvonne: X3D has a multi-container concept, authors need to understand it to work with the API and thus it should not be hidden in the serialization. Alternatively, one could discard the whole multi-container concept and use a single container like the DOM-API. And like XML3D ;)
>> 
>>> - Is it always Strings (as in XML) or to we use the JSON types?
>> Is there any good reason to not use JSON types?
>> 
>> Best,
>> Kristian
>> 
>> 
>> Am 08.07.2014 10:54, schrieb Behr, Johannes:
>>> Hi,
>>> 
>>>> I must admit, I prefer Kristian's approach. For one thing, it drastically simplifies parsing, and for another, it directly shows the underlying aggregation structure, which is completly missing in XML encoding but which eases learning X3D a lot.
>>> Agree. But the single-child-container has also some advantage:
>>> 
>>> - More compact content
>>> - Easier for user (don't have to know and care about field names)
>>> - Less error prone, user can not assign e.g. Geometry-types (e.g. Box) to appearance fields.
>>> - Intelligent is already well defined by spec and existing systems.
>>> 
>>> Best regards
>>> Johannes
>>> 
>>>> Cheers
>>>> Yvonne
>>>> 
>>>>> Am 08.07.2014 um 10:07 schrieb "Behr, Johannes" <johannes.behr at igd.fraunhofer.de>:
>>>>> 
>>>>> Hi,
>>>>> 
>>>>>> Hi Don,
>>>>>> 
>>>>>> just a few comments from my side (and my experience having written some X3D parsers / toolkits):
>>>>>> 
>>>>>> JSON is actually kind of similar to the VRML syntax, with some advantages (it needs less context to parse). The common feature of JSON and VRML -- despite the optical similarities -- is to define multiple named array entries that can in turn contain structured content (often called "containers"). XML encoding has only _one_ container for child elements (called "children" in the API). That is what X3D in XML always suffered from: X3D requires an additional attribute to specify the container. This concept is well hidden using default values, but requires the XSD (i.e. context). And it suddenly pops up, once a user is dealing with the most underestimated concept of X3D: Prototypes.
>>>>>> 
>>>>>> So I wonder why to (re-)import this flaw to JSON? Why not something like:
>>>>>> 
>>>>>> {
>>>>>> "type":"Group",
>>>>>> "children":[
>>>>>>    {
>>>>>>       "type":"Shape",
>>>>>>       "appearance":{
>>>>>>          "type":"Material",
>>>>>>          "diffuseColor":[0.1,0.5,1]
>>>>>>       },
>>>>>>       "geometry":{
>>>>>>          "type":"Box"
>>>>>>       }
>>>>>>    }
>>>>>> ]
>>>>>> }
>>>>> 
>>>>> The typesystem allows to select the correct target fields for generic addChild() operations (e.g. performed by the XML-parser)
>>>>> This is well defined and could be utilised in the json version again.
>>>>> 
>>>>> {
>>>>>  "type":"shape",
>>>>>  "children": [
>>>>>  {
>>>>>      "type":"shape",
>>>>>      children: [
>>>>>          {
>>>>>              "type":"Material"
>>>>>              "diffuseColor":[...]
>>>>>          },
>>>>>          {
>>>>>              "type":"Box"
>>>>>          }
>>>>>      ]
>>>>>  }
>>>>> }
>>>>> 
>>>>> Both versions have advantage and disadvantage which should be considered.
>>>>> 
>>>>> For me the big question is the field data encoding.
>>>>> 
>>>>> - Is it always Strings (as in XML) or to we use the JSON types?
>>>>> - Should we allow links to binary value container (similar to gltf?)
>>>>> - Are MF-Fields always Arrays?
>>>>> 
>>>>> Best regards
>>>>> Johannes
>>>>> 
>>>>>> This style would clearly expose the container names and thus also help users who want to use the API.
>>>>>> 
>>>>>> Apart from the encoding, I wonder how useful a JSON representation is. From our experience, the advantages in terms of size is negligible compared to XML because the numbers make the main workload. To transfer heavy geometry and texture data, there are many other, more compelling alternatives (see papers from last and next Web3D :).
>>>>>> 
>>>>>> Just my two cents :)
>>>>>> 
>>>>>> Kristian
>>>>>> 
>>>>>> 
>>>>>>> Still hoping to hear from x3dom developers on their use of JSON and relevant use cases of X3D in JSON form.
>>>>>>> 
>>>>>>>> 6.  Suggested discussion points for the X3D teleconference and future followups.
>>>>>>>> 
>>>>>>>> a.  What are the use cases for having X3D JSON available in JavaScript?  It would be good to spell them out and understand them well.
>>>>>>>> b.  Is there a good/consistent way for X3DOM to utilize such capabilities?
>>>>>>>> c.  Is there a way for Three.js or other javascript libraries to utilize such capabilities?
>>>>>>>> d.  Is there a single authoritative reference for JSON itself? and for JSON-XML conversions?
>>>>>>>> e.  Compare compression of a TestMesh.x3d.json.gz to TestMesh.x3db and TestMesh.x3d.exi
>>>>>>>>   (EXI will likely win because it includes data typing)
>>>>>>>> f.  Once a canonical form for X3D as JSON is established, add conversion capabilities to X3D-Edit and also autoconvert, test and publish JSON for all of the3800+ X3D Examples Archive scenes
>>>>>>>> g.  Decide if this capability needs to be defined in one of the X3D standards, or perhaps as an X3D best practice.
>>>>>>>> h.  Probably lots more... What else?
>>>>>>> More information on JSON <==> XML follows.
>>>>>>> 
>>>>>>> =====================================================================
>>>>>>> 
>>>>>>> Don McGregor (who has translated the DIS protocol to JSON) reports the following ECMA standard for JSON.
>>>>>>> 
>>>>>>> http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf  (14 pages)
>>>>>>> 
>>>>>>> It does not mention XML, and therefore does not address the potential use of - or @ (or whatever) characters to distinguish child JSON fields as elements or attributes. Absent any other specifications, this would seem to provide rationale for our defining an official JSON encoding for X3D.
>>>>>>> 
>>>>>>> The following commercial posting discusses this issue a little:
>>>>>>>  XSLT and XPath for JSON
>>>>>>>  By Mark Joseph - May 6, 2008
>>>>>>>  https://www.p6r.com/articles/2008/05/06/xslt-and-xpath-for-json
>>>>>>> 
>>>>>>> =====================================================================
>>>>>>> 
>>>>>>> The following reference from json.org is also interesting:
>>>>>>> 
>>>>>>>  JsonML.org
>>>>>>>  JSON Markup Language (JsonML)
>>>>>>> 
>>>>>>>> JsonML (JSON Markup Language) is an application of the JSON (JavaScript Object Notation) format. The purpose of JsonML is to provide a compact format for transporting XML-based markup as JSON which allows it to be losslessly converted back to its original form.
>>>>>>>> 
>>>>>>>> Native XML/XHTML doesn't sit well embedded in JavaScript. When XHTML is stored in script it must be properly encoded as an opaque string. JsonML allows easy manipulation of the markup in script before completely rehydrating back to the original form.
>>>>>>> This site also includes the following syntax definition for lossless round-trip conversion:
>>>>>>> 
>>>>>>>  http://www.jsonml.org/syntax
>>>>>>> 
>>>>>>>> Syntax Reasoning
>>>>>>>> 
>>>>>>>> JsonML was designed for lossless roundtrip conversion between XML and JSON and to handle "mixed-mode content" (i.e., textual data outside of or next to elements). Key considerations were lossless two-way conversion and compactness of form. JsonML handles namespaces in the same way that XML 1.0 does: attributes for xmlns and tag names with colons.
>>>>>>>> 
>>>>>>>> JsonML was never intended to be the way that everything that could be expressed in XML should be expressed in JSON. JsonML is the only transformation which can safely encode "mixed-mode" content in XML, which is a key characteristic in a markup language such as XHTML. Many other transformations exist for representing XML in a JSON format.
>>>>>>>> 
>>>>>>>> Grammar (BNF)
>>>>>>>> 
>>>>>>>> The following grammar represents how XML-based markup (e.g. XHTML) is encoded into JsonML. As per JSON, whitespace is permitted between tokens.
>>>>>>>> [...]
>>>>>>> =====================================================================
>>>>>>> 
>>>>>>> all the best, Don
>>>>>> 
>>>>>> -- 
>>>>>> _______________________________________________________________________________
>>>>>> 
>>>>>> Kristian Sons
>>>>>> Deutsches Forschungszentrum für Künstliche Intelligenz GmbH, DFKI
>>>>>> Agenten und Simulierte Realität
>>>>>> Campus, Geb. D 3 2, Raum 0.77
>>>>>> 66123 Saarbrücken, Germany
>>>>>> 
>>>>>> Phone: +49 681 85775-3833
>>>>>> Phone: +49 681 302-3833
>>>>>> Fax:   +49 681 85775–2235
>>>>>> 
>>>>>> kristian.sons at dfki.de
>>>>>> http://www.xml3d.org
>>>>>> 
>>>>>> 
>>>>>> Geschäftsführung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>>>>>> Dr. Walter Olthoff
>>>>>> 
>>>>>> Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
>>>>>> Amtsgericht Kaiserslautern, HRB 2313
>>>>>> _______________________________________________________________________________
>>>>>> 
>>>>>> _______________________________________________
>>>>>> X3D-Public mailing list
>>>>>> X3D-Public at web3d.org
>>>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>> ---
>>>>> Dr. Johannes Behr
>>>>> Head of Department: Visual Computing System Technologies
>>>>> Fraunhofer-Institut für Graphische Datenverarbeitung IGD
>>>>> Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
>>>>> Tel +49 6151 155-510  |  Fax +49 6151 155-196
>>>>> johannes.behr at igd.fraunhofer.de  |  www.igd.fraunhofer.de
>>>>> 
>>>>> 
>>>>> ------------------------------------------------------------------------------
>>>>> Open source business process management suite built on Java and Eclipse
>>>>> Turn processes into business applications with Bonita BPM Community Edition
>>>>> Quickly connect people, data, and systems into organized workflows
>>>>> Winner of BOSSIE, CODIE, OW2 and Gartner awards
>>>>> http://p.sf.net/sfu/Bonitasoft
>>>>> _______________________________________________
>>>>> x3dom-developers mailing list
>>>>> x3dom-developers at lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/x3dom-developers
>>> ---
>>> Dr. Johannes Behr
>>> Head of Department: Visual Computing System Technologies
>>> Fraunhofer-Institut für Graphische Datenverarbeitung IGD
>>> Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
>>> Tel +49 6151 155-510  |  Fax +49 6151 155-196
>>> johannes.behr at igd.fraunhofer.de |  www.igd.fraunhofer.de
>> 
>> 
>> -- 
>> _______________________________________________________________________________
>> 
>> Kristian Sons
>> Deutsches Forschungszentrum für Künstliche Intelligenz GmbH, DFKI
>> Agenten und Simulierte Realität
>> Campus, Geb. D 3 2, Raum 0.77
>> 66123 Saarbrücken, Germany
>> 
>> Phone: +49 681 85775-3833
>> Phone: +49 681 302-3833
>> Fax:   +49 681 85775–2235
>> kristian.sons at dfki.de
>> http://www.xml3d.org
>> 
>> Geschäftsführung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>> Dr. Walter Olthoff
>> 
>> Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
>> Amtsgericht Kaiserslautern, HRB 2313
>> _______________________________________________________________________________
> 
> ---
> Dr. Johannes Behr
> Head of Department: Visual Computing System Technologies
> Fraunhofer-Institut für Graphische Datenverarbeitung IGD
> Fraunhoferstr. 5  |  64283 Darmstadt  |  Germany
> Tel +49 6151 155-510  |  Fax +49 6151 155-196
> johannes.behr at igd.fraunhofer.de  |  www.igd.fraunhofer.de
>