[x3d-public] X3D JSON encoding: refactoring, simplifying handling of #comment and #ROUTE
Leonard Daly
Leonard.Daly at realism.com
Mon Jan 4 07:46:46 PST 2016
On 1/4/2016 5:06 AM, John Carlson wrote:
> If all else fails, we can think about adding an index to the comment that shows where it is. Or #precomment and #postcomment.
I have not been following this thread as closely as I probably should.
I think what is being discussed is the location of comments in the JSON
encoding compared their original location in the XML encoding. If that
is not the case, then ignore the rest of this message and let me know.
In all translators/parsers dealing with hash (aka objects in
JavaScript), I have always seen the disclaimer that the order of the
original data cannot be guaranteed to be preserved in the data structure
after translating/parsing. For example (using PHP and Perl because they
are not XML or JavaScript):
PHP (PHP calls all collections of elements arrays, whether they are
indedxed or not)
$hash = array('z' => 1, 'x' => 'c');
foreach ($hash as $key) {
print "hash[$key] = $hash[$key]\n";
}
hash[x] = c
hash[z] = 1
on conversion to Perl and using the code before for display
Perl
# Create data the same way as PHP
my $hash = {'z' => 1, 'x' => 'c'};
foreach (my $key keys(%hash)) {
print "hash{$key} = $hash{$key}\n";
}
hash{z} = 1
hash{x} = c
If the order of items is important, than an indexed array MUST be used.
This does go back to a point I raised earlier. By creating data elements
in JSON for the XML comments, you are changing the data model stored in
the file. Things that were not data nor parsed (XML comments) are now
appearing as data. It also means that full-escaping and character
translating to the allowed JSON character set must be done to ensure
that the result can be parsed in JavaScript. You are trying to create
something that does not exist in the language (comments). I feel that
this is not the right way to go.
Leonard Daly
>
> John
>
>
>> On Jan 4, 2016, at 12:26 AM, Don Brutzman <brutzman at nps.edu> wrote:
>>
>> OK this sounds great - thanks John. We will look at your attachments further during the morning call. Will later proceed with trying to achieve the refactoring since you are ready to track along - that tandem progress is really valuable to make sure we are getting things right (and not digging a gigantic hole). BCNU.
>>
>> On 1/3/2016 4:28 PM, John Carlson wrote:
>>> First example, Viewpoints are not the same. Otherwise roundtrip is fine. Second example doesn’t parse—it’s a fragment. HelloWorld JSON converted to HelloWorldCommented.x3d2 and attached for your review. No changed to X3D JSON Loader code yet—It appears like some comments are moved upward. I believe this is due to the XML -> JSON conversion, not the JSON -> XML conversion. Node.js script to convert JSON to X3D also attached. I don’t see a problem with the new format as long as we can work out the XML -> JSON conversion. I can’t magically figure out how to reattach comments in the old place when it’s not in the JSON document. I can make any necessary changes to the loader as long as I don’t run out of spare time. John
>>>
>>>
>>>> On Dec 19, 2015, at 4:09 PM, Don Brutzman <brutzman at nps.edu> wrote:
>>>>
>>>> 1. Am thinking about how we might improve the X3D JSON encoding by treating #comment and #ROUTE as objects in an array that is contained by parent. This would reduce problems with the partial resorting that currently occurs.
>>>>
>>>> Doing it as part of "-children" isn't particularly difficult; the trick will be doing it cleanly inside of nodes that contain other non-children nodes, e.g. Shape, Appearance, singleton nodes, etc. The path to success may lead through always treating any child content as part of an array.
>>>>
>>>> Continuing this line of inquiry: we could put them inside of a "-children" JSON field consistently throughout the scene graph. Use cases:
>>>> a. Within "-children":[array] of Scene (which is already produced)
>>>> b. Within "-children":[array] of grouping nodes (which is already produced)
>>>> c. Within "-children":[array] of any node, even when they don't have a /children/ field
>>>>
>>>> 2. So let's look at an example, in detail.
>>>>
>>>> Example scene created is named HelloWorldCommented.x3d with XML source, current (original) JSON encoding, and alternative JSON encoding all attached.
>>>>
>>>> First excerpt follows, with the focus just for regular children (use cases 1.a and 1.b from above).
>>>>
>>>> =========================================
>>>> *XML encoding*
>>>> =========================================
>>>> <X3D>
>>>> <Scene>
>>>> <!-- Example scene to illustrate comments interspersed among X3D nodes and fields (XML elements and attributes) -->
>>>> <!-- WorldInfo begin -->
>>>> <WorldInfo title='Hello world!'/>
>>>> <!-- WorldInfo complete, Group begin -->
>>>> <Group>
>>>> <!-- Viewpoint begin -->
>>>> <Viewpoint DEF='ViewUpClose' centerOfRotation='0 -1 0' description='Hello world!' position='0 -1 7'/>
>>>> <!-- Viewpoint complete, Transform begin -->
>>>> <Transform/>
>>>> <!-- Transform complete, Transform begin -->
>>>> <Transform/>
>>>> <!-- Transform complete -->
>>>> </Group>
>>>> <!-- Group complete -->
>>>> </Scene>
>>>> </X3D>
>>>> =========================================
>>>>
>>>> Now two corresponding JSON examples using #comment attached. (Net effect with #ROUTE would be the same.)
>>>>
>>>> Notice how comments are pulled up together in the current encoding (which partially re-sorts and thus changes the original order), but then comments are interspersed as expected in the second encoding:
>>>> =========================================
>>>> *current JSON encoding*
>>>> =========================================
>>>> { "X3D": {
>>>> "Scene": {
>>>> "#comment":[
>>>> "Example scene to illustrate comments interspersed among X3D nodes and fields (XML elements and attributes)",
>>>> "WorldInfo begin",
>>>> "WorldInfo complete, Group begin",
>>>> "Group complete"
>>>> ],
>>>> "-children":[
>>>> { "WorldInfo":
>>>> {
>>>> "@title":"Hello world!"
>>>> }
>>>> },
>>>> { "Group":
>>>> {
>>>> "#comment":[
>>>> "Viewpoint begin",
>>>> "Viewpoint complete, Transform begin",
>>>> "Transform complete, Transform begin",
>>>> "Transform complete"
>>>> ],
>>>> "-children":[
>>>> { "Viewpoint": { }
>>>> },
>>>> { "Transform": { }
>>>> },
>>>> { "Transform": { }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> }
>>>> =========================================
>>>> *alternative JSON encoding*
>>>> =========================================
>>>> { "X3D": {
>>>> "Scene": {
>>>> "-children":[
>>>> { "#comment":"Example scene to illustrate comments interspersed among X3D nodes and fields (XML elements and attributes)"},
>>>> { "#comment":"WorldInfo begin"},
>>>> { "WorldInfo":
>>>> {
>>>> "@title":"Hello world!"
>>>> }
>>>> },
>>>> { "#comment":"WorldInfo complete, Group begin"},
>>>> { "Group":
>>>> {
>>>> "-children":[
>>>> { "#comment":"Viewpoint begin"},
>>>> { "Viewpoint": { }
>>>> },
>>>> { "#comment":"Viewpoint complete, Transform begin"},
>>>> { "Transform": { }
>>>> },
>>>> { "#comment":"Transform complete, Transform begin"},
>>>> { "Transform": { }
>>>> },
>>>> { "#comment":"Transform complete"}
>>>> ]
>>>> }
>>>> },
>>>> { "#comment":"Group complete"}
>>>> ]
>>>> }
>>>> }
>>>> }
>>>> =========================================
>>>>
>>>> 2. So far so good. Now let's look at how this approach ripples through a Shape node, as an example of use case 1.c above.
>>>>
>>>> =========================================
>>>> *XML encoding*
>>>> =========================================
>>>> <!-- Shape begin -->
>>>> <Shape>
>>>> <!-- Text begin -->
>>>> <Text DEF='TextMessage' string='"Hello" "world!"'>
>>>> <!-- FontStyle begin -->
>>>> <FontStyle justify='"MIDDLE" "MIDDLE"'/>
>>>> <!-- FontStyle complete -->
>>>> </Text>
>>>> <!-- Text complete, Appearance begin -->
>>>> <Appearance>
>>>> <!-- Material begin -->
>>>> <Material USE='MaterialLightBlue'/>
>>>> <!-- Material complete -->
>>>> </Appearance>
>>>> <!-- Material complete -->
>>>> </Shape>
>>>> <!-- Shape complete -->
>>>> =========================================
>>>>
>>>> Once again the current/original X3D JSON encoding partially re-sorts the comments:
>>>> =========================================
>>>> *current JSON encoding*
>>>> =========================================
>>>> "#comment":[
>>>> "Shape begin",
>>>> "Shape complete"
>>>> ],
>>>> "-children":[
>>>> { "Shape":
>>>> {
>>>> "#comment":[
>>>> "Text begin",
>>>> "Text complete, Appearance begin",
>>>> "Material complete"
>>>> ],
>>>> "-geometry":[
>>>> { "Text":
>>>> {
>>>> "@DEF":"TextMessage",
>>>> "@string":["Hello","world!"],
>>>> "#comment":[
>>>> "FontStyle begin",
>>>> "FontStyle complete"
>>>> ],
>>>> "-fontStyle":[
>>>> { "FontStyle":
>>>> {
>>>> "@justify":["MIDDLE","MIDDLE"]
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> ],
>>>> "-appearance":[
>>>> { "Appearance":
>>>> {
>>>> "#comment":[
>>>> "Material begin",
>>>> "Material complete"
>>>> ],
>>>> "-material":[
>>>> { "Material":
>>>> {
>>>> "@USE":"MaterialLightBlue"
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> ]
>>>> =========================================
>>>>
>>>> And once again the alternative intersperses the comments as part of a "-children" JSON key. Not so very different.
>>>>
>>>> Notable is that Shape and Appearance do not have /children/ fields, but they do have child ("-children") comments. Thus scene graph structure is better (though not perfectly) preserved.
>>>>
>>>> Some re-sorting of comments prior to peer nodes still appears to be unavoidable, but it is reduced overall.
>>>>
>>>> The pattern for encoding comments (and ROUTEs) is fully consistent, so parsing should still be straightforward.
>>>> =========================================
>>>> *alternative JSON encoding*
>>>> =========================================
>>>> "-children":[
>>>> { "#comment":"Shape begin"},
>>>> { "Shape":
>>>> {
>>>> "#comment":[
>>>> "Sphere begin",
>>>> "Sphere complete, Appearance begin",
>>>> "Appearance complete"
>>>> ],
>>>> "-geometry":[
>>>> { "Sphere":
>>>> {
>>>> }
>>>> }
>>>> ],
>>>> "-appearance":[
>>>> { "Appearance":
>>>> {
>>>> "-children":[
>>>> { "#comment":"Viewpoint begin"},
>>>> { "#comment":"Material begin"},
>>>> { "#comment":"Material complete, ImageTexture begin"},
>>>> { "#comment":"ImageTexture complete"}
>>>> ],
>>>> "-material":[
>>>> { "Material":
>>>> {
>>>> "@DEF":"MaterialLightBlue",
>>>> "@diffuseColor":[0.1,0.5,1]
>>>> }
>>>> }
>>>> ],
>>>> "-texture":[
>>>> { "ImageTexture":
>>>> {
>>>> "@DEF":"ImageCloudlessEarth",
>>>> "@url":["earth-topo.png","earth-topo.jpg","earth-topo-small.gif","http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png","http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg","http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif"]
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> }
>>>> ]
>>>> }
>>>> },
>>>> { "#comment":"Shape complete"}
>>>> ]
>>>> =========================================
>>>>
>>>> 3. Analysis and next steps.
>>>>
>>>> I think that this alternative approach further reduces the special-case handling of #comment and #ROUTE which exists in the current/original JSON encoding.
>>>>
>>>> A more consistent JSON encoding simplifies parsing rules and hopefully makes production/consumption easier to accomplish.
>>>>
>>>> Treating #comment and #ROUTE in the same fashion as other nodes, rather than as field keys, also is more consistent with the primary design pattern of the overall X3D JSON encoding.
>>>>
>>>> The alternative-encoding example was manually edited and validated using jslint in X3D-Edit. If group review indicates that this approach looks superior, I'll work on a refactoring of the X3dToJson.xslt stylesheet and re-build all the examples to support continued in-depth testing, implementation and evaluation.
>>>>
>>>> Thanks in advance for all scrutiny and review.
>>>>
>>>> all the best, Don
>>>> --
>>>> Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
>>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
>>>> X3D graphics, virtual worlds, navy roboticshttp://faculty.nps.edu/brutzman
>>>> <HelloWorldCommented.x3d><HelloWorldCommentedOriginalEncoding.json><HelloWorldCommentedAlternativeEncoding.json>
>>
>> all the best, Don
>> --
>> Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
--
*Leonard Daly*
3D Systems & Cloud Consultant
X3D Co-Chair on Sabbatical
LA ACM SIGGRAPH Chair
President, Daly Realism - /Creating the Future/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20160104/ba4ce36e/attachment-0001.html>
More information about the x3d-public
mailing list