[x3d-public] Fwd: Good parsers for lousy JSON implementations.
John Carlson
yottzumm at gmail.com
Sun Nov 15 21:01:55 PST 2015
> Begin forwarded message:
>
> From: John Carlson <yottzumm at gmail.com>
> Date: February 21, 2015 at 3:19:04 AM CST
> To: x3dom-developer mlist <x3dom-developers at lists.sourceforge.net>, X3D Graphics public mailing list <x3d-public at web3d.org>
> Subject: Re: Good parsers for lousy JSON implementations.
>
> If you want to see if your JSON object will fly with standard JavaScript tools,
> Add
>
> "use strict";
>
> var json =
>
> to the front of your JSON file.
>
> I hope this is the last message I will send on this topic. You might not even need "var json =".
>
> On Feb 21, 2015 2:21 AM, wrote:
> Here's a reference to good JSON parsers in JavaScript if you find some of your nodes missing in X3D JSON. Thank you stackoverflow! http://stackoverflow.com/questions/28641443/node-js-json-parse-reviver-doesnt-report-duplicate-keys <http://stackoverflow.com/questions/28641443/node-js-json-parse-reviver-doesnt-report-duplicate-keys>
> On Feb 21, 2015 1:58 AM, "John Carlson" <yottzumm at gmail.com <mailto:yottzumm at gmail.com>> wrote:
> I just realized I removed the replace function from the previously posted JSON2D3.js. It's intention is to quote newlines in text nodes. Please don't include text elements as values in an array in your tests until I get a chance to fix it.
>
> On Feb 21, 2015 1:48 AM, "John Carlson" <yottzumm at gmail.com <mailto:yottzumm at gmail.com>> wrote:
> I believe that this misunderstanding stems from VRML. If it's not painfully apparent that JavaScript will drop the first transform from something like
>
> { "Transform": ....,
> "Transform":....}, in JSON, I present you with the results of running Don's HelloWorld.json (I copied and pasted to code into a separate file called x3d.json) through the standard JSON parser in node.js. As you can see from the ms sage included, the first transform was dropped totally. Do we have a JSON loader yet? If we have one written in JavaScript, I would like to see it. When this dropping behavior was discusses on esdiscuss, it was reported that the last Transform should be kept. Which is exactly what we are seeing in JavaScript. There's no easy way I can see to detect this difference programmatically at least in JavaScript because JavaScript will just keep the last node. Is there a JSONdiff (something with a multimap implementation) that will show differences, or must we do everything by hand? I hear Douglas Crockford has a pretty good parser, perhaps I should try that, and watch the warnings fly by...
>
> I have a JS loader which essentially implements a DOM loader in X3DOM. I have dumped x3dom_geoComponent.html to my JSON I translate to JS, then include the result in the page. I would like to write a JSON loader, but my poor brain isn't up to the task. Someone smart could probably take my JS translator and implement a JavaScriot JSON loader for my JSON or another JSON. You can find it here:
> https://github.com/coderextreme/X3DJSONTOD3/blob/master/JSON2D3.js <https://github.com/coderextreme/X3DJSONTOD3/blob/master/JSON2D3.js>
> ---------- Forwarded message ----------
> From: "John Carlson" <yottzumm at gmail.com <mailto:yottzumm at gmail.com>>
> Date: Feb 20, 2015 6:03 PM
> Subject: Re: [x3dom-users] simple X3D -> JSON -> X3DOM, D3.js
> To: "Don Brutzman" <brutzman at nps.edu <mailto:brutzman at nps.edu>>, "Douglas Crockford" <douglas at crockford.com <mailto:douglas at crockford.com>>
> Cc:
>
> Okay, here's the results! Douglas, I am including you, because JSON.parse or JSON.stringify in node.js silently eliminate a whole lot of "JSON".
> Or it could be my code. Let's figure out which it is.
> //////////////////////////////////
>
>
> // ValidateJSON.js, which should be runnable in any node.js (please point out any bugs):
>
> var content = '';
>
> // read content into buffer
>
> process.stdin.resume();
>
> process.stdin.on('data', function(buf) { content += buf.toString(); });
>
>
>
> process.stdin.on('end', function() {
>
> console.log(JSON.stringify(JSON.parse(content), null, " "));
>
>
> });
>
> //////////////////////////////////////////
>
>
>
> // INPUT
>
> http://coderextreme.net/x3d.json <http://coderextreme.net/x3d.json>
>
> // OUTPUT
>
> http://coderextreme.net/roundtrip.json <http://coderextreme.net/roundtrip.json>
> ///////////////////////////////////////////
>
>
> // Running results
>
> $ node ValidateJSON.js < x3d.json > roundtrip.json
>
>
>
> $ grep Transform x3d.json
>
> "Transform": [
>
> "Transform": [
>
> $ grep Transform roundtrip.json
>
>
> "Transform": [
>
> Where's the missing Transform? Did my code eat it? Did JSON.parse eat it? Did JSON.stringify eat it? did console.log eat it?
>
>
> On Fri, Feb 20, 2015 at 4:33 PM, Don Brutzman <brutzman at nps.edu <mailto:brutzman at nps.edu>> wrote:
> As before, your insights are much appreciated. Thanks John.
>
> It is looking like there may be some terminology mismatches? That might be an important cause of misunderstanding. Hoping to get clearer.
>
> When talking about "name" values, please clarify. Are your referring to element and attribute names, or DEF/USE ID/IDREF names?
>
> In general, we have been trying to create a JSON representation for X3D that is round-trippable with any other encoding.
>
> Not a goal is to create something incorrect or unusable. This isn't a marketing activity. So no need for a BogusJSON or whatever, there is simply no need for something broken.
>
> The X3dToJson.xslt is trying to take an XML-based .x3d scene and produce a usable JSON representation. If information is lost, and the result can't be written back out as the original scene, then an encoding fails the round-trip test.
>
> Multiple approaches exist for converting XML to JSON. Some were explored and compared on the earlier email threads. First one that pops up on today's search is
>
> http://www.utilities-online.info/xmltojson/ <http://www.utilities-online.info/xmltojson/>
>
> When given the HelloWorld.x3d scene, it replies with
>
> {
> "X3D": {
> "-profile": "Immersive",
> "-version": "3.3",
> "-xmlns:xsd": "http://www.w3.org/2001/XMLSchema-instance <http://www.w3.org/2001/XMLSchema-instance>",
> "-xsd:noNamespaceSchemaLocation": " http://www.web3d.org/specifications/x3d-3.3.xsd <http://www.web3d.org/specifications/x3d-3.3.xsd> ",
> "head": {
> "meta": [
> {
> "-name": "title",
> "-content": "HelloWorld.x3d"
> },
> {
> "-name": "description",
> "-content": "Simple X3D scene example: Hello World!"
> },
> {
> "-name": "created",
> "-content": "30 October 2000"
> },
> {
> "-name": "modified",
> "-content": "23 November 2014"
> },
> {
> "-name": "creator",
> "-content": "Don Brutzman"
> },
> {
> "-name": "Image",
> "-content": "HelloWorld.tall.png"
> },
> {
> "-name": "reference",
> "-content": " http://en.wikipedia.org/wiki/Hello_world <http://en.wikipedia.org/wiki/Hello_world> "
> },
> {
> "-name": "reference",
> "-content": " en.wikipedia.org/wiki/Hello#\ <http://en.wikipedia.org/wiki/Hello#%5C>"Hello,_World\"_computer_program "
> },
> {
> "-name": "reference",
> "-content": " http://en.wikibooks.org/w/index.php?title=Computer_Programming/Hello_world <http://en.wikibooks.org/w/index.php?title=Computer_Programming/Hello_world> "
> },
> {
> "-name": "reference",
> "-content": " http://www.HelloWorldExample.net <http://www.helloworldexample.net/> "
> },
> {
> "-name": "reference",
> "-content": " http://www.web3D.org <http://www.web3d.org/> "
> },
> {
> "-name": "reference",
> "-content": " http://www.web3d.org/realtime-3d/news/internationalization-x3d <http://www.web3d.org/realtime-3d/news/internationalization-x3d> "
> },
> {
> "-name": "reference",
> "-content": " http://www.web3d.org/x3d/content/examples/HelloWorld.x3d <http://www.web3d.org/x3d/content/examples/HelloWorld.x3d> "
> },
> {
> "-name": "reference",
> "-content": " http://X3dGraphics.com/examples/X3dForAdvancedModeling/HelloWorldScenes <http://x3dgraphics.com/examples/X3dForAdvancedModeling/HelloWorldScenes> "
> },
> {
> "-name": "identifier",
> "-content": " http://X3dGraphics.com/examples/X3dForWebAuthors/Chapter01-TechnicalOverview/HelloWorld.x3d <http://x3dgraphics.com/examples/X3dForWebAuthors/Chapter01-TechnicalOverview/HelloWorld.x3d> "
> },
> {
> "-name": "license",
> "-content": " http://www.web3d.org/x3d/content/examples/license.html <http://www.web3d.org/x3d/content/examples/license.html> "
> },
> {
> "-name": "generator",
> "-content": "X3D-Edit 3.3, https://savage.nps.edu/X3D-Edit <https://savage.nps.edu/X3D-Edit>"
> },
> {
> "-name": "reference",
> "-content": "HelloWorld.wrl"
> },
> {
> "-name": "reference",
> "-content": "HelloWorld.x3dv"
> },
> {
> "-name": "reference",
> "-content": "HelloWorld.x3db"
> },
> {
> "-name": "reference",
> "-content": "HelloWorld.xhtml"
> },
> {
> "-name": "reference",
> "-content": "HelloWorld.json"
> }
> ]
> },
> "Scene": {
> "Group": {
> "Viewpoint": {
> "-DEF": "ViewUpClose",
> "-centerOfRotation": "0 -1 0",
> "-description": "Hello world!",
> "-position": "0 -1 7"
> },
> "Transform": [
> {
> "-rotation": "0 1 0 3",
> "Shape": {
> "Appearance": {
> "Material": {
> "-DEF": "MaterialLightBlue",
> "-diffuseColor": "0.1 0.5 1"
> },
> "ImageTexture": {
> "-DEF": "ImageCloudlessEarth",
> "-url": " \"earth-topo.png\" \"earth-topo.jpg\" \"earth-topo-small.gif\" \"http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png\ <http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png%5C>" \"http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg\ <http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg%5C>" \"http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif\ <http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif%5C>" "
> }
> }
> }
> },
> {
> "-translation": "0 -2 0",
> "Shape": {
> "Text": {
> "-DEF": "TextMessage",
> "-string": "\"Hello\" \"world!\"",
> "FontStyle": { "-justify": "\"MIDDLE\" \"MIDDLE\"" }
> },
> "Appearance": {
> "Material": { "-USE": "MaterialLightBlue" }
> }
> }
> }
> ]
> }
> }
> }
> }
>
> This varies from the current HelloWorld.json by including X3D/head/meta* tags, using a different attribute prefix (- vice @), and dropping comments and inclusion/exclusion of default attribute values.
> http://www.web3d.org/x3d/content/examples/HelloWorld.json <http://www.web3d.org/x3d/content/examples/HelloWorld.json>
>
> So is the XML -> JSON conversion from http://www.utilities-online.info/xmltojson <http://www.utilities-online.info/xmltojson> producing illegal JSON too?
>
> Looking forward to continued scrutiny and improved understanding.
>
>
> On 2/20/2015 11:15 AM, John Carlson wrote:
>
>
> On Feb 19, 2015 12:21 AM, "John Carlson" <yottzumm at gmail.com <mailto:yottzumm at gmail.com> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>>>> wrote:
>
> How does the X3D JSON encoding handle two Shapes or Transforms or Groups of the same type at the same depth in the tree with the same parent? Do you modify the tree structure significantly? I'm not that familiar with the schema. I just know you're going to have to do something else in JavaScript, even if your JSON parses and works. You can't use the JSON directly as Javascript. JSON != JavaScript. We desire a *JavaScript* and JSON encoding.
>
>
> This is a crucial point and I'm really glad you are raising it.
>
> You're right it's a crucial point. It's so crucial I brought it up with Douglas Crockford. He said the standard did not need to be changed. What's important is how the standard is *used*. Say I'm in Java, and I'm reading in a A JSON object. What happens if I see a name in and object twice? What do I do? Do I instantly change my attribute to a list? How? Must I use a list for every single attribute because that's what is allowed by JSON? No, is the answer. JSON doesn't specify a set of name/list pairs, it specifies name/value pairs. JSON objects are meant to be associative arrays or struct or record as is mentioned in the standard. I asked Douglas to change the standard so names appeared only once in an object. He demurred. The thing is, it's the *use* that matters, and you're going to pull a lot of programmer's hair out and stick their fingers in electrical sockets if you continue to use names more than once in a JSON object. It violates the spirit of the
> standard if not the letter.
>
> If we think about use cases for Javascript applications loading X3D models, I think they are most interested in reusing geometry for their own nefarious/magnificent purposes. Javascript programmers are likely less interested in X3D event models, Scripts, protos and whatnot. Library loaders might convert such things to match their paradigm, or might also ignore non-geometry nodes completely.
>
> I guess what matters ultimately is that JSON parsers must reject names appearing in objects more that once. Also, please do not call your encoding JSON because it
> just *isn't*. If we must, programmers will move away from your JSON encoding, and use their own, as I have already done. All I'm actually doing is to asking you to make an X3D JSON standard which makes good sense, and if not, don't call it JSON. FJSON or Fake JSON might be more appropriate.
>
> Rephrase, Let JSON be JSON (a subset that is good for data objects) and Javascript be Javascript (a full-blown programming language).
>
> Not trying to downplay or avoid anything, just looking at what the most common uses of an X3D scene encoded as JSON might be.
>
> Such an idea actually brings us full circle to the design premise of all the scene encodings (XML .x3d, Classic VRML .x3dv, binary .x3db) which is to completely/compatibly represent an X3D scene, period.
>
> If we achieve that with JSON, making common geometry constructs easy for programmers to access, then great they will do so. If we also try to tell them "For Advanced Graphics Using X3D JSON You Must Program Your Javascript Application This Way" they will laugh.
>
> Viva la difference, programming styles and practices will always vary and evolve. Whatever works.
>
> Yes, whatever works. Names or field slots appearing in an object more than once in a struct or record don't work in programmers' minds. You will force every value to be a list in their minds. Which may be what you intend. If so, then explicitly do that in your JSON output (my restructuring in my previous message). Don't force the programmer into cognitive overload. Please only have names appearing once in your objects, I beg you desperately, if you want your X3D JSON standard to be relevant. If not, then call it X3DON not *JS*ON
>
>
>
> all the best, Don
> --
> Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu <mailto:brutzman at nps.edu>
> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 <tel:%2B1.831.656.2149>
> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman <http://faculty.nps.edu/brutzman>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20151115/cbb9c4d2/attachment-0001.html>
More information about the x3d-public
mailing list