[x3d-public] Refinement of X3D JSON Merged with: FW: [x3dom-users] simple X3D -> JSON -> X3DOM, D3.js
John Carlson
yottzumm at gmail.com
Wed Jul 24 08:36:15 PDT 2019
Found previous email from Douglas Crockford, included below.
I will proceed with this, as long as you want to include something like:
“tag”: “meta”
or
“tag”: “unit”
or
“tag”: “component”
as object field in each JSON object made a child of head. This was similar to my original JSON proposed encoding and others, but using $ for tag like JSON5 for all objects.
Otherwise, I see no way forward except to modify most JSON (JavaScript) string parsers (or write our own exclusive one).
Please try parsing and generating your proposed JSON in a Java JSON parser/generator. Roundtripping JSON is important too!
An example will help!
Thanks!
John
Sent from Mail for Windows 10
From: Douglas Crockford
Sent: Monday, February 23, 2015 3:24 PM
To: Don Brutzman; John Carlson
Subject: Re: [x3dom-users] simple X3D -> JSON -> X3DOM, D3.js
I recommend failing on duplicate keys.
On 2015-02-23 1:21 PM, Don Brutzman wrote:
> Thanks John.
>
> Our goal with trying to find a good X3D encoding using JSON is to
> support programmers who might want to load scene data into a
> javascript program. Our objective is to represent the scene graph
> information, no more no less, just like the XML and ClassicVRML and
> Compressed Binary programs. How that information is utilized and
> adapted by the programmer (or library loader) is up to them.
>
> Many XML documents have duplicates of elements and attributes, so
> trying to create unique keys for each repeated element and attribute
> doesn't seem meaningful or round-trippable.
>
> Mailing list discussion seems to be the best way to proceed, there are
> a lot of expert programmers there.
>
>
> On 2/20/2015 6:08 PM, John Carlson wrote:
>> I found this. It has been discussed at length.
>> https://esdiscuss.org/topic/json-duplicate-keys
>>
>> On Fri, Feb 20, 2015 at 7:18 PM, John Carlson <yottzumm at gmail.com
>> <mailto:yottzumm at gmail.com>> wrote:
>>
>> And whee, node.js reports nothing. So basically Don, if you
>> leave the duplicated JSON object name in, you will be tossing all
>> your JavaScript programmers which use standard utilities in node.js
>> into a black hole. Here's the final code for node. I will test in a
>> browser next:
>>
>> "use strict";
>>
>>
>> var content = '';
>>
>> process.stdin.resume();
>>
>> process.stdin.on('data', function(buf) { content +=
>> buf.toString(); });
>>
>> process.stdin.on('end', function() {
>>
>> try {
>>
>> console.log(JSON.stringify(JSON.parse(content), null, " "));
>>
>> } catch (e) {
>>
>> console.log(e);
>>
>> }
>>
>> });
>>
>>
>> On Fri, Feb 20, 2015 at 7:05 PM, John Carlson <yottzumm at gmail.com
>> <mailto:yottzumm at gmail.com>> wrote:
>>
>> Okay, I figured it out. JavaScript silently ignores
>> duplicate keys unless use strict is used.
>>
>> "use strict";
>>
>> try {
>>
>> var json = {"name":"n","name":"v"};
>>
>> console.log(json);
>>
>> } catch (e) {
>>
>> console.log(e);
>>
>> }
>>
>>
>> On Fri, Feb 20, 2015 at 6:23 PM, John Carlson
>> <yottzumm at gmail.com <mailto:yottzumm at gmail.com>> wrote:
>>
>> Here is a very simple test case. I am posting to
>> stackoverflow. I hope I don't get voted down.
>>
>> try {
>>
>> var json = '{"name":"n","name":"v"}';
>>
>> console.log(json);
>>
>> var jsonout = JSON.stringify(JSON.parse(json));
>>
>> console.log(jsonout);
>>
>> } catch (e) {
>>
>> console.log(e);
>>
>> }
>>
>>
>> On Fri, Feb 20, 2015 at 6:22 PM, John Carlson
>> <yottzumm at gmail.com <mailto:yottzumm at gmail.com>> wrote:
>>
>> I just found out that JSON.parse dies silently.
>> Here's better code, that still succeeds and removes a Transform. I
>> think this is either a problem with the standard, or a problem with
>> JSON.stringify or JSON.parse in node.js, or my code, but we better
>> fix one of them. Anyone know the authors of node.js and could tap
>> into them for not complying with the standard? I'm not saying my code
>> is perfect. Should I go to stackoverflow and ask them to fix my code?
>>
>> // Java Style Sheet Language, implementation 2
>>
>>
>> var content = '';
>>
>> // read content into buffer
>>
>> process.stdin.resume();
>>
>> process.stdin.on('data', function(buf) { content +=
>> buf.toString(); });
>>
>>
>> process.stdin.on('end', function() {
>>
>> try {
>>
>> console.log(JSON.stringify(JSON.parse(content), null,
>> " "));
>>
>> } catch (e) {
>>
>> console.log(e);
>>
>> }
>>
>> });
>>
>>
>> Here's an example of running my code with a very
>> simple example:
>>
>> $ cat simple.json
>>
>> {
>>
>> "name": "n",
>>
>> "name": "v"
>>
>> }
>>
>> $ node ValidateJSON.js < simple.json
>>
>> {
>>
>> "name": "v"
>>
>> }
>>
>>
>> On Fri, Feb 20, 2015 at 6:03 PM, John Carlson
>> <yottzumm at gmail.com <mailto:yottzumm at gmail.com>> wrote:
>>
>> Okay, here's the results! Douglas, I am
>> including you, because JSON.parse or JSON.stringify in node.js
>> silently eliminate a whole lot of "JSON".
>> Or it could be my code. Let's figure out which
>> it is.
>>
>> //////////////////////////////////
>>
>>
>> // ValidateJSON.js, which should be runnable in
>> any node.js (please point out any bugs):
>>
>> var content = '';
>>
>> // read content into buffer
>>
>> process.stdin.resume();
>>
>> process.stdin.on('data', function(buf) { content
>> += buf.toString(); });
>>
>>
>> process.stdin.on('end', function() {
>>
>> console.log(JSON.stringify(JSON.parse(content), null, " "));
>>
>> });
>>
>> //////////////////////////////////////////
>>
>>
>> // INPUT
>>
>> http://coderextreme.net/x3d.json
>>
>>
>> // OUTPUT
>>
>> http://coderextreme.net/roundtrip.json
>>
>> ///////////////////////////////////////////
>>
>> // Running results
>>
>> $ node ValidateJSON.js < x3d.json > roundtrip.json
>>
>>
>> $ grep Transform x3d.json
>>
>> "Transform": [
>>
>> "Transform": [
>>
>> $ grep Transform roundtrip.json
>>
>> "Transform": [
>>
>> Where's the missing Transform? Did my code eat
>> it? Did JSON.parse eat it? Did JSON.stringify eat it? did
>> console.log eat it?
>>
>>
>> On Fri, Feb 20, 2015 at 4:33 PM, Don Brutzman
>> <brutzman at nps.edu <mailto:brutzman at nps.edu>> wrote:
>>
>> As before, your insights are much
>> appreciated. Thanks John.
>>
>> It is looking like there may be some
>> terminology mismatches? That might be an important cause of
>> misunderstanding. Hoping to get clearer.
>>
>> When talking about "name" values, please
>> clarify. Are your referring to element and attribute names, or
>> DEF/USE ID/IDREF names?
>>
>> In general, we have been trying to create a
>> JSON representation for X3D that is round-trippable with any other
>> encoding.
>>
>> Not a goal is to create something incorrect
>> or unusable. This isn't a marketing activity. So no need for a
>> BogusJSON or whatever, there is simply no need for something broken.
>>
>> The X3dToJson.xslt is trying to take an
>> XML-based .x3d scene and produce a usable JSON representation. If
>> information is lost, and the result can't be written back out as the
>> original scene, then an encoding fails the round-trip test.
>>
>> Multiple approaches exist for converting XML
>> to JSON. Some were explored and compared on the earlier email
>> threads. First one that pops up on today's search is
>>
>> http://www.utilities-online.__info/xmltojson/
>> <http://www.utilities-online.info/xmltojson/>
>>
>> When given the HelloWorld.x3d scene, it
>> replies with
>>
>> {
>> "X3D": {
>> "-profile": "Immersive",
>> "-version": "3.3",
>> "-xmlns:xsd":
>> "http://www.w3.org/2001/__XMLSchema-instance
>> <http://www.w3.org/2001/XMLSchema-instance>",
>> "-xsd:__noNamespaceSchemaLocation": "
>> http://www.web3d.org/__specifications/x3d-3.3.xsd
>> <http://www.web3d.org/specifications/x3d-3.3.xsd> ",
>> "head": {
>> "meta": [
>> {
>> "-name": "title",
>> "-content": "HelloWorld.x3d"
>> },
>> {
>> "-name": "description",
>> "-content": "Simple X3D scene
>> example: Hello World!"
>> },
>> {
>> "-name": "created",
>> "-content": "30 October 2000"
>> },
>> {
>> "-name": "modified",
>> "-content": "23 November 2014"
>> },
>> {
>> "-name": "creator",
>> "-content": "Don Brutzman"
>> },
>> {
>> "-name": "Image",
>> "-content": "HelloWorld.tall.png"
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://en.wikipedia.org/wiki/__Hello_world
>> <http://en.wikipedia.org/wiki/Hello_world> "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> en.wikipedia.org/wiki/Hello#\
>> <http://en.wikipedia.org/wiki/Hello#%5C>"__Hello,_World\"_computer___program
>> "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://en.wikibooks.org/w/__index.php?title=Computer___Programming/Hello_world
>> <http://en.wikibooks.org/w/index.php?title=Computer_Programming/Hello_world>
>> "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://www.HelloWorldExample.__net <http://www.HelloWorldExample.net> "
>> },
>> {
>> "-name": "reference",
>> "-content": " http://www.web3D.org "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://www.web3d.org/realtime-__3d/news/internationalization-__x3d
>> <http://www.web3d.org/realtime-3d/news/internationalization-x3d> "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://www.web3d.org/x3d/__content/examples/HelloWorld.__x3d
>> <http://www.web3d.org/x3d/content/examples/HelloWorld.x3d> "
>> },
>> {
>> "-name": "reference",
>> "-content": "
>> http://X3dGraphics.com/__examples/__X3dForAdvancedModeling/__HelloWorldScenes
>> <http://X3dGraphics.com/examples/X3dForAdvancedModeling/HelloWorldScenes>
>> "
>> },
>> {
>> "-name": "identifier",
>> "-content": "
>> http://X3dGraphics.com/__examples/X3dForWebAuthors/__Chapter01-TechnicalOverview/__HelloWorld.x3d
>> <http://X3dGraphics.com/examples/X3dForWebAuthors/Chapter01-TechnicalOverview/HelloWorld.x3d>
>> "
>> },
>> {
>> "-name": "license",
>> "-content": "
>> http://www.web3d.org/x3d/__content/examples/license.html
>> <http://www.web3d.org/x3d/content/examples/license.html> "
>> },
>> {
>> "-name": "generator",
>> "-content": "X3D-Edit 3.3,
>> https://savage.nps.edu/X3D-__Edit <https://savage.nps.edu/X3D-Edit>"
>> },
>> {
>> "-name": "reference",
>> "-content": "HelloWorld.wrl"
>> },
>> {
>> "-name": "reference",
>> "-content": "HelloWorld.x3dv"
>> },
>> {
>> "-name": "reference",
>> "-content": "HelloWorld.x3db"
>> },
>> {
>> "-name": "reference",
>> "-content": "HelloWorld.xhtml"
>> },
>> {
>> "-name": "reference",
>> "-content": "HelloWorld.json"
>> }
>> ]
>> },
>> "Scene": {
>> "Group": {
>> "Viewpoint": {
>> "-DEF": "ViewUpClose",
>> "-centerOfRotation": "0 -1 0",
>> "-description": "Hello world!",
>> "-position": "0 -1 7"
>> },
>> "Transform": [
>> {
>> "-rotation": "0 1 0 3",
>> "Shape": {
>> "Appearance": {
>> "Material": {
>> "-DEF": "MaterialLightBlue",
>> "-diffuseColor": "0.1 0.5 1"
>> },
>> "ImageTexture": {
>> "-DEF":
>> "ImageCloudlessEarth",
>> "-url": "
>> \"earth-topo.png\" \"earth-topo.jpg\" \"earth-topo-small.gif\"
>> \"http://www.web3d.org/x3d/__content/examples/Basic/earth-__topo.png\
>> <http://www.web3d.org/x3d/content/examples/Basic/earth-topo.png%5C>"
>> \"http://www.web3d.org/x3d/__content/examples/Basic/earth-__topo.jpg\
>> <http://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg%5C>"
>> \"http://www.web3d.org/x3d/__content/examples/Basic/earth-__topo-small.gif\
>> <http://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif%5C>"
>> "
>> }
>> }
>> }
>> },
>> {
>> "-translation": "0 -2 0",
>> "Shape": {
>> "Text": {
>> "-DEF": "TextMessage",
>> "-string": "\"Hello\"
>> \"world!\"",
>> "FontStyle": { "-justify":
>> "\"MIDDLE\" \"MIDDLE\"" }
>> },
>> "Appearance": {
>> "Material": { "-USE":
>> "MaterialLightBlue" }
>> }
>> }
>> }
>> ]
>> }
>> }
>> }
>> }
>>
>> This varies from the current HelloWorld.json
>> by including X3D/head/meta* tags, using a different attribute prefix
>> (- vice @), and dropping comments and inclusion/exclusion of default
>> attribute values.
>> http://www.web3d.org/x3d/__content/examples/HelloWorld.__json
>> <http://www.web3d.org/x3d/content/examples/HelloWorld.json>
>>
>> So is the XML -> JSON conversion from
>> http://www.utilities-online.__info/xmltojson
>> <http://www.utilities-online.info/xmltojson> producing illegal JSON too?
>>
>> Looking forward to continued scrutiny and
>> improved understanding.
>>
>>
>> On 2/20/2015 11:15 AM, John Carlson wrote:
>>
>>
>> On Feb 19, 2015 12:21 AM,
>> "John Carlson" <yottzumm at gmail.com <mailto:yottzumm at gmail.com>
>> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>>
>> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>
>> <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>>>> wrote:
>>
>> How does the X3D JSON
>> encoding handle two Shapes or Transforms or Groups of the same type
>> at the same depth in the tree with the same parent? Do you modify the
>> tree structure significantly? I'm not that familiar with the
>> schema. I just know you're going to have to do something else in
>> JavaScript, even if your JSON parses and works. You can't use the
>> JSON directly as Javascript. JSON != JavaScript. We desire a
>> *JavaScript* and JSON encoding.
>>
>>
>>
>> This is a crucial point and I'm
>> really glad you are raising it.
>>
>>
>> You're right it's a crucial point. It's
>> so crucial I brought it up with Douglas Crockford. He said the
>> standard did not need to be changed. What's important is how the
>> standard is *used*. Say I'm in Java, and I'm reading in a A JSON
>> object. What happens if I see a name in and object twice? What do I
>> do? Do I instantly change my attribute to a list? How? Must I use a
>> list for every single attribute because that's what is allowed by
>> JSON? No, is the answer. JSON doesn't specify a set of name/list
>> pairs, it specifies name/value pairs. JSON objects are meant to be
>> associative arrays or struct or record as is mentioned in the
>> standard. I asked Douglas to change the standard so names appeared
>> only once in an object. He demurred. The thing is, it's the *use*
>> that matters, and you're going to pull a lot of programmer's hair out
>> and stick their fingers in electrical sockets if you continue to use
>> names more than once in a JSON object. It
>> violates the spirit of the
>> standard if not the letter.
>>
>> If we think about use cases for
>> Javascript applications loading X3D models, I think they are most
>> interested in reusing geometry for their own nefarious/magnificent
>> purposes. Javascript programmers are likely less interested in X3D
>> event models, Scripts, protos and whatnot. Library loaders might
>> convert such things to match their paradigm, or might also ignore
>> non-geometry nodes completely.
>>
>>
>> I guess what matters ultimately is that
>> JSON parsers must reject names appearing in objects more that once.
>> Also, please do not call your encoding JSON because it
>> just *isn't*. If we must, programmers
>> will move away from your JSON encoding, and use their own, as I have
>> already done. All I'm actually doing is to asking you to make an X3D
>> JSON standard which makes good sense, and if not, don't call it
>> JSON. FJSON or Fake JSON might be more appropriate.
>>
>>
>> Rephrase, Let JSON be JSON (a subset
>> that is good for data objects) and Javascript be Javascript (a
>> full-blown programming language).
>>
>> Not trying to downplay or avoid
>> anything, just looking at what the most common uses of an X3D scene
>> encoded as JSON might be.
>>
>> Such an idea actually brings us full
>> circle to the design premise of all the scene encodings (XML .x3d,
>> Classic VRML .x3dv, binary .x3db) which is to completely/compatibly
>> represent an X3D scene, period.
>>
>> If we achieve that with JSON, making
>> common geometry constructs easy for programmers to access, then great
>> they will do so. If we also try to tell them "For Advanced Graphics
>> Using X3D JSON You Must Program Your Javascript Application This Way"
>> they will laugh.
>>
>> Viva la difference, programming
>> styles and practices will always vary and evolve. Whatever works.
>>
>>
>> Yes, whatever works. Names or field
>> slots appearing in an object more than once in a struct or record
>> don't work in programmers' minds. You will force every value to be a
>> list in their minds. Which may be what you intend. If so, then
>> explicitly do that in your JSON output (my restructuring in my
>> previous message). Don't force the programmer into cognitive
>> overload. Please only have names appearing once in your objects, I
>> beg you desperately, if you want your X3D JSON standard to be
>> relevant. If not, then call it X3DON not *JS*ON
>>
>>
>>
>> all the best, Don
>> --
>> Don Brutzman Naval Postgraduate School, Code
>> USW/Br brutzman at nps.edu <mailto:brutzman at nps.edu>
>> Watkins 270, MOVES Institute, Monterey CA
>> 93943-5000 USA +1.831.656.2149 <tel:%2B1.831.656.2149>
>> X3D graphics, virtual worlds, navy robotics
>> http://faculty.nps.edu/__brutzman <http://faculty.nps.edu/brutzman>
>>
>>
>>
>>
>>
>>
>>
>
>
> all the best, Don
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190724/e5418626/attachment-0001.html>
More information about the x3d-public
mailing list