[x3d-public] renewed development of X3D JSON support

Mon Oct 11 18:26:56 PDT 2021

My first task will be downloading the 2020-12 JSON schema, and reviewing 
it against draft07 JSON schema, to see differences. Then, I will pursue 
X3D JSON 4.0 schema generation using python (again).   We had decided 
before to use draft07, because no tools were available to pursue 2020-12 
or 2019-09 JSON schema in Java. Now there is one tool to do validation 
(see below). One reason I liked the draft07 approach is that the Python 
JSON validator I use, fastjsonschema, does not support 2012-02, and I 
already have draft07 working with multiple versions of X3D JSON schema 
in the same program (note that JSON schema has been converted to python):

```

import sys
import json

# the below packages are available in X3DJSONLD, they are X3D JSON 
schemas converted to python:

import schemaparser30
import schemaparser31
import schemaparser32
import schemaparser33
import schemaparser40

jsobj = json.loads(sys.stdin.read())
version = jsobj["X3D"]["@version"].replace(".", "")
try:
     if version in [ "30", "31", "32", "33", "40" ]:
         eval("schemaparser"+version+".validate(jsobj)")
except:
         print('Invalid')

```

At sometime, I think independent X3D JSON schema generation would be 
appropriate.  I did state that converting X3D XML Schema to X3D JSON 
schema with XMLSpy was probably infeasible due to field and comment 
prefixes in X3D JSON--You will have to provide hooks in XMLSpy to 
achieve this, or ask for more options to XMLSpy (which is possible) or 
XML schema.  I suggest running a test to see if the X3D JSON Schema 
produced will validate the 4000 examples generated by X3dToJson.xslt. 
The Web3D Consortium may send my sample X3D JSON schemas to Altova to 
meet requirements of X3D JSON Schema, with the requirement that their 
schema generation product will meet the Web3D Consortium's IP 
requirements, that is, we want everyone to be able to use the schema 
used, whether they have XMLSpy or not.

What I'm kind of confused about is whether we'll provide JSON validation 
with X3D*SAIL or /JSON schema/+/validation tools/ or both. I think 
having separate implementations would be a good plan. I am fairly 
confident that Don committed to supporting X3D JSON import into 
X3DJSAIL, but not using X3DJSONLD.java.  In particular, he suggested 
translating JSON to DOM document was not the preferred approach, but he 
was open to doing X3D JSON import with Saxon. I didn't ask him how he 
was going to get Saxon output into X3DJSAIL--would you like to comment, 
Don?  I know I haven't fully elucidated my thought processes around 
X3DJSONLD.java.   Why did I choose to convert to DOM?  Probably because 
I didn't see myself writing or generating X3D*SAIL for ECMAScript, I 
already tried to write X3DPSAIL and es6x3d, I merely don't seem to have 
the brain power or time.  I need to support my old tools, or transfer to 
one that has wider demand.   There are other issues like competing with 
X3DOM and X_ITE--I chose to play nice.  I didn't want to support 2 
different SAI solutions, when X3DOM and X_ITE were already far down that 
path.

Here's the code in X3D4 JSON schema to support multiple versions:

         "@version": {
           "pattern": "^(\\s|\\S)*$",
           "enum": [
             "3.0",
             "3.1",
             "3.2",
             "3.3",
             "4.0"
           ],
           "default": "4.0",
           "$comment": "SFString inputOutput",
           "type": "string"
         }
       },

So I am fairly confident that the new 4.0 schema will be able to support 
older versions.  I would ask about XvlShell though. I think we may have 
some old files that have that node.

I also note that one can change the @version property (inputOutput).  
This seems counter-intuitive.

John

On 10/11/21 6:48 PM, Brutzman, Donald (Don) (CIV) wrote:
>
> Attendees: John Carlson, Don Brutzman
>
>  1. /JSON Schema Status/
>
> - *JSON Schema* is a vocabulary that allows you to *annotate* and 
> *validate* JSON documents.
>
> - https://json-schema.org
>
> Looks like they have an updated numbering scheme. Latest published 
> IETF Draft is 2020-12.  IETF requires that each draft gets updated 
> each 6 months, but they have long missed meeting such goals.  We have 
> to settle on what has meaningful tool support.
>
>   * https://json-schema.org/specification-links.html#understanding-draft-names-and-numbers
>
> Many implementations are listed at
>
>   * https://json-schema.org/iplementations.html
>
> Looking at versions of Java implementations:
>
>   * We previously used Everit but that looks to be stopped 2 years ago
>   * Several implements support version 2019-09, only one reports
>     2020-12 support
>   * Snow https://github.com/ssilverman/snowy-json
>   * Vert.x json schema
>     https://github.com/eclipse-vertx/vertx-json-schema
>     <https://github.com/eclipse-vertx/vertx-json-schema> (looks recent)
>   * Jsonschmafriend https://github.com/jimblackler/jsonschemafriend
>     (looks recent, supports 2020-12)
>   * Gson https://en.wikipedia.org/wiki/Gson
>
> Very interesting test page, can feed it a JSON schema and JSON file 
> for validation:
>
>   * https://tryjsonschematypes.appspot.com
>
> Interestingly the JSON Schema Store indicates 419 schemas are 
> available, of various versions reaching back to draft-04..
>
> Gson (aka Google Gson) is an open-source 
> <https://en.wikipedia.org/wiki/Open-source_software> Java 
> <https://en.wikipedia.org/wiki/Java_(programming_language)> library to 
> serialize <https://en.wikipedia.org/wiki/Serialize> and deserialize 
> Java objects to (and from) JSON <https://en.wikipedia.org/wiki/JSON>. 
> Conceivably it might be integrated with X3DJSAIL objects for X3D nodes 
> and statements.  Supports many versions.
>
> *Conclusion #1: given multiple tool support, we will pursue JSON 
> Schema 2020-12 for X3D JSON, and further test across multiple 
> programming languages.*
>
> **
>
> Reaction: thank goodness we waited a couple of years rather than dig a 
> hole with prior versions of tools that are now apparently defunct.  
> Latest version of JSON schema looks pretty stable and mature now, 
> though we are certainly waiting for a stable standard (many years in 
> development, still not done).  Good enough to give it a shot again.
>
> We are pretty adept at converting X3DUOM into other things.  Once we 
> have a good pattern for X3D JSON Schema matching 2020-12, we can 
> easily (1 week’s effort) autogenerate it.
>
> NPS will soon get an updated version of XML Spy.  They appear to 
> support 2020-12. Interesting they have a new feature, apparently 
> matching what we want to do.  Worth checking:
>
>   * Convert XML Schema to/from JSON Schema
>   * https://www.altova.com/manual/XMLSpy/spyenterprise/xsjson_schview_version.html
>   * https://www.altova.com/manual/XMLSpy/spyenterprise/urefconvert_xmlschematojsonschema.html
>
> We will look at this in the next meeting.
>
> ---
>
>  2. /John’s implementations/
>
> Existing X3D JSON Schema drafts by John are found at
>
> - https://github.com/coderextreme/X3DJSONLD/tree/master/src/main/schema
>
> John’s current autogenerated X3D JSON schema is draft-07, he has prior 
> work that is 2019-09.
>
> Looking at versions of JavaScript implementations:
>
>   * Ajv appears to support latest 2020-12 and 2019-09
>   * John will upgrade his use of this library to latest
>
> Suggestion: let’s work on a X3D4 JSON Schema that also validates X3D 
> versions 3.0-3.3.  (This might be the case already.)
>
> *Plan: we will work on updating JSON Schema, then validating examples 
> first with JavaScript and Java.*
>
> ---
>
>  3. /X3D JSON pages/
>
> Initial design-planning page:
>
>           * X3D JSON Encoding
>           * https://www.web3d.org/wiki/index.php/X3D_JSON_Encoding
>
> Detailed design and implementation:
>
>           * X3D to JSON Stylesheet Converter
>           * https://www.web3d.org/x3d/stylesheets/X3dToJson.html
>
> These pages need to be reviewed and confirmed or updated.  Upcoming 
> meeting.
>
> ----
>
>  4. /Library support/
>
> //
>
>      1. X3DJSONLD
>
> - https://github.com/coderextreme/X3DJSONLD
>
> This is where John’s collected and latest work is found.  This library 
> runs under node.js and browsers, and has many features, and has many 
> tests for X3D JSON in X3DOM and X_ITE.
>
> We might try to document it better.
>
>      2. X3DJSAIL Java
>
> X3D Java Scene Access Interface Library (X3DJSAIL)
>
> https://www.web3d.org/specifications/java/X3DJSAIL.html
>
> John has written some Java code that might import JSON to create 
> X3DJSAIL objects, that might work. However that works via the DOM, and 
> we hope to avoid dual translations for one pass to avoid brittleness.
>
> X3DJSAIL includes JSON output (export), which was pretty easy, but not 
> parsing (import).
>
>   * https://www.web3d.org/specifications/java/X3DJSAIL.html#Conversions
>
> We looked at whether Saxon library might work for JSON import, since 
> X3DJSAIL already includes Saxon saxon-he-10.6 (HE = Home Edition = 
> open source). Interestingly there is a function built in that might work,
>
>   * https://www.saxonica.com/html/documentation10/functions/fn/json-to-xml.html
>   * https://www.saxonica.com/html/documentation10/functions/fn/parse-json.html
>
>      3. X3DPSAIL Python
>
> There is a Python package “jsonschema” that appears to have 2020-12 
> support.
>
>   * Python package jsonschema, available on py
>   * https://github.com/Julian/jsonschema
>
> ----
>
>  5. /Online examples/
>
> Once we get an updated java library, we will resume unit testing of 
> all X3D JSON examples with Ant. Our build scripts currently support 
> jslint only.
>
>   * X3D Resources: Examples
>   * https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>
> ----
>
>  6. /X3D JSON Specification plans/
>
> Still TODO is make the C/C++/C# draft specifications available in 
> GitHub and web3d.org
>
> We have to finish X3D Architecture specification before we can add a 
> JSON specification.
>
> See the following for the road map of all X3D Specifications.
>
>   * X3D Specifications: Schema and DOCTYPE Validation
>   * https://www.web3d.org/specifications
>   * X3D Graphics Standards Relationships
>   * https://www.web3d.org/specifications/X3dSpecificationRelationships.png
>     <https://www.web3d.org/specifications/X3dSpecificationRelationships.png>
>
> ----
>
>  7. John’s movie recommendation
>
>   * "The Billion Dollar Code" NetFlix
>   * https://www.netflix.com/title/81074012
>
> ----
>
> Having fun diversifying X3D4 again!  8)
>
> all the best, Don
> -- 
> Don Brutzman  Naval Postgraduate School, Code USW/Br brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics 
> http://faculty.nps.edu/brutzman <http://faculty.nps.edu/brutzman>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20211011/b8e2ca49/attachment-0001.html>