[x3d-public] Minutes for X3D working group Wednesday 20 MAR 2018: X3D JSON encoding

[Don Brutzman]

Here is today's meeting agenda plus integrated minutes for X3D JSON encoding.

This X3D Working Group topic is being performed with public visibility.

Attendees: Puk, Carlson, Vince (late arrival), Brutzman.

On 3/13/2018 8:29 AM, Don Brutzman wrote:
> Subject: Re: Agendas for X3D working group Wednesday 13 and 20 MAR 2018

> Background: Web3D Consortium Working Groups
> ===========================================
>      Web3D Consortium Working Groups are technical committees that consider and develop solutions for specific technical challenges and opportunities.
>      Working group charters are reviewed by the Board of Directors to meet the needs of the Consortium and the community.
> 
>      The Extensible 3D (X3D) Working Group coordinates all technical development efforts.
>      Working groups are essentially driven by the efforts of participants.
>      They focus on issues and technologies that produce improvements to our open standards, always achieving results that are royalty free for any purpose.
>      All efforts are geared towards improving a coordinated set of steadily evolving ISO standards including X3D Version 4.
> 
> http://www.web3d.org/working-groups
> ===========================================
> 
> A. X3D Working Group schedule planning for this week and next week.
> [...]
> 3. Next week Wednesday 20 MAR 2018 we will focus on X3D JSON issues.  Background:
> 
>      X3D JSON Encoding
> http://www.web3d.org/wiki/index.php/X3D_JSON_Encoding

John noted that multiple implementations now exist for the "Initial Implementations" section.

John and Don will meet separately to update this section and report back to the group.

>      X3D to JSON Stylesheet Converter
> http://www.web3d.org/x3d/stylesheets/X3dToJson.html

This page has been maintained up to date.  We have tested ~4000 X3D scenes and the vast majority work.  In the past we appear to have fixed one of the most difficult/idiosyncratic bugs, namely escaping of quote and backslash characters.

John and Don are currently working on
- Java, XML, JSON equivalence for a several dozen (no more than a hundred) test scenes that are not meeting symmetric round-trip testing.
- Each bug is noted when corrected.
- We need to refresh our list of problem scenes and shrink to zero anomalies.
- Specifically we are testing  java to .x3d (canonical XML) round trip, then comparing (.java to JSON) with (.x3d to JSON).

>      X3D Resources: Example Scene Archives for X3D
> http://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>      includes 3900 example X3D .json scenes, for example
> http://www.web3d.org/x3d/content/examples/HelloWorld.json

These scenes provide our 3900 primary unit tests.

> a. John Carlson will review progress with X3DJSONLD implementation, and we will both discuss alignment with X3DJSAIL.
> https://github.com/coderextreme/X3DJSONLD
> http://www.web3d.org/specifications/X3dJsonSchemaDocumentation3.3/x3d-3.3-JSONSchema.html

John is further working on X3DJSONLD round-tripping by reading JSON and producing X3D.  This will give us full-circle round tripping between Java, JSON and .x3d XML.

John is also adding X3DJSONLD capabilities for both X3DOM and X_ITE in order to add visual testing.  Those groups can decide whether to integrate these capabilities.

> b. What steps are necessary to updating our X3D JSON Schema to handle bugs discovered in unit testing of example archives.
> http://www.web3d.org/specifications/x3d-3.3-JSONSchema.json

There is one official X3D JSON Schema, crafted by Roy Walmsley and published with other X3D validation assets.

John reported a spelling error "IndexedTrangleFanSet" to "IndexedTriangleFanSet" which is now fixed and checked in.

John has 2 additional JSON Schemas, including version 06.

> c. Consider potential upgrades of X3D JSON Schema to latest IETF working draft.
> https://en.wikipedia.org/wiki/JSON#JSON_Schema
> http://json-schema.org

For this external work, the JSON Schema specification itself is advancing to version 07.  So far additional changes there do not appear to be too great.

When X3D updates are at 06 (or possibly draft 07) we will compare the three existing JSON Schemas and agree on the design changes.

> d. Consider potential autogeneration of X3D JSON Schema using X3D Unified Object Model.
>      X3D Unified Object Model (X3DUOM)
> http://www.web3d.org/specifications/X3DUOM.html
> http://www.web3d.org/specifications/X3DObjectModel-3.3.xml

Of note is that XML and JSON Schemas both include the ability for Regular Expressions (regexes) of X3D datatype values.  Closely similar, possibly slight differences.  We will resume updating the following:

	X3D Regular Expressions (regexes)
	http://www.web3d.org/specifications/X3dRegularExpressions.html

Once we have that asset updated, Don will work on X3DUOM to capture all of these JSON and Regex patterns coherently.

John will continue work on his schema as well.

We will then use the mailing list to compare these patterns, deciding on "best of breed" design.

We will then work on autogeneration of X3D JSON Schema to ensure that all design patterns are sustainable, letting X3D v4.0 and v4.1 remain 100.0% consistent with v3.3.

> e. Specification drafting plans (Web3D member access, on request)
>      GitHub Web3D Consortium Specifications: X3D
> https://github.com/Web3DConsortium/X3D

Dick's review indicates that it seems pretty complete.  I agree.

We will need to review it again when our current implementation work completes.

There are no strict dependencies for JSON encoding on either XML or ClassicVRML or binary encoding specifications.  There is a design requirement for round-trip interoperability with no loss of information.  So each X3D encoding might list the others as informative references.

There will be a requirement for X3D JSON encoding to match latest official JavaScript (ECMAScript) references, making them normative.  We will further need to reconcile (or perhaps list both) the ECMA and IETF References.  We will take that for action.  Spec comment and subsequent Mantis comment seems appropriate, Dick will take next steps.

We are thinking that we should try to stay fully focused on finishing all aspects of X3D JSON Encoding, and also X3D Java SAI update, before pursuing autotranslation of a new X3D Python SAI from this work.  That might avoid much potential confusion; we'll see.

Given the maturity of the X3D JSON work, and the newness of the 19776-5 specification documents, X3D JSON Encoding will be an excellent test case for confirming a refreshed set of Web3D procedures matching ISO regulations.  (Thus an ISO NWIP will be part of the upcoming steps.)

We plan to meet monthly on working group on this topic.

All feedback welcome.  Have fun with X3D JSON!  8)

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman