[x3d-public] Well, @rotation errors (normalization) all originals; pajekOriginals; getting serious about X3D JSON Schema in draft-07
John Carlson
yottzumm at gmail.com
Sat Apr 3 20:57:41 PDT 2021
On 4/3/21 8:23 PM, Don Brutzman wrote:
> Hmmm. Several twists and turns here.
>
> 1. We already have a HelloWorld.json (of course) and it looks like
> yours strips out a bunch of stuff an inserts a Transform
>
> "@rotation":[2,1,0,3]
>
> There is a Special Place in Hell where everything has the same name.
> Let's not go there, please...
>
> All of our original models have .x3d in version control, with other
> file encodings and language bindings autogenerated.
>
> If you want to create a TestRotationsNormalization.x3d scene, then a
> good place for that to be maintained will be with the 15 other test
> scenes in
>
> * https://www.web3d.org/x3d/content/examples/Basic/development/
>
> Suggest you will want both an SFRotation (such as the Transform
> @rotation above) and an MFRotation, perhaps most familiar node might
> be an OrientationInterpolator.
>
> Please craft a TestRotationsNormalization.x3d scene and we can add it
> to the examples archive.
Agreed. I will create and validate (XML) as you suggest and check in. I
do not know how to use OrientationInterpolator at this point (I may have
known at one point).
Well, I have to recover my sourceforge password (new OS, linux). I hope
that sourceforge's emailer will work. Pauses. I guess I'll do
something when sourceforge gets back to me. Otherwise, I'll have to
create another account, and we'll need to do some admin.
Meanwhile, I pass you the file as an attachment. There was a diagnostic
in the Schematron (?) part of the X3D validator.
There doesn't appear to be a report on the hand-generated JSON schema
(4.0H), as previously reported. Here's the results of testing with JSON
schema draft 07 and X3D JSON schema 3.3: Probably adding an
OrientationInterpolator will tickle that part of the hand-generated schema?
~/x3dschema$ bash validateone.sh
/c/x3d-code/www.web3d.org/x3d/content/examples/Basic/development/TestRotationsNormalization.json
json-schema 3.3 Validation error
org.everit.json.schema.InternalValidationException:
#/X3D/Scene/-children/2/Group: #: 0 subschemas matched instead of one
/c/x3d-code/www.web3d.org/x3d/content/examples/Basic/development/TestRotationsNormalization.json
json-schema 3.3 Validation error
org.everit.json.schema.InternalValidationException:
#/X3D/Scene/-children/2/Group: #: 0 subschemas matched instead of one
/c/x3d-code/www.web3d.org/x3d/content/examples/Basic/development/TestRotationsNormalization.json
No report on Ajv means both 3.3 and 4.0H schemas passed or both failed.
Likely I will change the report at some point.
John
>
> ---
>
> 2. Glad that you are handling pajekOriginals by postprocessing. Since
> these are "original" models created by a tool, we wont' be changing them.
>
> * Pajek Analysis and visualization of very large networks
> http://mrvar.fdv.uni-lj.si/pajek
>
> * Pajek manual, p. 80, chapter 5 Exports to EPS/SVG/X3D/VRML
> https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.726.5730&rep=rep1&type=pdf
>
> Perhaps we should come up with a metadata scheme to indicate when a
> model has known errors so that automatic corrections are disable and
> diagnostics are silenced.
I have that for errors. Not sure about warnings.
>
> ---
>
> 3. Primary task remains.
>
> What I have been referring to with HelloWorld was confirming that JSON
> Schema and JSON X3D model are truly validating, and the patterns to
> accomplish that. Silent (non-reporting) command-line invocations
> aren't strong evidence and don't add much insight.
>
> As ever, the JSON Schema patterns are what we want to focus on using
> JSON Schema Draft 7. We should update our X3D JSON pages to match.
>
> * https://json-schema.org
>
> These people remain a curious bunch. While draft-07 now appears well
> supported, they are up to releasing draft 20.
>
> The following is their latest documentation, written for draft-07 but
> updated in 2021.
>
> * https://json-schema.org/understanding-json-schema/
>
> Once patterns are confirmed we can look at X3D JSON Schema
> autogeneration from X3DUOM.
>
> Based on your emails one might deduce that you may have such a thing,
> but it has not been reviewed and not been integrated. Please let's
> backtrack and get on firm footing.
I have been trying to improve the autogenerated schemas while looking
backwards at Roy's schema for possible good patterns, like @rotation.
We may need to fix @geoSystem next, I thought I brought @geoSystem
straight over from Roy's schema. I want to "look good" and cover major
potholes before reviewing and integrating. These are the two "big"
gotchas, along with not requiring @name in Metadata* nodes/statements.
Roy's schema is quickly becoming outdated by X3D4 standards. It has not
been updated for X3D4, except in tiny chunks.
Meanwhile, I believe that the autogenerated schemas have been tracking
fairly well with X3DUOM--all versions.
The differences in # of reports between autogenerated and handgenerated
schemas were pretty shocking with Everit. I am still recovering. I
feel pretty comfortable with Ajv, but we're a couple of versions behind,
as we are with JSON schema draft 7. We are using Ajv 6 and Ajv 8 exists
for the most recent standard. It appears that Ajv 6 and Ajv 8 are
incompatible, as Ajv 8 does not really handle JSON schema Draft 7 in my
first attempt (I think I am probably missing some sleuthing here).
>
> If you do have a single best-effort X3D JSON schema, please provide a
> stable url so that we can study it.
https://github.com/coderextreme/x3dschema/blob/master/x3d-4.0-JSONSchema.json
(autogenerated, should be compatible with X3D4)
https://github.com/coderextreme/x3dschema/blob/master/x3d-4.0H-JSONSchema.json
(handgenerated, not yet X3D4 compatible--PBR, Acoustics, and most
modifications to X3D4 needed)
Where X3D4 versions of X3D JSON schema (auto) may appear first (but no
guarantees):
https://github.com/coderextreme/X3DJSONLD/tree/master/src/main/schema
(same name as autogenerated above).
Versions 3.* and 4.* should appear along-side each other.
Long term maintainance of hand-generated schema remains on sourceforge,
but needs updating. Autogenerated schemas are checked into
sourceforge, but probably not updated often.
I added you as a collaborator to coderextreme/x3dschema, if desired.
If this is desired to be a private repository, please let me have access
to a git repository under Web3DConsortium.
Current autogenerating script (all versions).
https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/shell/generateSchema.sh
Current autogenerator (per version).
https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/python/etgenerateJSONschema.py
>
> ---
>
> 4. Everit
>
> Are you still getting this from the following site?
>
> * https://github.com/everit-org/json-schema
>
> * https://github.com/everit-org/json-schema/releases
You can see what I'm downloading here:
https://github.com/coderextreme/x3dschema/blob/master/validate.sh
(sorry for making you click on links, see below)
git clone https://github.com/everit-org/json-schema
I am not familiar with releases link (thanks, they should put that on
the main repo webpage somewhere), but it is useful. I'm not sure if
that's a fat jar or not. Using a maven build instead of a jar allows
you to download jars for addition to classpath (perhaps not ideal) in
validate.sh and validateone.sh.
I will add you as a collaborator on x3dschema, Don.
>
> ---
>
> 5. Next
>
> This week I cannot meet Monday but have sent you an update to meet on
> Web3D line Tuesday 14-1500 pacific, hope that can work for you.
>
> Thanks for continued pounding on the problem. Let's hammer this into
> a repeatable understandable result, rather than scattered pieces.
Thanks for working with me!
John
>
> On 4/2/2021 10:50 PM, John Carlson wrote:
>>
>> Here's a small test (attached.).
>>
>> John
>>
>> On 4/3/21 12:44 AM, John Carlson wrote:
>>> Likely we need a very small test case. I will work on one.
>>>
>>> John
>>>
>>> On 4/3/21 12:42 AM, John Carlson wrote:
>>>> No worries about lax or non-lax validation. The only cases (1800)
>>>> showing validation errors with @rotation are pajekOriginals. This is
>>>> easily handled in a post process filter. Sorry for not filtering them
>>>> earlier. Everit results next (Ajv doesn't show up, likely because
>>>> both schemas fail).
>>>>
>
>
>>
>>
>> The strange thing is, the autogenerated code did not throw any errors
>> with Everit for rotation.
>>
>> On 4/2/21 11:07 PM, John Carlson wrote:
>>> It appears that all the rotation errors were in originals:
>>>
>>> https://coderextreme.net/rot.txt
>>>
>>> So I could go either way with the auto-generated X3D JSON schema,
>>> pasting fixed text or generating less restrictive code.
>>>
>>>
>>> John
>>>
>
>
>>>> "@rotation":[2,1,0,3],
>>>>
>>>> I made this change to show the errors. I do not know why the hand
>>>> generated schema errors do not show up. That seems to be an issue.
>>>> Thanks for the backwards step Don.
>>>>
>>>> So strict validation is still in place, as 2 > 1.
>>>>
>>>> So the current issue seems to be the hand generated schema. The
>>>> definitions are in different order. Changing order doesn't fix
>>>> things.
>
> We need to get on the "same sheet of music" with the same best-case
> schema.
>
>>>> I guess i would propose to file an issue with Everit about the two
>>>> schemas and the HelloWorld.json
>>>>
>>>> Good catch!
>>>>
>>>> John
>
> If we can apply Everit in the Ant build.xml then we can repeat the
> error. At that point any problem report becomes measurably fix-able.
>
>>>> On 4/2/21 11:22 PM, Don Brutzman wrote:
>>>>> Please backtrack. Not clear you heard what we said this morning or
>>>>> posted.
>>>>>
>>>>> Don't worry about rotation normalization now please. Lax validation
>>>>> is good, normalize() routines in our libraries can help.
>>>>>
>>>>> You are showing results from gigantic visualization files done by a
>>>>> grad student with the pajek tool eleven years ago.
>>>>>
>>>>> Sorry but can't keep up if we can't do Hello World.
>>>>>
>>>>> Thanks John.
>
> all the best, Don
-------------- next part --------------
A non-text attachment was scrubbed...
Name: TestRotationsNormalization.x3d
Type: model/x3d+xml
Size: 2529 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20210403/0799d4f2/attachment.x3d>
More information about the x3d-public
mailing list