[x3d-public] Are you motivated by schema checking? Volunteer position
John Carlson
yottzumm at gmail.com
Sat Sep 2 22:22:54 PDT 2023
If anyone wants to add i18n and explainable messages to x3dvalidate with
npm ajv-i18n, there's a volunteer position open.
ajv-i18n - npm (npmjs.com) <https://www.npmjs.com/package/ajv-i18n>
coderextreme/x3dvalidate: Validate JSON files against various versions of
X3D JSON schema with Ajv (github.com)
<https://github.com/coderextreme/x3dvalidate>
Also, one can get involved with X3D JSON schema checking and X3D JSON
schema generation (I guess XSLT or python is OK for that). The latter will
use X3D Unified Object Model (X3DUOM) as a source. There is precedent to
build on with python, but a clean extended schema is welcome.
If one prefers a more Java approach with X3DSAIL validation, there's
x3dschema:
coderextreme/x3dschema: X3D Schema work (JSON) (github.com)
<https://github.com/coderextreme/x3dschema>
You may choose a better name.
As an extra bonus, add line numbers reports from JSON document to Ajv, if
something doesn't already exist.
Now, back to GraalVM+NetBeans
Enjoy!
On Sun, Sep 3, 2023 at 12:02 AM John Carlson <yottzumm at gmail.com> wrote:
> Okay, here's where I stepped off wrong:
>
>
> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-IS.proof/Part01/components/hanim.html#HAnimMotion
>
> Lists the channels field as an SFString, not an MFString, so. That
> doesn't explain why I couldn't find it or the HAnimMotion on the website,
> here:
>
>
> https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.json
>
> I can check the XML again:
>
> [image: image.png]
>
> Note that .x3dj is the extension that x3d-tidy (Holger's) uses. I will
> now convert with that.
>
> $ npx x3d-tidy at 1.0.91 -i KoreanCharacterMotionAnnexD01Jin.x3d -o
> KoreanCharacterMotionAnnexD01Jin.x3dj
>
> Apparently the current version of x3d-tidy doesn't produce anything???
>
> Here is link to test with:
>
> x3d / Code / [r35406]
> /www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.x3d
> (sourceforge.net)
> <https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.x3d>
>
> Or you can get it from web3d.org X3D resources.
>
> Here's what version 1.0.91 produces:
>
> { "HAnimMotion":
> {
> "@DEF": "hanim_motion_animation",
> "@enabled": false,
> "@joints": "HumanoidRoot l_hip l_knee l_talocrural r_hip
> r_knee r_talocrural vl5 IGNORED l_shoulder l_elbow l_radiocarpal IGNORED
> r_shoulder r_elbow r_radiocarpal IGNORED skullbase",
> "@channels": "6 Xposition Yposition Zposition Zrotation
> Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation
> Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3
> Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation
> Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation
> Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3
> Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation
> Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation
> Yrotation 3 Zrotation Xrotation Yrotation",
>
>
> I will check against X3D JSON schema. with
> https://github.com/coderextreme/x3dvalidate:
>
>
> $ node x3dvalidate.js KoreanCharacterMotionAnnexD01Jin.*
> Loading schema
> Parsing schema
> Adding schema
> Schema 4.0 added
> Schema 4.0 compiled
>
> ================================================================================
> File: KoreanCharacterMotionAnnexD01Jin.json
> Suppressing @USE missing property. Use --fullreport to reveal possibly
> confusing errors
> Suppressing @USE missing property. Use --fullreport to reveal possibly
> confusing errors
> Suppressing null passingSchemas. Use --fullreport to reveal possibly
> confusing errors
> Suppressing null passingSchemas. Use --fullreport to reveal possibly
> confusing errors
> Error invalid file KoreanCharacterMotionAnnexD01Jin.json
> keyword: type
> location in document:
> /X3D/Scene/-children/3/HAnimHumanoid/-motions/0/HAnimMotion/@channels
> message: must be array
> params: {"type":"array"}
> file: KoreanCharacterMotionAnnexD01Jin.json
> version: 4.0
>
>
> ================================================================================
> File: KoreanCharacterMotionAnnexD01Jin.x3dj
> Suppressing @USE missing property. Use --fullreport to reveal possibly
> confusing errors
> Suppressing @USE missing property. Use --fullreport to reveal possibly
> confusing errors
> Suppressing null passingSchemas. Use --fullreport to reveal possibly
> confusing errors
> Suppressing null passingSchemas. Use --fullreport to reveal possibly
> confusing errors
> Error invalid file KoreanCharacterMotionAnnexD01Jin.x3dj
> keyword: type
> location in document:
> /X3D/Scene/-children/3/HAnimHumanoid/-motions/0/HAnimMotion/@channels
> message: must be array
> params: {"type":"array"}
> file: KoreanCharacterMotionAnnexD01Jin.x3dj
> version: 4.0
>
> This is where I got the idea that @channels was an MFString?
>
> I see in the X3D JSON schema that:
>
> "@channels": {
> "$comment": "MFString inputOutput",
> "type": "array",
> "items": {
> "type": "string"
> }
> },
>
> Says it's an MFString. Let's look in X3DUOM. Here's the kicker, X3DUOM
> doesn't agree with X3D4, from my reading, but I am old.
>
> <ConcreteNode name="HAnimMotion">
> <InterfaceDefinition specificationUrl="
> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-IS.proof/Part01/components/hanim.html#HAnimMotion
> "
> appinfo="An HAnimMotion node supports
> discrete frame-by-frame playback for H-Anim motion data animation.">
> <componentInfo name="HAnim" level="2"/>
> <Inheritance baseType="X3DChildNode"/>
> <field name="channels"
> type="MFString"
> accessType="inputOutput"
> description="list of number of channels for
> transformation, followed by transformation type of each channel of data."/>
>
>
> <xs:element name="HAnimMotion">
> [ snip for readability ]
> <xs:complexType>
> <xs:complexContent>
> <xs:extension base="X3DChildNode">
> <xs:attribute name="channels"
> type="MFString"/>
>
> So X3DUOM agrees with the XML schema, which agrees with the X3D JSON
> schema.
>
>
>
> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-IS.proof/Part01/components/hanim.html#HAnimMotion
>
>
> 26.3.4 HAnimMotion
>
> HAnimMotion : X3DChildNode {
> SFBool [in] next
> SFBool [in] previous
> SFString [in,out] channels ""
> MFBool [in,out] channelsEnabled []
> SFString [in,out] description ""
> SFBool [in,out] enabled TRUE
> SFInt32 [in,out] endFrame 0 [0,∞)
> SFTime [in,out] frameDuration 0.1 (0,∞)
> SFInt32 [in,out] frameIncrement 1 (-∞,∞)
> SFInt32 [in,out] frameIndex 0 [0,∞)
> SFString [in,out] joints ""
> SFInt32 [in,out] loa -1 [-1,4]
> SFBool [in,out] loop FALSE
> SFNode [in,out] metadata NULL [X3DMetadataObject]
> SFString [in,out] name ""
> SFInt32 [in,out] startFrame 0 [0,∞)
> MFFloat [in,out] values [] (-∞,∞)
> SFTime [out] cycleTime [0,∞)
> SFTime [out] elapsedTime (0,∞)
> SFInt32 [out] frameCount [0,∞)
> }
>
> The channels field is definitely SFString. It's OK with me if it's an SFString, by gummit!
>
>
> Please update your files and/or the standard.
>
> Don, I hope you appreciate this. My 7th sense detects this kind of stuff, and it just pops out at me.
>
> John
>
>
> On Sat, Sep 2, 2023 at 10:24 PM John Carlson <yottzumm at gmail.com> wrote:
>
>> Easy pickings! This email is meant to help determine whether MFStrings
>> should be encoded as JSON arrays or JSON string. This will help when we
>> get to X3D JSON schema (if ever). It will also help guide tool developers
>> as they start putting out JSON encoded X3D.
>>
>> You have a @url property that is an MFString. Like:
>>
>> "@url": [
>>
>> "earth-topo.png",
>>
>> "earth-topo.jpg",
>>
>> "earth-topo-small.gif",
>>
>> "
>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo.png",
>>
>> "
>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg",
>>
>> "
>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif"
>>
>> ]
>>
>>
>> And I see: X3D to JSON Stylesheet Converter (web3d.org)
>> <https://www.web3d.org/x3d/stylesheets/X3dToJson.html>
>> MFString is array of quoted SFString (XML attribute) values JSON array
>> of *string* type
>>
>>
>> That looks like a JSON array of strings. All is well and good from my
>> point of view, for the @url and @justify MFString you mentioned. Now to
>> address the @channels MFString ...
>>
>>
>> We'll go with the sample I posted: Humanoid Animation X3D Examples
>> Archive, Specifications, Korean Character Motion Annex D 01 Jin (web3d.org)
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01JinIndex.html>
>>
>>
>> In XML:
>>
>>
>> <HAnimMotion DEF='hanim_motion_animation
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.html#hanim_motion_animation>
>> ' channels='6 Xposition Yposition Zposition Zrotation Xrotation
>> Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3
>> Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation
>> Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation
>> Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3
>> Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation
>> Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation
>> Yrotation 3 Zrotation Xrotation Yrotation 3 Zrotation Xrotation Yrotation 3
>> Zrotation Xrotation Yrotation'
>>
>>
>> That looks good.
>>
>>
>> So I dig into the JSON...
>>
>>
>>
>> web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.json
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications/KoreanCharacterMotionAnnexD01Jin.json>
>>
>>
>> There's no "@channels" property at all. There's not even HAnimMotion in
>> the file at all!
>>
>>
>> Hmm. What do I see on my local machine" Which may have an out of date
>> JSON with the website, I admit:
>>
>>
>> c:/x3d-code/
>> www.web3d.org/x3d/content/examples/HumanoidAnimation/Specifications
>>
>> $ grep channels *Motion* | cut -c1-150
>>
>> KoreanCharacterMotionAnnexD01Jin.json: "@channels":"6
>> Xposition Yposition Zposition Zrotation Xrotation Yrotation 3 Zrotation
>> Xrota
>>
>> KoreanCharacterMotionAnnexD01Jin.x3d: <HAnimMotion
>> DEF='hanim_motion_animation' channels='6 Xposition Yposition Zposition
>> Zrotation Xrotation Yro
>>
>>
>> So I conclude that there's no @channels or HAnimMotion in the JSON on the
>> website. Where did it go? No, I'm not going to capture a record of your
>> every move, Don.
>>
>>
>> Next we look at another popular discussion, NavigationInfo @type MFString
>>
>>
>> Basic X3D Examples Archive, X3D Specifications, Red Sphere Blue Box
>> (web3d.org)
>> <https://www.web3d.org/x3d/content/examples/Basic/X3dSpecifications/RedSphereBlueBoxIndex.html>
>>
>>
>> Confirmed MFString, see extra quotes in XML:
>>
>>
>> <Transform>
>> <NavigationInfo headlight='false' type='"EXAMINE"'/>
>> <DirectionalLight/>
>>
>>
>> Now we look a the JSON:
>>
>>
>> "Transform": {
>>
>> "-children": [
>>
>> {
>>
>> "NavigationInfo": {
>>
>> "@headlight": false,
>>
>> "@type": "\"EXAMINE\""
>>
>> }
>>
>> },
>>
>>
>> Hmm. an MFString, @type encoded at a JSON string!
>>
>>
>> So...is an MFString a JSON array of strings or a JSON string? Please
>> document things extremely well, thanks! We need to call out exceptions in
>> your documentation.
>>
>>
>> Indeed, in the documentation I listed, there is:
>>
>>
>> { "*NavigationInfo*":
>> {
>> "@type":["NONE"]
>> }
>> },
>>
>>
>> So I'm guessing we're concluding that MFStrings are JSON arrays.
>>
>>
>> Any input, Vince?
>>
>>
>> Thanks!
>>
>>
>> John
>>
>>
>>
>> On Sat, Sep 2, 2023 at 7:06 PM Brutzman, Donald (Don) (CIV) <
>> brutzman at nps.edu> wrote:
>>
>>> John, the authoritative X3dToJson.xslt stylesheet is at
>>>
>>>
>>>
>>> - https://www.web3d.org/x3d/stylesheets/X3dToJson.xslt
>>> -
>>> https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X3dToJson.xslt
>>>
>>>
>>>
>>> Here is an X3D JSON model with several MFString values, three excerpts
>>> follow. Please identify a problem with these MFString values in JSON, if
>>> there is one.
>>>
>>>
>>>
>>> -
>>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter01TechnicalOverview/HelloWorld.json
>>>
>>>
>>>
>>> "WorldInfo": {
>>>
>>> "@info": [
>>>
>>> "Example scene to illustrate a simple X3D model"
>>>
>>> ],
>>>
>>> "@title": "Hello World!"
>>>
>>> }
>>>
>>>
>>>
>>>
>>>
>>> "ImageTexture": {
>>>
>>> "@DEF": "ImageCloudlessEarth",
>>>
>>> "@url": [
>>>
>>> "earth-topo.png",
>>>
>>> "earth-topo.jpg",
>>>
>>> "earth-topo-small.gif",
>>>
>>> "
>>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo.png",
>>>
>>> "
>>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo.jpg",
>>>
>>> "
>>> https://www.web3d.org/x3d/content/examples/Basic/earth-topo-small.gif"
>>>
>>> ]
>>>
>>> }
>>>
>>>
>>>
>>> "Shape": {
>>>
>>> "-geometry": {
>>>
>>> "Text": {
>>>
>>> "@DEF": "TextMessage",
>>>
>>> "@string": [
>>>
>>> "Hello",
>>>
>>> "world!"
>>>
>>> ],
>>>
>>> "-fontStyle": {
>>>
>>> "FontStyle": {
>>>
>>> "@justify": [
>>>
>>> "MIDDLE",
>>>
>>> "MIDDLE"
>>>
>>> ]
>>>
>>> }
>>>
>>> }
>>>
>>> }
>>>
>>> },
>>>
>>>
>>>
>>> all the best, Don
>>>
>>> --
>>>
>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>>
>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>>
>>> X3D graphics, virtual worlds, navy robotics
>>> https://faculty.nps.edu/brutzman
>>>
>>>
>>>
>>> *From:* x3d-public <x3d-public-bounces at web3d.org> *On Behalf Of *John
>>> Carlson
>>> *Sent:* Saturday, September 2, 2023 2:04 PM
>>> *To:* X3D Graphics public mailing list <x3d-public at web3d.org>
>>> *Subject:* Re: [x3d-public] @channels in HAnimMotion JSON. MFString
>>> again
>>>
>>>
>>>
>>> To clarify:
>>>
>>>
>>>
>>> I'm not writing server side code to convert my XML to JSON. Holger
>>> already offers conversions on the web. I don't want to support a huge web
>>> server infrastructure.
>>>
>>>
>>>
>>> If someone really wants MFStrings as JSON strings, please by all means,
>>> take a copy of X3dToJson.xslt and modify it to your hearts delight! I can
>>> provide Java-based infrastructure, here:
>>> https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/java/net/coderextreme/RunSaxon.java
>>>
>>>
>>>
>>> This is well tested, but perhaps poorly documented.
>>>
>>>
>>>
>>> Here are some examples running it in a pipeline. The "$@" means a list
>>> of X3D XML files. I am running the conversions on multiple processors with
>>> the -P ${PROCESSORS}. So something like xargs -P 8, for example , would run
>>> on 8 cores. If you want to distribute, I recommend GNU parallel, but I
>>> haven't used it.
>>>
>>>
>>>
>>> $ javac -cp "${CLASSPATH}" net/coderextreme/RunSaxon.java
>>>
>>> $ ls -d "$@" | grep -v intermediate | grep -v "\.new" | tr '\n' '\0'|
>>> xargs -0 -P "$PROCESSORS" java net.coderextreme.RunSaxon --- "${OVERWRITE}"
>>> --"${STYLESHEETDIR}/X3dToJson.xslt" -json
>>>
>>>
>>>
>>> $ ls -d "$@" | xargs -P $PROCESSORS java net.coderextreme.RunSaxon
>>> ---overwrite ---silent --../lib/stylesheets/X3dToNodeJS.xslt -js
>>> ---../node/net/x3djsonld/data/
>>>
>>> $ ls -d "$@" | xargs -P $PROCESSORS java net.coderextreme.RunSaxon
>>> ---overwrite ---silent --../lib/stylesheets/X3dToJava.xslt -java
>>> ---../java/net/x3djsonld/data/
>>>
>>> $ ls -d "$@" | xargs -P $PROCESSORS java net.coderextreme.RunSaxon
>>> ---overwrite ---silent --../lib/stylesheets/X3dToPython.xslt -py
>>> ---../python/net/x3djsonld/data/
>>>
>>>
>>>
>>> I will probably add the X3dToES5.xslt in the near future as I'm working
>>> with GraalVM.
>>>
>>>
>>>
>>> Enjoy!
>>>
>>>
>>>
>>> John
>>>
>>>
>>>
>>> On Sat, Sep 2, 2023 at 2:37 PM John Carlson <yottzumm at gmail.com> wrote:
>>>
>>> So no one wants have a frank conversation about this?
>>>
>>>
>>>
>>> We need to figure out a solution to MFString in JSON. We can’t do
>>> onesies twosies.
>>>
>>>
>>>
>>> My web page now uses Holger’s Browser.toJSONString(). No other solution
>>> works half decently on the web for converting XML to JSON in a browser.
>>> X_ITE creates JSON arrays for MFStrings.
>>>
>>>
>>>
>>> I have brought up this issue many, many times. Our current Saxon/XSLT
>>> solution does not work in a web client. The Saxon JavaScript version just
>>> isn’t there. I am not going to write server side code.
>>>
>>>
>>>
>>> I am converting XML to JSON over from stylesheets to x3d-tidy, even
>>> though Saxon itself works perfectly well in X3D-Edit and in my
>>> RunSaxon.java. Ideally, we could match a web solution with a server-side
>>> solution.
>>>
>>>
>>>
>>> At this point, I am following Holger’s lead because there’s no Saxon
>>> solution for web clients.
>>>
>>>
>>>
>>> Are we killing X3D JSON? Then let’s be upfront about it and not talk in
>>> smoke-filled rooms.
>>>
>>>
>>>
>>> John
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Sat, Sep 2, 2023 at 2:58 AM John Carlson <yottzumm at gmail.com> wrote:
>>>
>>>
>>>
>>> The @channels field value in the translated JSON appears as a JSON
>>> string, not as a JSON array. Here we have the MFString problem again.
>>>
>>>
>>>
>>> It appears that we have different ideas of what "MF" means. To me, MF
>>> means a JSON array in JSON. Holger's x3d-tidy tool and X_ITE renders
>>> MFStrings as arrays. I created a special case NavigatorInfo at type. Can
>>> we convert all MFStrings to JSON strings on output, please? I think that
>>> would make everyone happy?
>>>
>>>
>>>
>>> If our tools don't agree, that will confuse people.
>>>
>>>
>>>
>>> There's could be issues around MFStrings in Text/fontStyle still. and
>>> other places, like urls...do you want those rendered as strings or arrays.
>>>
>>>
>>>
>>> I'm going to proceed with modifying my schema generator, I'll keep a
>>> backup of current stuff.
>>>
>>>
>>>
>>> John
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230903/80fe1dbf/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 48029 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230903/80fe1dbf/attachment-0001.png>
More information about the x3d-public
mailing list