[x3d-public] X3D JSON schema, identifying X3D type for items within an array. (was element)

[Don Brutzman]

On 4/6/2021 7:04 PM, John Carlson wrote:
> 
> Good feedback!  Here's some feedback on the feedback.  Would it satisfy
> you if we put appinfo from X3DUOM FieldType in an items' $comment?  If
> that's okay, you can probably skip most of the rest of this message.

Looking into JSON schema: yes just put an informational schema comment within each field that lists type/accessType as found in X3DUOM (as found in XML Schema, as found in X3D Specfication).

Looking out of JSON schema: we want JSON Schema to allow "#comment" in JSON files anywhere we put them.  We can consider modifying this syntax if needed, since it is our creation to support fuller interoperability/roundtrip-aility with other file encodings.

Please don't agonize over type correspondences, they are really simple.  If you want a correspondence table I will produce it.

* https://www.web3d.org/x3d/tooltips/X3dTooltips.html#FieldTypesTable

Overall goal restated: we want JSON schema to validate as might be practical, and we don't want JSON schema to do what it can't do.

Same for you please John: do what you can, and don't do what you can't!  Your ability to check and test is impressive and much appreciated.

Some future good news: if we get everything working, and documented as best we can, and even successfully specified via ISO: then future maintenance will be everyone else's problem, we won't have to worry about it!  8)

I think that covers about everything in this message.  Simplify, simplify...


> There is potentially some possibility of parsing ContentModel for
> ordering stuff in "head" and "X3D"--let me know if ContentModel is the
> right place to look for order. I don't think that accessType is doable
> due to potential shared fields (see below).
> 
> On 4/6/21 7:07 PM, Don Brutzman wrote:
>> thanks for review and progress
>>
>> On 4/6/2021 4:08 PM, John Carlson wrote:
>>>
>>>
>>> Decision should be made whether array X3D type to array item X3D type
>>> map should be in X3D JSON schema generator or X3DUOM.
>>
>> all types (array or singleton) already defined in X3DUOM so no
>> decision needed.
>>
>> what question do you have?
> 
> 
> For example, take FieldType SFVec3f in X3DUOM.    FieldType SFVec3f says
> it's an array, but does not specify the X3D type of the items in the
> array, except in appinfo (SFFloat), and I don't have a desire to parse
> through that.   Perhaps we could add "itemType" to each FieldType in
> X3DUOM which is an array--or is "itemType" too JSONic?  It's okay with
> me if I put the proposed map in the JSON schema generator, I just figure
> you would complain about hard-coded things that we're trying to make
> explicit in X3DUOM, like the order of objects in head and X3D.
> 
> If you've got the order expressed in X3DUOM, then we can do something
> like this in JSON schema:
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstackoverflow.com%2Fquestions%2F31691247%2Fhow-can-i-define-the-sequence-of-properties-in-json-schema&data=04%7C01%7Cbrutzman%40nps.edu%7C7608645de739424bb1b408d8f969a088%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637533579357994823%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=pPe3GF%2FopftsSkSRBgQFAR0oH3xBjDE4oThMn3YG%2FAs%3D&reserved=0
> I am really surprised at how easy some stuff is in JSON Schema! As for
> order, is that done in ContentModel in X3DUOM?  Just confirming before I
> run off to program.  I am not familiar with how ContentModel works at
> this time.   Do you have documentation on it?
> 
> 
> What I feel we're trying to do is add "X3D" information to the X3D JSON
> schema, so it would be good to remain "X3D-like" for item types, to
> maintain historical information, and so we can avoid creating extra
> files from X3DUOM for my JS programs--see fieldTypes.js and
> mapToMethod.js in X3DJSONLD/src/main/node  (the JavaScript remains out
> of date unless a full build is run).   Also it would be good to look at
> mapToMethod2.js as these were the things I found missing from
> mapToMethod.js (but I need to run through that with the newer X3DUOM,
> sorry).
> 
>>
>>> As discussed in our meeting today, I am adding a $comment with X3D type
>>> before "type": occurs in the X3D JSON schema.
>>>
>>> This will require more work than initially required, although the
>>> changes so far have been fairly straightforward in the somewhat messy
>>> code.
>>>
>>> For example:
>>>
>>>              "@url": {
>>>                "pattern": "^(\\s|\\S)*$",
>>> +              "$comment": "X3D type: MFString",
>>>                "type": "array",
>>>                "minItems": 1,
>>>                "items": {
>>>                  "format": "uri-reference",
>>> +                "$comment": "X3D type: MFString",
>>>                  "type": "string"
>>>                }
>>>              },
>>>
>>>
>>> The X3D type for the items object, should be SFString one would think.
>>> So I'll have to check to see if I'm in an array.
>>
>> Great.  No need to say "X3D type: "
> 
> See suggestion on including appinfo in some $comment's.
> 
>>
>> I would add accessType as well please.
>>
>> example: "$comment": "MFString inputOutput",
> 
> I'm a little bit fuzzy on adding an accessType, because shared field
> definitions may have different accessTypes per node/statement.  If we're
> not sharing field definitions, then this would be OK.
> 
> I like to write self-documenting code (note my lack of actual
> documentation except for emails :().  Sometimes I do better than
> others.   I was frankly puzzled when someone commented on the amount of
> documentation in my repositories.  I was pretty shocked, and I wasn't
> sure if they were kidding or not.
> 
> Someday, I'm going to write an web crawler that reads all my emails and
> constructs documentation :)  Or I'll just let a startup, OpenAI or
> Google do it.
> 
>>
>>> Probably by looking for "items".  Then I'll look for MF in the field
>>> type, and replace the first M with an S.
>>>
>>>
>>> But here are added problems looking at arrays:
>>>
>>>
>>>              "@bboxSize": {
>>>                "pattern":
>>> "^\\s*(([+-]?((0|[1-9][0-9]*)(\\.[0-9]*)?|\\.[0-9]+)([Ee][+-]?[0-9]+)?)\\s+){2}([+-]?((0|[1-9][0-9]*)(\\.[0-9]*)?|\\.[0-9]+)([Ee][+-]?[0-9]+)?)\\s*$",
>>>
>>> +              "$comment": "X3D type: SFVec3f",
>>>                "type": "array",
>>>                "minItems": 3,
>>>                "maxItems": 3,
>>>                "items": {
>>>                  "default": -1,
>>> +                "$comment": "X3D type: SFVec3f",
>>>                  "type": "number"
>>>                }
>>>              },
>>>
>>>
>>> In this case, the SFVec3f would be found, and a mapping between SFVec3f
>>> to SFFloat for individual items should be made.
>>
>> I think your actual mapping is SFVec3f to Javascript "type": "number"
>> with min/maxItems 3  which appears above.
> 
> 
> Yes, we could leave off the $comment for native JavaScript.  I don't
> actually know what the items of a SFVec3f are except that the appinfo in
> X3DUOM says the numbers are SFFloat's.  That's the relationship I'm
> trying to directly put in the X3DUOM so I don't have to build the map
> myself.  I believe it will be more informative to declare the types of
> the items of an array in a $comment in the schema (and in X3DUOM too).
> If we're in JSON, we pretty much know it's a number, since there are no
> quotes around it.
> 
>>
>>> So perhaps a simple map could be constructed which maps array type to
>>> item type.
>>>
>>>
>>> Does this sound pretty feasible, essentially a hard coded map from array
>>> type to item type in the generator code?
>>
>> sounds trivially simple and workable
> 
> 
> See above where I attempt a design for putting the map into X3DUOM, or
> just copying appinfo into the $comment value for items.
> 
>>
>>
>>> I will look for additional issues now. I'm not seeing much right away.
>>> I know I ignore some of the "type" fields right now, particularly in
>>> hard coded definitions.
>>>
>>> Note that I'm inserting documentation.  This shouldn't affect the
>>> running of the generator.
>>>
>>> My main question is, where does the information reside?  Should it be in
>>> X3DUOM?
>>
>> (on loudspeaker) "I Say Again" all of the needed information is
>> already in X3DUOM.
> 
> 
> Yes, but perhaps not easily accessible? I don't want you to have to say
> something twice, and I can certainly attempt to parse appinfo.  I don't
> particularly want you to have to parse appinfo either.  I can egrep
> 'FieldType|appinfo' to collect information that is relevant from X3DUOM
> to start--actually gathering the info by hand might be easier and then
> you can verify the information I present and then put it into X3DUOM.
> Then I'll start querying FieldType (or whatever you prefer) in the X3D
> JSON schema generator. I currently only query for regex in FieldType, it
> should be easy to add more features!  Hurrah!
> 
>>
>>> Does anyone mind if I start calling my generators "synthesizers"?  I'm
>>> on a new kick!
>>
>> yes, please don't kick us with mixed terms...
> 
> 
> Okay!
> 
>>
>>> Note that we are moving towards XSL for our primary JSON schema
>>> synthesizer, lessening requirements on future maintainers. We're
>>> currently using python to suss out patterns and requirements for
>>> X3DUOM.   The python pretty much is impossible to maintain (even the guy
>>> that wrote it admits that).  If you know any "younguns" who love XSL,
>>> encourage them to join and contribute to X3D!
>>
>> when you have a good pattern in a good draft-07 schema, i will create
>> an XSLT autogenerator that converts X3DUOM XML to build an X3D JSON
>> schema.
> 
> 
> Think about how to add JSON schemas for various profiles. That was one
> thing I hadn't gotten to yet, and I know you added that to X3DUOM a
> while ago.   Sorry for not keeping up-to-date.
> 
>>
>>> Building holistic schema synthesizers.
>>>
>>> My next job will be to build holistic spaceship synthesizers.
>>
>> please leave me behind
> 
> 
> :)  maybe we'll take a saliva sample if we need an expert on XSL
> programming.
> 
> I think later JSON schema versions are adding stuff like "how to show on
> a GUI" so it would be good to keep a handle on what they are adding to
> the JSON schema when we want to upgrade.  Perhaps we could just put
> appinfo in $comment and call it a day?
> 
> It seems like there's still a lot of work surrounding X3D JSON. I enjoy
> that everyone puts up with me, but I don't know of any customers for X3D
> JSON!
> 
> Respectfully (with a bit of tongue in check).
> 
> John

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman