[x3d-public] [x3d] Ideas for the x3d.py package development - JSON support

Mon Nov 22 07:37:44 PST 2021

I will continue to support X3DJSAIL and X3D JSON.  Otherwise, I’m waiting
on interested parties with use cases.  What are the top priorities on the
wish list?  My top priority creating/fixing test cases for JSON.  AFAIK,
 JSON continues to work in X3DOM and X_ITE?

I realize that the interfaces between Java and Python to JavaScript may not
be working properly.

Should we review the JSON working draft?  Has it changed recently?

 Tasks in X3DJSONLD will be ongoing work with bundling.   I have introduced
some window . globals that need to be addressed.

John

On Mon, Nov 22, 2021 at 7:44 AM Brutzman, Donald (Don) (CIV) <
brutzman at nps.edu> wrote:

> Hi John, you addressed this to me so I will reply.
>
>
>
> As ever, extensions and examples are always welcome.  Any changes to
> x3d.py itself will need to be native Python with clearly understood use
> cases.
>
>
>
> Good luck with your Python programming efforts.
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics https://
> faculty.nps.edu/brutzman
>
>
>
> *From:* John Carlson <yottzumm at gmail.com>
> *Sent:* Sunday, November 21, 2021 7:25 PM
> *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> *Cc:* vmarchetti at kshell.com; X3D Graphics Working Group <x3d at web3d.org>
> *Subject:* Re: [x3d] Ideas for the x3d.py package development - JSON
> support
>
>
>
> What I’m envisioning is a set of python methods for building a dict for
> output with the json package or a set of methods for outputting a JSON
> string.
>
>
>
> I’m also envisioning a set of methods for taking a dict and producing a
> x3d.py scene graph.
>
>
>
> Let me know if I am off base.
>
>
>
> John
>
>
>
> Sent from my iPad
>
>
>
> On Nov 19, 2021, at 3:30 AM, Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
> 
>
> Developers and users might do anything, but one of the declared goals of
> X3DPSAIL x3d.py is minimalist pure python.  So far it has no library
> dependencies other than built-in Python capabilities.  See current NumPy
> discussion for more rationale about staying lightweight, avoiding namespace
> pollution and hidden dependencies/conflicts, etc.
>
>
>
> Here is more about use of Saxon with Pyt
>
>
>
>    - StackOverflow use saxon with python,
>    - https://stackoverflow.com/questions/29443364/use-saxon-with-python
>
>
>
>    - Saxon/C - XML document processing for C/C++, PHP and Python
>    - https://www.saxonica.com/saxon-c/index.xml
>
>
>
> If you want to develop an example python program showing how to use Saxon
> in combination with x3d.py, then am happy to add that to the project.
>
>
>
> Similar approach, different outcome: if you can write a simple Python
> program showing how to use built-in library json.load() then those code
> patterns can be integrated into x3d.py autogeneration.
>
>
>
> No magic here, we’re simply being Pythonic.   8)
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics https://
> faculty.nps.edu/brutzman
>
>
>
> *From:* John Carlson <yottzumm at gmail.com>
> *Sent:* Thursday, November 18, 2021 2:26 PM
> *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>;
> vmarchetti at kshell.com
> *Cc:* X3D Graphics Working Group <x3d at web3d.org>
> *Subject:* Re: [x3d] Ideas for the x3d.py package development - JSON
> support
>
> Another thing we could try in X3DJSAIL and X3DPSAIL is use Saxon's
> capability for reading JSON and converting it.
>
> It might be worth a try. We could add our own functions with/in Saxon/XSLT
> or something.
>
> John
>
> On 11/18/21 16:21, John Carlson wrote
>
> On 11/18/21 09:09, Brutzman, Donald (Don) (CIV) wrote:
>
> Hi John, thanks for your note.
>
>
>
> The X3DJSAIL page has a Conversions section which notes that JSON loading
> is still a TODO item.  How do you recommend we add JSON loading similar to
> other loading?
>
> We can convert JSON to a DOM document, or otherwise change the stylesheet
> to create an X3DLoaderJSON.java
>
>
>
>
>    1. http://www.web3d.org/specifications/java/X3DJSAIL.html#Conversions
>
>
>
> The x3d.py X3DPSAIL page was silent about this, now updated to note need
> for JSON exporter and loader.
>
>
>
>    1. http://www.web3d.org/specifications/java/X3DJSAIL.html#Conversions
>
>
>
> For x3d.py python note that XML, VRML97 and ClassicVRML export are
> provided already.  I think it would be a straightforward matter to add
> export of JSON via print statements.  Shall we write that together?
> Hopefully I can meet next Monday afternoon.
>
> I'm pretty sure I will be available. My brain is toast after a long
> session where I'm struggling with JavaScript on X3DJSONLD.
>
>
>
> For x3d.py python import of JSON, there is a Python built-in standard
> library which might be used.  No doubt special configuration or data
> massaging will be needed.  What do you think?
>
>
>
>
>    1. https://docs.python.org/3/library/json.html
>    <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.python.org%2F3%2Flibrary%2Fjson.html&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939897849%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=uKty3QAZDsalhGeOvD%2Fi44WiFBQi95R5%2BxBMFgRB3II%3D&reserved=0>
>
>
>
> I'm guessing we would take the output from json.load() or json.loads() and
> convert it to a full scenegraph.
>
>
>    1.
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics https://
> faculty.nps.edu/brutzman
>
>
>
> *From:* x3d <x3d-bounces at web3d.org> <x3d-bounces at web3d.org> *On Behalf Of
> *John Carlson
> *Sent:* Wednesday, November 17, 2021 7:10 PM
> *To:* vmarchetti at kshell.com
> *Cc:* X3D Graphics Working Group <x3d at web3d.org> <x3d at web3d.org>
> *Subject:* Re: [x3d] Ideas for the x3d.py package development
>
>
>
> Don,
>
>
>
> Your input on converting schemas or X3DUOM to general purpose programming
> language code, so the schemas are no longer required (for speed) would be
> nice, especially for python.   How do we properly validate X3D-encoded
> files in a fairly performant manner?
>
>
>
> I am familiar with X3DJSAIL, X3DLoaderDOM.java and X3DUOM.   These methods
> are highly successful now.
>
>
>
> I know how to load JSON into a DOM document.
>
>
>
> Where I’m not clear is creating scenes from parsed JSON files (glTF and
> X3D) inside x3d.py and X3DJSAIL.
>
>
>
> Would the preferred approach be calling X3DJSAIL methods inside a
> stylesheet?
>
>
>
> John
>
>
>
> On Wed, Nov 17, 2021 at 1:35 PM John Carlson <yottzumm at gmail.com> wrote:
>
> One thing we should probably work on for python is a fast JSON and/or XML
> validation.   This will probably mean converting schemas to python code,
> like the fastjsonschema package.   We should find out if this package is
> still maintained/upgraded.
>
> My experience with JSON Schema draft 2020-12 is that it’s extremely slow
> with the available python validator.   I hope that something like numba
> will speed it up.  Jeff, can we start looking into speeding up the python
> code we wrote for validation with numba?
>
> Thanks all!
>
> Sent from my iPad
>
> > On Nov 16, 2021, at 11:30 AM, vmarchetti at kshell.com wrote:
> >
> > As a topic for the X3D WG I have some observations and proposals for
> the x3d.py Python package developed by
> > this working groups' members and distributed by the PyPI repository at
> https://pypi.org/project/x3d/
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpypi.org%2Fproject%2Fx3d%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939907808%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=6QXBpsy7mGpNpnGWJ4rlr6YGpPX3ppL9%2BlmTiNonsBw%3D&reserved=0>
> >
> > I base these on experience in using this package in a Jupyter notebook,
> most recently as prepared for the Web3D 2021 tutorial session. The Jupyter
> notebook may be viewed at
> http://nbviewer.org/github/vincentmarchetti/web3d2021_jupyter_tutorial/blob/main/conic_figure.ipynb
> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fnbviewer.org%2Fgithub%2Fvincentmarchetti%2Fweb3d2021_jupyter_tutorial%2Fblob%2Fmain%2Fconic_figure.ipynb&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939907808%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=wR5cbkSOreCPJQDasF%2Flq%2BHNT3T5gbeBSfLOS5RBtMY%3D&reserved=0>
> ; the tutorial session is at
> https://web3d.siggraph.org/program/tutorials-3/
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fweb3d.siggraph.org%2Fprogram%2Ftutorials-3%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939917768%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nNPFzn%2F6dNAo0CcoiJ9V4%2B1hRhR3ABd%2F3mqT%2BiItW6o%3D&reserved=0>
> >
> > Andreas Plesch also used the x3d package in Jupyter in his  work on
> creating X3D from STEP files:
> > See the notebook at
> https://nbviewer.org/github/andreasplesch/OCCToX3D/blob/conda2/notebooks/doc_to_wire_vrml.ipynb
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnbviewer.org%2Fgithub%2Fandreasplesch%2FOCCToX3D%2Fblob%2Fconda2%2Fnotebooks%2Fdoc_to_wire_vrml.ipynb&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939917768%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=FSqP%2BONYXsBINosGtIy66LhypzVLb0D0sUi4ZJEAUlw%3D&reserved=0>
> in which the x3d package is used to generate a Viewpoint node
> >
> > My observations and proposals.
> >
> > 1. The x3d package can, for some cases, emit invalid X3D markup for
> SFString values. As an example,  a MetadataSet node with a name field of
> items'   -- meaning the SFString is a 5 character string, and the last
> character is an apostrophe. The Python script
> evaluate_SFString_XML_encoding.py attached below demonstrates constructing
> this case using x3d package, the resulting XML output is:
> > <MetadataSet name='items''/>
> > which is not valid XML.
> >
> > The markup generated by the xml.etree library, not using the x3d.py
> package is:
> > <MetadataSet name="items'" />
> > which is valid XML
> >
> > 2. I do not see that there is a way to generate the XML or HTML5 markup
> for the few remaining cases where the XML and HTML5 encoding will require
> the use of the XML 'containerField' XML attribute to resolve the
> interpretation of a XML element. One essential case in which this appears
> is an XML MetadataString element as an XML child of an MetadataSet XML
> element, but can be either a value of the "metadata" X3D field or a member
> of the "value" X3D field. I don't believe a modification of the default
> values rules can fully resolve this ambiguity without use of an explicit
> containerField attribute for some cases.
> >
> > 3. A proposed extension of the x3d package is to modify the type
> checking done so that an array created in the popular numpy package (
> https://numpy.org/
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnumpy.org%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C25fa2b400fbf458faa3608d9ad67a156%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637731483939927716%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=89gfZ%2BNXDWFugYp27SHvNy159NSKAnEdAwHvDiscKw4%3D&reserved=0>
> ) can be used as the value of an MFVec3f field. The attached script
> evaluate_numpy_typechecking.py demonstrates a usage of this, in which a
> rank 2 numpy array is used as the value of an MFVec3f for the Coordinate
> node of a IndexedTriangleSet
> >
> >
> > Vince Marchetti
> >
> > _______________________________________________
> > x3d mailing list
> > x3d at web3d.org
> > http://web3d.org/mailman/listinfo/x3d_web3d.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20211122/ddd523e7/attachment-0001.html>