[x3d-public] A few suggestions from the Python API (x3d.py)

Wed Feb 9 22:40:49 PST 2022

Moritz:

Thanks 1M for your close scrutiny and excellent examples.  I particularly like how your presentation of multiple criss-crossing point sources would likely not be discernable if only shown in 2D, and possibly even confusing as a movie portrayal.  User ability to navigate and inspect quickly revealed the nature of intersecting, tightly bound signal paths.  Cool!

Your points regarding Python architecture of x3d.py are helpful.  Am meeting with our resident Python guru on Friday to discuss in detail.  Looks like your proposed bugfix will help.  In general, I think that first-class numpy scientific-stack compatibility is a high priority, and this will be our first chance to pursue that.  Am hopeful that we can first perform duck typing through the X3D type classes (SFVec3f, MFFloat, etc. etc.) and corresponding validation functions (Boolean and assertion).  At that point a little more experimentation will likely help us quickly decide whether further code generation is warranted throughout the individual fields of X3D.  It is further simple to perform corresponding upgrades to the stylesheet converter and all of the online examples.  Simple is good.

Your diagnosis of import and init issues is also welcome, as an unrepentant Java programmer I've always thought that approach useful but a bit gnarly.  No doubt we'll ponder official guidance and various textbooks closely.  We might also find that simple screensharing reveals an easily fixed tangle... onward we go, one step at a time.

We are only building and releasing one version of x3d.py so no worries about possible versionitis woes.  Package is pretty stable but internal patterns can easily evolve. We want to Do The Right Thing which means supporting the Python community, which uhh is measurable.  8)

Documentation can always be improved and indeed can be a helpful confirmation of shared clarity and group understanding.  Please note any question marks or confounders as you go.

Glad that x3d.py is working for you, we strive to be Pythonic!  Will follow up, looking forward to further progress together, good luck with your work.

all the best, Don
--
Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
X3D graphics, virtual worlds, Navy robotics https:// faculty.nps.edu/brutzman

From: x3d-public <x3d-public-bounces at web3d.org> On Behalf Of Hans Moritz Guenther
Sent: Wednesday, February 9, 2022 8:23 AM
To: x3d-public at web3d.org
Subject: [x3d-public] A few suggestions from the Python API (x3d.py)

Hi,

I'm starting to use the x3d.py library to generate x3d output. I'm very much a Python programmer using what people call the "scientific stack" in Python (the libraries numpy, scipy, pandas, etc.) with very little experience in 3D visualization or web-programming.

I understand that you autogenerate the programming language API from the X3DOUM, but I do not know which, if any, of the comments generalize to the APIs in other programming languages or which ones are specific to the Python implementation in x3d.py.

For a little bit of background, I'm an astronomer and one of the things I do is ray-tracing of the designs for new space-based observatories. It is then very useful to have some form of output that allows me to put images of how the instrument looks and what path the photons take into presentations or on the web. As it turns out, X3D is a great format for that, see e.g. https://space.mit.edu/home/guenther/ARCUS/3Dview.html<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspace.mit.edu%2Fhome%2Fguenther%2FARCUS%2F3Dview.html&data=04%7C01%7Cbrutzman%40nps.edu%7C2c7d4294f26747a1ac8308d9ebe89cde%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637800209456879782%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=RRyCO0F1KB8HFPMxhcPAiyPcQsDSJS0OPUjl%2B2JhWYI%3D&reserved=0> In that past, I've generated the X3D output through a python package called mayavi (https://docs.enthought.com/mayavi/mayavi/<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.enthought.com%2Fmayavi%2Fmayavi%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C2c7d4294f26747a1ac8308d9ebe89cde%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637800209456879782%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=4cq4pmXSaLHsQcBlO%2BeXsQBRSd5G6sgNa%2BW5ilD2pyU%3D&reserved=0>) but that is a rather heavy dependency for this very limited purpose. So, I'm trying to convert to use your x3d.py library instead.

First, let me thank you for your effort to make x3d.py at all. In general it works great!

I'd like to mention a few glitches I've run into so far:

- IndexTriangleSet: Coordinates are given as a list of tuples [(x1, y1, z1), (x2, y2, y3), ...]. I think that makes sense. It orders stuff in a natural way and is easy to understand. However, the index needs to be given as a flat list [ind11, ind12, ind13, ind21, ind22, ind23, ...]. Since there are three indices per triangle, I think the same list of tuples as for the coordinates would make more sense [( ind11, ind12, ind13), (ind21, ind22, ind23) ...]. I've not checked if the same thought applied to e.g. the IndexedFaceSet etc. I'm still pretty new to this and can't claim to know all relevant nodes.

- the __init__.py file. The file is present, but not correct. It lists all the classes in __all__ but does not actually import them. Thus, that can't actually be used.

In [1]: import x3d

In [2]: x3d.IndexedTriangleSet
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-2-93a98d5f0414> in <module>
----> 1 x3d.IndexedTriangleSet

Instead, I need to use "x3d.x3d". That's of course possible but a little cumbersome.

In [4]: import x3d.x3d
x3d.py package loaded, have fun with X3D Graphics!

In [5]: x3d.x3d.IndexedTriangleSet
Out[5]: x3d.x3d.IndexedTriangleSet

I think all that's missing is the following line in the __init__.py

from x3d import *

which will import everything that's defined in x3d.py into the __init__.py namespace and than the user an get it from "import x3d" which gives you the names defined in x3d/__init__.py
- Python usually makes heavy use of duck-typing (you can use any object as long as it behaves as you expect) instead of explicit "isinstance" checks. A lot of numerical computations in Python are done using the numpy (https://numpy.org<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnumpy.org%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C2c7d4294f26747a1ac8308d9ebe89cde%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637800209456879782%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WzfgxdbvTKLoZC5Oh87ZfjEzfjOYdFdv4nFBGyP9xFk%3D&reserved=0>) library, which implements array-based math and is optimized in C. Numpy has datatypes that are essentially similar to Python int but fail an isinstance check, and I can pass those in some, but not all places in x3d.py. Here is an example that fails:

~/mambaforge/envs/kitchensink/lib/python3.10/site-packages/x3d/x3d.py in __init__(self, ccw, colorPerVertex, index, normalPerVertex, solid, color, coord, fogCoord, normal, texCoord, attrib, DEF, USE, IS, metadata, class_, id_, style_)

  48191         self.ccw = ccw

  48192         self.colorPerVertex = colorPerVertex

> 48193         self.index = index

  48194         self.normalPerVertex = normalPerVertex

  48195         self.solid = solid

~/mambaforge/envs/kitchensink/lib/python3.10/site-packages/x3d/x3d.py in index(self, index)

  48230         if  index is None:

  48231             index = MFInt32.DEFAULT_VALUE()

> 48232         assertValidMFInt32(index)

  48233         assertNonNegative('index', index)

  48234         self.__index = index

~/mambaforge/envs/kitchensink/lib/python3.10/site-packages/x3d/x3d.py in assertValidMFInt32(value)

   2622         if not isinstance(each, int):

   2623             # print(flush=True)

-> 2624             raise X3DTypeError('MFInt32 list has contained value=' + str(each) + ' with type=' + str(type(each)) + ' which is not a valid int')

   2625     if not isValidMFInt32(value):

   2626         # print(flush=True)

X3DTypeError: MFInt32 list has contained value=1 with type=<class 'numpy.int64'> which is not a valid int

There is an easy workaround:

x3d.IndexedTriangleSet(index = [int(x) for x in myindex]

but it would be nice if IndexedTriangleSet accepted my numbers directly.

There are several ways of doing that, but the easiest is probably to change line 2622 to:

if not int(x) == x:

That will work for any object that can be converted to an int, including numpy, python decimal, fraction, ..

- Numpy: One might consider taking numpy arrays directly, i.e. instead of

x3d.Coordinate(point=[(x1, y1, z1), (x2, y2, y3), ...])

one could do

x3d.Coordinate(point=arr)

where arr is a (3, n) numpy array. Now, if done naively that would require numpy as a dependency to x3d.py and it's probably good to avoid that. However, there are ways to accept numpy arrays without requiring numpy. That's a little more involved, but can be done (for example using decorators or a separate module (x3d.numpy_interface) or separate package (x3d-numpy)). Not sure if it's worth the effort at this point - that depends on what your future plans for this package are and how fast this is developing.

- Changelog. From the pypi entry and the docs on https://www.web3d.org/x3d/stylesheets/python/python.html it was not quite clear to me how stable the package is or where I would see changes listed, for example, if you do the change that I suggested above (accepting index values as tuples instead of a flat list) that would break the interface. Would you do that? Where would I find a list of changes from one version to another?

- Jupyter notebook: The Jupyter notebook seems like an ideal tool for work with X3D in Python, since it is rendered on the web and can display any web output. Here is a simple addition of a `_repr_html_`method to the Scene class in the x3d.py that will render any valid scene with no additional effort to the screen. Sure, the header is a little simplistic, but it's just a quick way to look at what your are specifying. Since I did not want to edit x3d.py itself, I simply made a new class that inherits from the x3d.Scene, but it would obviously be even easier if this was part of x3d itself. See http://nbviewer.org/github/hamogu/x3d-experiements/blob/main/Scence_for_notebook.ipynb<https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fnbviewer.org%2Fgithub%2Fhamogu%2Fx3d-experiements%2Fblob%2Fmain%2FScence_for_notebook.ipynb&data=04%7C01%7Cbrutzman%40nps.edu%7C2c7d4294f26747a1ac8308d9ebe89cde%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637800209456879782%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=hbOIRtm2y3e67fscFuslz8j8keaEeEZz34%2BNVhYcC2U%3D&reserved=0> for an example and note how the X3D output at the bottom is not just a screenshot, but a live output that your can zoom and rotate with your mouse, even though the notebook is not running live, but instead you just see the rendered output of what I run some time in the past on my laptop. (I admit that this is a naive implementation and it might be useful to add a few <meta> or <WorldInfo> nodes. Also, maybe Scene is not the best node, or not the only node, where to define this functionality, but it seems to work well.)
class Scene(x3d.Scene):
    js_source = 'https://www.x3dom.org/download/x3dom.js'
    css_source = 'https://www.x3dom.org/download/x3dom.css'
    dimension_px = (600, 400)

    def _repr_html_(self):

        return(f"""
<html>
  <head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
     <script type='text/javascript' src='{self.js_source}'> </script>
     <link rel='stylesheet' type='text/css' href='{self.css_source}'></link>
  </head>
  <body>
    <x3d width='{self.dimension_px[0]}px' height='{self.dimension_px[1]}px'>
      {self.XML()}
    </x3d>
  </body>
</html>
""")

Please let me know if there is anything I can do to help with this awesome package, that really makes generating X3D from Python so much simpler already.

Yours,
Moritz

--

Hans Moritz Günther

Massachusetts Institute of Technology

Kavli Institute for Astrophysics and Space Research

77 Massachusetts Avenue

NE83-569

Cambridge, MA 02139

hgunther at mit.edu<mailto:hgunther at mit.edu>

https://space.mit.edu/home/guenther/<https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fspace.mit.edu%2Fhome%2Fguenther%2F&data=04%7C01%7Cbrutzman%40nps.edu%7C2c7d4294f26747a1ac8308d9ebe89cde%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637800209456879782%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HqU3N0HBlXFbNy9gvv%2BOKKLoEq2njwGf%2BVxTLSa0gSw%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220210/f32f95df/attachment-0001.html>