[x3d-public] file converter based on pythonocc

Andreas Plesch andreasplesch at gmail.com
Sun Apr 19 18:15:03 PDT 2020 

Thanks for the response, see below for more.

On Sun, Apr 19, 2020 at 7:29 PM Don Brutzman <brutzman at nps.edu> wrote:
>
> cooler and cooler!
>
> On 4/19/2020 1:54 PM, Andreas Plesch wrote:
> > I am happy to report that the core OCCT vrml writer works fine with
> > pythonocc. It generates more optimized vrml than the pythonocc x3d
> > exporter.
>
> obvious statement: if the vrml exporter works well, it might be a simple variation to copy/refactor the output-producing sections of that code to output XML instead of WRL.
>
> ... reading on ...
>
> > I updated https://github.com/andreasplesch/OCCToX3D with example vrml
> > output, and used tovrmlx3d to convert to xml which is also available
> > in the repo.
>
> very pretty outputs.  is it using LineProperties to give colored borders along mesh edges?

There is an option to generate wireframe output in addition to what
they call shaded shape output. The edges are IndexedLineSets but the
Appearance does not have a LineProperties. This is something which
x3d.py could presumably do in postprocessing.

Another goal would be to add better Metadata since it should be
possible to get those - probably prettyy easily - with pythonocc from
the original source.

> > I could add tovrmlx3d to the docker image, and then call it from
> > python. Are there other options to convert from vrml2 to xml x3d,
> > using python ? x_ite could do it but requires a browser.
>
> i suspect that Holger knows how to run x_ite headless without a browser.

John went with headless chrome which is an option but it seems heavier
than using castle tovrmlx3d.

>
> > Enhancing c++ OCCT itself with a x3d writer, parallel to the vrml
> > writer is in principle possible but pretty daunting.
>
> this is what i was trying to express above... don't rewrite the whole functionality, just look for output-producing code and put a selector on those code blocks for one X3D encoding or another - .wrl for VRML97, .x3dv for ClassicVRML (differs only in header line), then .x3d for XML.

Yes but even with this strategy in mind it seems a major effort:

mostly in:
http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/Vrml;h=d24d7db215fb0c4f26ac73af52fe68735bf8adee;hb=master

but also in:
http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlAPI;h=bacc62597d161a9cb978386b13cb8ba8f072d6e4;hb=master
http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlConverter;h=79507809ac27c8b4ed3674ede2c3198b06b352e1;hb=master\
http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlData;h=18683ffff1dac6a17cd9eef9a3e37b25e642e50b;hb=master

This would make x3d available to all OCCT users.

Another option is to improve x3d generation by pythonocc:

https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/WebGl/x3dom_renderer.py
which uses
https://github.com/tpaviot/pythonocc-core/blob/master/src/Tesselator/ShapeTesselator.cpp#L427

> > The pythonocc generated vrml and x3d does not include a nice default
> > viewpoint. If x3d.py would have a createX3DfromString() method, it may
> > be possible to add a viewpoint and then create xml with toXML.
>
> somebody with better python skills than me should write that parser.  shouldn't be hard for a python guru since all of the X3D nodes have Python classes with essentially the same name.

Yes, I think that would make x3d.py a lot more useful. In addition a
createVRMLfromString() would help with legacy content, and conversion.
x_ite has good parsers in js which could serve as a template for
python parsers. titanias parsers are probably similar.

> > John suggested to use the stylesheet to convert from xml to Python
> > code. Saxon is probably the only processor which can handle the
> > stylesheet ? There is saxon-c/he which is free and has apparently a
> > python wrapper: https://www.saxonica.com/saxon-c/doc/html/saxonc.html
>
> I'm using Saxon Home Edition jar in all of the ant files for builds and regression testing.  Also embedded in X3DJSAIL.  Even checked in to help builders avoid versionitis.
>
> Better to find and use a Python package for XSLT 2 (not 1).

Agree but there seem to be no native Python packages for XSLT 2. The
Python recommendation seems to be to use saxon he. In addition to the
java version, there is the c version which has an official Python
wrapper. Do you want to give it a quick try ?

> We would not want to hardwire such a package into x3d.py since it might collide with other uses.  am trying to fit x3d.py well in the python ecology.
>
> meanwhile I believe there is a python incantation to check at run time and load such-and-such a package (if necessary) if a specialty method gets invoked.  we might also create a x3d-tools.py package for simplfying addition of commonly used packages (such as XSLT or even pythonocc).

pythonocc is big, since it depends on c++ occ.

There is a pip package:
https://pypi.org/project/pythonocc-core/#description
but it has not been updated to 7.4.0, the current version.

But there is also a conda package
https://anaconda.org/pythonocc/pythonocc-core
which is uptodate. Hm, I wonder why this is not used for the binderhub
pythonocc docker image. It is compiled from scratch in the Dockerfile,
instead. Perhaps the package does not work well.

> > Did anybody try saxon-c/he with the python api ? It may work.
> >
> > If that is succesful, it would be probably possible to eval the
> > resulting python code in python and get a x3d document/object. This
> > could then be manipulated with x3d.py.
>
> with enough coercion many things can be forced to work... but cross-language solutions are notoriously difficult to port and maintain.  also worth noting is that 'eval()' opens up an endless security hole, usually frowned upon...

I do not think there would a lot of coercion sicne there is an
official Python API for saxon.

>
> don't want to sound discouraging!  a build workflow can work even if cross-language.  so those might establish initial capability, but we are likely best served by seeking general Pythonic solutions when possible/practical and then folding them into x3d.py itself.

I agree, this should stay Python as much as possible.

Cheers,  -Andreas

> > So think the plan would be for now:
> >
> > - create a Dockerfile which includes also view3dscene for tovrmlx3d
> > - try Jupyter viola and/or appmode, there is an example with upload of a file
> > - use binder to host for now, get a viable service
> > - try saxon-c/he python api, by installing in dockerfile and then
> > trying it in Jupyter.
> >
> > -Andreas
>
> good luck out there!  8)
>
>
> > On Sun, Apr 19, 2020 at 12:35 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
> >>
> >> OpenCascade is the only realistic, open-source option to digest step
> >> format. It is a c++ library, so there is a need for backend/server
> >> side processing. The easiest option for that is docker and there is a
> >> preconfigured, working docker image with both, a working OpenCascade
> >> installation, and the pythonocc wrapper which also adds some
> >> functionality such as x3d export. Also, python is faster to iterate
> >> than c++, and jupyter gets you even more convenience.
> >>
> >> I added a small example jupyter notebook which work on top of
> >> https://mybinder.org/v2/gh/tpaviot/pythonocc-binderhub/master to
> >> https://github.com/andreasplesch/OCCToX3D, and an example x3d
> >> conversion as proof of concept.
> >>
> >> I also discovered that OpenCascade supports generally compilation to
> >> wasm using emscripten. So longer term it may be possible to compile
> >> the OCC libraries required for loading, triangulation and exporting
> >> (to VRML) to wasm/ecmascript, and get a completely client based
> >> solution. However, the content typically is large and complex, and
> >> there is a lot of computation involved. So in practice it may not work
> >> out. A main advantage would be that users would not have to trust a
> >> remote server with their valuable content.
> >>
> >> I think next I will try for a bit if is feasible to use the OCC VRML
> >> writer, rather than the relatively simple pythonocc x3d exporter. The
> >> python wrapper seems to exist but I do not think there is an example.
> >>
> >> -Andreas
> >>
> >> On Sat, Apr 18, 2020 at 9:47 PM John Carlson <yottzumm at gmail.com> wrote:
> >>>
> >>> X3DJSAIL has www.web3d.org/x3d/stylesheets/java/src/org/web3d/x3d/jsail/X3DLoaderDOM.java which supports DOM scenegraphs.
> >>>
> >>> Examples of usage are here:
> >>>
> >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Json.js
> >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Nashorn.js
> >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/X3d.js
> >>>
> >>> Basically, the examples parse JSON into a document using  X3DJSONLD.java then convert DOM into a X3DJSAIL scenegraph.
> >>>
> >>> So to answer your createX3DFromString() question, if you start with JSON, I think you could get a X3DJSAIL scenegraph from a JSON string using the X3DJSAIL-like python API (the one we got to work with Jupyter).
> >>>
> >>> Otherwise, you probably have to wait for enhancements to x3d.py.  It looks like toXML is supported in x3d.py currently.  One could use a stylesheet X3dToPython.xslt to convert XML to Python.
> >>>
> >>> So one could use the stylesheet in a web program (GL with that!) to output python.  Then you would have to use something like eval (ugh!).
> >>>
> >>> John
> >>>
> >>>
> >>>
> >>>
> >>> On Sat, Apr 18, 2020 at 5:20 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
> >>>>
> >>>> Thanks for the interest. I initialized a project at
> >>>>
> >>>> https://github.com/andreasplesch/OCCToX3D
> >>>>
> >>>> The wiki has some collected info and a short outline. The wiki is
> >>>> editable by anyone.
> >>>>
> >>>> pythonocc has a method to generate X3D xml. It is a simple Shape with
> >>>> a TriangleSet, nothing sophisticated. If x3d.py would have  a
> >>>> createX3DFromString() function, it would be possible to use x3d.py to
> >>>> enhance the generated X3D. But I do not think x3d.py has that. But
> >>>> x3d.py may come in handy in other ways.
> >>>>
> >>>> The first goal is to add a simple jupyter notebook to the repo based
> >>>> on the available pythonocc docker image which demonstrate how to load
> >>>> step and output X3D.
> >>>>
> >>>> Any feedback or ideas welcome, using Issues on the project or here,
> >>>>
> >>>> -Andreas
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> On Sat, Apr 18, 2020 at 1:27 PM Don Brutzman <brutzman at nps.edu> wrote:
> >>>>>
> >>>>> Hi Andreas, sounds like a great idea to me.  Here are some peripheral points of support.
> >>>>>
> >>>>> Last weekend i updated and deployed the x3d.py package (sometimes called X3DPSAIL) at
> >>>>>
> >>>>> [1]     Python Package Index (PyPi) x3d 0.0.25
> >>>>>          Package support for Extensible 3D (X3D) Graphics International Standard (IS)
> >>>>>          https://pypi.org/project/x3d/
> >>>>>
> >>>>> [2]     Python X3D Package x3d.py,X3D Python Scene Access Interface Library (X3DPSAIL)
> >>>>>          https://www.web3d.org/x3d/stylesheets/python/python.html
> >>>>>
> >>>>> "The x3d.py Python X3D Package supports programmers with Python interfaces and objects for standards-based X3D programming, all as open source.
> >>>>> The presentation *Python X3D Package Implementation* provides an overview and shows examples."
> >>>>>
> >>>>> [2.1]   Python X3D Package Implementation
> >>>>>          https://www.web3d.org/x3d/stylesheets/python/documents/PythonX3dPackageImplementation.pdf
> >>>>>
> >>>>> So you would use this library for model creation and data-driven improvement, no need to reinvent X3D data structures or methods.
> >>>>>
> >>>>> This is autogenerated from X3DUOM so it will always stay in sync with latest/greatest X3D4.
> >>>>>
> >>>>> We've done some preliminary work with Jupyter notebook, listed there.  Lots of potential projects in TODO section.
> >>>>>
> >>>>> [2.2]   X3DPSAIL: Jupyter notebook, TODO
> >>>>>          https://www.web3d.org/x3d/stylesheets/python/python.html#Jupyter
> >>>>>          https://www.web3d.org/x3d/stylesheets/python/python.html#TODO
> >>>>>
> >>>>> For pythonocc conversion efforts, will be happy to ensure that
> >>>>> a. all improvements to API are applied,
> >>>>> b. ensure that validation of all scene-graph inputs remains rigorous,
> >>>>> b. utility methods are added for easy use together,
> >>>>> d. examplar conversion models are properly saved in X3D Examples Archive,
> >>>>> e. Whatever else!
> >>>>>
> >>>>> Looks like there is vast potential here.  Hoping folks start picking it up and applying capabilities.
> >>>>>
> >>>>> Have fun with X3D Python!  8)
> >>>>>
> >>>>>
> >>>>> On 4/17/2020 5:15 PM, Andreas Plesch wrote:
> >>>>>> Did somebody put together an x3d converter based on pythonocc ?
> >>>>>>
> >>>>>> Here is an example:
> >>>>>>
> >>>>>> https://senties-martinelli.com/software/cad-viewer
> >>>>>>
> >>>>>> It could convert  step, stp, igs, iges, obj and stl formats. Format
> >>>>>> import and export is only a small subset of pythonocc
> >>>>>> (https://github.com/tpaviot/pythonocc-core) but I think just making
> >>>>>> this functionality easy to use with a dedicated converter could be
> >>>>>> really valuable.
> >>>>>>
> >>>>>> In the end, I think a  web service just for conversion would be most useful.
> >>>>>>
> >>>>>> I played with the Jupyter notebook and could generate with only a few lines x3d.
> >>>>>>
> >>>>>> And there are ways to generate a standalone webapp from a notebook:
> >>>>>> https://discourse.jupyter.org/t/how-to-transform-a-jupyter-notebook-into-a-webapp/758/6
> >>>>>>
> >>>>>> That could be quick way to offer a web service.
> >>>>>>
> >>>>>> Have there been attempts of this kind ? Would there be interest in a
> >>>>>> open source github hosted project ?
> >>>>>>
> >>>>>> -Andreas
> >>>>>
> >>>>> all the best, Don
> >>>>> --
> >>>>> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> >>>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> >>>>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Andreas Plesch
> >>>> Waltham, MA 02453
> >>>>
> >>>> _______________________________________________
> >>>> x3d-public mailing list
> >>>> x3d-public at web3d.org
> >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> >>
> >>
> >>
> >> --
> >> Andreas Plesch
> >> Waltham, MA 02453
> >
> >
> >
> > --
> > Andreas Plesch
> > Waltham, MA 02453
> >
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman



-- 
Andreas Plesch
Waltham, MA 02453

More information about the x3d-public mailing list