[x3d-public] file converter based on pythonocc

John Carlson yottzumm at gmail.com
Mon Apr 20 04:57:22 PDT 2020 

One can get a snapshot out of chrome --headless, or perhaps use puppeteer,
but I'm not pursuing that until I have an X_ITE SAI HelloWorld file example
that passes through the browser in normal mode.

I spent the last couple of days polishing up X3DJSONLD and pythonSAI.

John

On Mon, Apr 20, 2020 at 6:48 AM John Carlson <yottzumm at gmail.com> wrote:

> what I do plan on doing is generating an XHTML package around my
> VRMLScript.  Then I will go headless if necessary--But I'm not clear at all
> how to get data out of chrome --headless (docs seem to be wrong). There was
> a previous email, not sure.
>
> On Mon, Apr 20, 2020 at 6:44 AM John Carlson <yottzumm at gmail.com> wrote:
>
>> I did not "go with" headless chrome.  Node.js yes.  That's different from
>> headless chrome.   Chrome is a full browser environment. I currently am
>> using your suggestion of pasting VRMLScript into the headed browser.
>>
>> I am supporting Nashorn, Node.js, Python, Java on server side, XML, JSON,
>> EXI, PLY (not done) and STL (not done), and soon (hopefully, still thinking
>> of how I want to handle MF nodes), VRMLScript.
>>
>> Thanks for the link on tessellation, I will look into it.   By chance is
>> there a JavaScript tessellator?
>>
>> John
>>
>> On Sun, Apr 19, 2020 at 8:15 PM Andreas Plesch <andreasplesch at gmail.com>
>> wrote:
>>
>>> Thanks for the response, see below for more.
>>>
>>> On Sun, Apr 19, 2020 at 7:29 PM Don Brutzman <brutzman at nps.edu> wrote:
>>> >
>>> > cooler and cooler!
>>> >
>>> > On 4/19/2020 1:54 PM, Andreas Plesch wrote:
>>> > > I am happy to report that the core OCCT vrml writer works fine with
>>> > > pythonocc. It generates more optimized vrml than the pythonocc x3d
>>> > > exporter.
>>> >
>>> > obvious statement: if the vrml exporter works well, it might be a
>>> simple variation to copy/refactor the output-producing sections of that
>>> code to output XML instead of WRL.
>>> >
>>> > ... reading on ...
>>> >
>>> > > I updated https://github.com/andreasplesch/OCCToX3D with example
>>> vrml
>>> > > output, and used tovrmlx3d to convert to xml which is also available
>>> > > in the repo.
>>> >
>>> > very pretty outputs.  is it using LineProperties to give colored
>>> borders along mesh edges?
>>>
>>> There is an option to generate wireframe output in addition to what
>>> they call shaded shape output. The edges are IndexedLineSets but the
>>> Appearance does not have a LineProperties. This is something which
>>> x3d.py could presumably do in postprocessing.
>>>
>>> Another goal would be to add better Metadata since it should be
>>> possible to get those - probably prettyy easily - with pythonocc from
>>> the original source.
>>>
>>> > > I could add tovrmlx3d to the docker image, and then call it from
>>> > > python. Are there other options to convert from vrml2 to xml x3d,
>>> > > using python ? x_ite could do it but requires a browser.
>>> >
>>> > i suspect that Holger knows how to run x_ite headless without a
>>> browser.
>>>
>>> John went with headless chrome which is an option but it seems heavier
>>> than using castle tovrmlx3d.
>>>
>>> >
>>> > > Enhancing c++ OCCT itself with a x3d writer, parallel to the vrml
>>> > > writer is in principle possible but pretty daunting.
>>> >
>>> > this is what i was trying to express above... don't rewrite the whole
>>> functionality, just look for output-producing code and put a selector on
>>> those code blocks for one X3D encoding or another - .wrl for VRML97, .x3dv
>>> for ClassicVRML (differs only in header line), then .x3d for XML.
>>>
>>> Yes but even with this strategy in mind it seems a major effort:
>>>
>>> mostly in:
>>>
>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/Vrml;h=d24d7db215fb0c4f26ac73af52fe68735bf8adee;hb=master
>>>
>>> but also in:
>>>
>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlAPI;h=bacc62597d161a9cb978386b13cb8ba8f072d6e4;hb=master
>>>
>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlConverter;h=79507809ac27c8b4ed3674ede2c3198b06b352e1;hb=master\
>>>
>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlData;h=18683ffff1dac6a17cd9eef9a3e37b25e642e50b;hb=master
>>>
>>> This would make x3d available to all OCCT users.
>>>
>>> Another option is to improve x3d generation by pythonocc:
>>>
>>>
>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/WebGl/x3dom_renderer.py
>>> which uses
>>>
>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Tesselator/ShapeTesselator.cpp#L427
>>>
>>> > > The pythonocc generated vrml and x3d does not include a nice default
>>> > > viewpoint. If x3d.py would have a createX3DfromString() method, it
>>> may
>>> > > be possible to add a viewpoint and then create xml with toXML.
>>> >
>>> > somebody with better python skills than me should write that parser.
>>> shouldn't be hard for a python guru since all of the X3D nodes have Python
>>> classes with essentially the same name.
>>>
>>> Yes, I think that would make x3d.py a lot more useful. In addition a
>>> createVRMLfromString() would help with legacy content, and conversion.
>>> x_ite has good parsers in js which could serve as a template for
>>> python parsers. titanias parsers are probably similar.
>>>
>>> > > John suggested to use the stylesheet to convert from xml to Python
>>> > > code. Saxon is probably the only processor which can handle the
>>> > > stylesheet ? There is saxon-c/he which is free and has apparently a
>>> > > python wrapper:
>>> https://www.saxonica.com/saxon-c/doc/html/saxonc.html
>>> >
>>> > I'm using Saxon Home Edition jar in all of the ant files for builds
>>> and regression testing.  Also embedded in X3DJSAIL.  Even checked in to
>>> help builders avoid versionitis.
>>> >
>>> > Better to find and use a Python package for XSLT 2 (not 1).
>>>
>>> Agree but there seem to be no native Python packages for XSLT 2. The
>>> Python recommendation seems to be to use saxon he. In addition to the
>>> java version, there is the c version which has an official Python
>>> wrapper. Do you want to give it a quick try ?
>>>
>>> > We would not want to hardwire such a package into x3d.py since it
>>> might collide with other uses.  am trying to fit x3d.py well in the python
>>> ecology.
>>> >
>>> > meanwhile I believe there is a python incantation to check at run time
>>> and load such-and-such a package (if necessary) if a specialty method gets
>>> invoked.  we might also create a x3d-tools.py package for simplfying
>>> addition of commonly used packages (such as XSLT or even pythonocc).
>>>
>>> pythonocc is big, since it depends on c++ occ.
>>>
>>> There is a pip package:
>>> https://pypi.org/project/pythonocc-core/#description
>>> but it has not been updated to 7.4.0, the current version.
>>>
>>> But there is also a conda package
>>> https://anaconda.org/pythonocc/pythonocc-core
>>> which is uptodate. Hm, I wonder why this is not used for the binderhub
>>> pythonocc docker image. It is compiled from scratch in the Dockerfile,
>>> instead. Perhaps the package does not work well.
>>>
>>> > > Did anybody try saxon-c/he with the python api ? It may work.
>>> > >
>>> > > If that is succesful, it would be probably possible to eval the
>>> > > resulting python code in python and get a x3d document/object. This
>>> > > could then be manipulated with x3d.py.
>>> >
>>> > with enough coercion many things can be forced to work... but
>>> cross-language solutions are notoriously difficult to port and maintain.
>>> also worth noting is that 'eval()' opens up an endless security hole,
>>> usually frowned upon...
>>>
>>> I do not think there would a lot of coercion sicne there is an
>>> official Python API for saxon.
>>>
>>> >
>>> > don't want to sound discouraging!  a build workflow can work even if
>>> cross-language.  so those might establish initial capability, but we are
>>> likely best served by seeking general Pythonic solutions when
>>> possible/practical and then folding them into x3d.py itself.
>>>
>>> I agree, this should stay Python as much as possible.
>>>
>>> Cheers,  -Andreas
>>>
>>> > > So think the plan would be for now:
>>> > >
>>> > > - create a Dockerfile which includes also view3dscene for tovrmlx3d
>>> > > - try Jupyter viola and/or appmode, there is an example with upload
>>> of a file
>>> > > - use binder to host for now, get a viable service
>>> > > - try saxon-c/he python api, by installing in dockerfile and then
>>> > > trying it in Jupyter.
>>> > >
>>> > > -Andreas
>>> >
>>> > good luck out there!  8)
>>> >
>>> >
>>> > > On Sun, Apr 19, 2020 at 12:35 PM Andreas Plesch <
>>> andreasplesch at gmail.com> wrote:
>>> > >>
>>> > >> OpenCascade is the only realistic, open-source option to digest step
>>> > >> format. It is a c++ library, so there is a need for backend/server
>>> > >> side processing. The easiest option for that is docker and there is
>>> a
>>> > >> preconfigured, working docker image with both, a working OpenCascade
>>> > >> installation, and the pythonocc wrapper which also adds some
>>> > >> functionality such as x3d export. Also, python is faster to iterate
>>> > >> than c++, and jupyter gets you even more convenience.
>>> > >>
>>> > >> I added a small example jupyter notebook which work on top of
>>> > >> https://mybinder.org/v2/gh/tpaviot/pythonocc-binderhub/master to
>>> > >> https://github.com/andreasplesch/OCCToX3D, and an example x3d
>>> > >> conversion as proof of concept.
>>> > >>
>>> > >> I also discovered that OpenCascade supports generally compilation to
>>> > >> wasm using emscripten. So longer term it may be possible to compile
>>> > >> the OCC libraries required for loading, triangulation and exporting
>>> > >> (to VRML) to wasm/ecmascript, and get a completely client based
>>> > >> solution. However, the content typically is large and complex, and
>>> > >> there is a lot of computation involved. So in practice it may not
>>> work
>>> > >> out. A main advantage would be that users would not have to trust a
>>> > >> remote server with their valuable content.
>>> > >>
>>> > >> I think next I will try for a bit if is feasible to use the OCC VRML
>>> > >> writer, rather than the relatively simple pythonocc x3d exporter.
>>> The
>>> > >> python wrapper seems to exist but I do not think there is an
>>> example.
>>> > >>
>>> > >> -Andreas
>>> > >>
>>> > >> On Sat, Apr 18, 2020 at 9:47 PM John Carlson <yottzumm at gmail.com>
>>> wrote:
>>> > >>>
>>> > >>> X3DJSAIL has
>>> www.web3d.org/x3d/stylesheets/java/src/org/web3d/x3d/jsail/X3DLoaderDOM.java
>>> which supports DOM scenegraphs.
>>> > >>>
>>> > >>> Examples of usage are here:
>>> > >>>
>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Json.js
>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Nashorn.js
>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/X3d.js
>>> > >>>
>>> > >>> Basically, the examples parse JSON into a document using
>>> X3DJSONLD.java then convert DOM into a X3DJSAIL scenegraph.
>>> > >>>
>>> > >>> So to answer your createX3DFromString() question, if you start
>>> with JSON, I think you could get a X3DJSAIL scenegraph from a JSON string
>>> using the X3DJSAIL-like python API (the one we got to work with Jupyter).
>>> > >>>
>>> > >>> Otherwise, you probably have to wait for enhancements to x3d.py.
>>> It looks like toXML is supported in x3d.py currently.  One could use a
>>> stylesheet X3dToPython.xslt to convert XML to Python.
>>> > >>>
>>> > >>> So one could use the stylesheet in a web program (GL with that!)
>>> to output python.  Then you would have to use something like eval (ugh!).
>>> > >>>
>>> > >>> John
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>>
>>> > >>> On Sat, Apr 18, 2020 at 5:20 PM Andreas Plesch <
>>> andreasplesch at gmail.com> wrote:
>>> > >>>>
>>> > >>>> Thanks for the interest. I initialized a project at
>>> > >>>>
>>> > >>>> https://github.com/andreasplesch/OCCToX3D
>>> > >>>>
>>> > >>>> The wiki has some collected info and a short outline. The wiki is
>>> > >>>> editable by anyone.
>>> > >>>>
>>> > >>>> pythonocc has a method to generate X3D xml. It is a simple Shape
>>> with
>>> > >>>> a TriangleSet, nothing sophisticated. If x3d.py would have  a
>>> > >>>> createX3DFromString() function, it would be possible to use
>>> x3d.py to
>>> > >>>> enhance the generated X3D. But I do not think x3d.py has that. But
>>> > >>>> x3d.py may come in handy in other ways.
>>> > >>>>
>>> > >>>> The first goal is to add a simple jupyter notebook to the repo
>>> based
>>> > >>>> on the available pythonocc docker image which demonstrate how to
>>> load
>>> > >>>> step and output X3D.
>>> > >>>>
>>> > >>>> Any feedback or ideas welcome, using Issues on the project or
>>> here,
>>> > >>>>
>>> > >>>> -Andreas
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> On Sat, Apr 18, 2020 at 1:27 PM Don Brutzman <brutzman at nps.edu>
>>> wrote:
>>> > >>>>>
>>> > >>>>> Hi Andreas, sounds like a great idea to me.  Here are some
>>> peripheral points of support.
>>> > >>>>>
>>> > >>>>> Last weekend i updated and deployed the x3d.py package
>>> (sometimes called X3DPSAIL) at
>>> > >>>>>
>>> > >>>>> [1]     Python Package Index (PyPi) x3d 0.0.25
>>> > >>>>>          Package support for Extensible 3D (X3D) Graphics
>>> International Standard (IS)
>>> > >>>>>          https://pypi.org/project/x3d/
>>> > >>>>>
>>> > >>>>> [2]     Python X3D Package x3d.py,X3D Python Scene Access
>>> Interface Library (X3DPSAIL)
>>> > >>>>>
>>> https://www.web3d.org/x3d/stylesheets/python/python.html
>>> > >>>>>
>>> > >>>>> "The x3d.py Python X3D Package supports programmers with Python
>>> interfaces and objects for standards-based X3D programming, all as open
>>> source.
>>> > >>>>> The presentation *Python X3D Package Implementation* provides an
>>> overview and shows examples."
>>> > >>>>>
>>> > >>>>> [2.1]   Python X3D Package Implementation
>>> > >>>>>
>>> https://www.web3d.org/x3d/stylesheets/python/documents/PythonX3dPackageImplementation.pdf
>>> > >>>>>
>>> > >>>>> So you would use this library for model creation and data-driven
>>> improvement, no need to reinvent X3D data structures or methods.
>>> > >>>>>
>>> > >>>>> This is autogenerated from X3DUOM so it will always stay in sync
>>> with latest/greatest X3D4.
>>> > >>>>>
>>> > >>>>> We've done some preliminary work with Jupyter notebook, listed
>>> there.  Lots of potential projects in TODO section.
>>> > >>>>>
>>> > >>>>> [2.2]   X3DPSAIL: Jupyter notebook, TODO
>>> > >>>>>
>>> https://www.web3d.org/x3d/stylesheets/python/python.html#Jupyter
>>> > >>>>>
>>> https://www.web3d.org/x3d/stylesheets/python/python.html#TODO
>>> > >>>>>
>>> > >>>>> For pythonocc conversion efforts, will be happy to ensure that
>>> > >>>>> a. all improvements to API are applied,
>>> > >>>>> b. ensure that validation of all scene-graph inputs remains
>>> rigorous,
>>> > >>>>> b. utility methods are added for easy use together,
>>> > >>>>> d. examplar conversion models are properly saved in X3D Examples
>>> Archive,
>>> > >>>>> e. Whatever else!
>>> > >>>>>
>>> > >>>>> Looks like there is vast potential here.  Hoping folks start
>>> picking it up and applying capabilities.
>>> > >>>>>
>>> > >>>>> Have fun with X3D Python!  8)
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On 4/17/2020 5:15 PM, Andreas Plesch wrote:
>>> > >>>>>> Did somebody put together an x3d converter based on pythonocc ?
>>> > >>>>>>
>>> > >>>>>> Here is an example:
>>> > >>>>>>
>>> > >>>>>> https://senties-martinelli.com/software/cad-viewer
>>> > >>>>>>
>>> > >>>>>> It could convert  step, stp, igs, iges, obj and stl formats.
>>> Format
>>> > >>>>>> import and export is only a small subset of pythonocc
>>> > >>>>>> (https://github.com/tpaviot/pythonocc-core) but I think just
>>> making
>>> > >>>>>> this functionality easy to use with a dedicated converter could
>>> be
>>> > >>>>>> really valuable.
>>> > >>>>>>
>>> > >>>>>> In the end, I think a  web service just for conversion would be
>>> most useful.
>>> > >>>>>>
>>> > >>>>>> I played with the Jupyter notebook and could generate with only
>>> a few lines x3d.
>>> > >>>>>>
>>> > >>>>>> And there are ways to generate a standalone webapp from a
>>> notebook:
>>> > >>>>>>
>>> https://discourse.jupyter.org/t/how-to-transform-a-jupyter-notebook-into-a-webapp/758/6
>>> > >>>>>>
>>> > >>>>>> That could be quick way to offer a web service.
>>> > >>>>>>
>>> > >>>>>> Have there been attempts of this kind ? Would there be interest
>>> in a
>>> > >>>>>> open source github hosted project ?
>>> > >>>>>>
>>> > >>>>>> -Andreas
>>> > >>>>>
>>> > >>>>> all the best, Don
>>> > >>>>> --
>>> > >>>>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>> > >>>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>>  +1.831.656.2149
>>> > >>>>> X3D graphics, virtual worlds, navy robotics
>>> http://faculty.nps.edu/brutzman
>>> > >>>>
>>> > >>>>
>>> > >>>>
>>> > >>>> --
>>> > >>>> Andreas Plesch
>>> > >>>> Waltham, MA 02453
>>> > >>>>
>>> > >>>> _______________________________________________
>>> > >>>> x3d-public mailing list
>>> > >>>> x3d-public at web3d.org
>>> > >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>> > >>
>>> > >>
>>> > >>
>>> > >> --
>>> > >> Andreas Plesch
>>> > >> Waltham, MA 02453
>>> > >
>>> > >
>>> > >
>>> > > --
>>> > > Andreas Plesch
>>> > > Waltham, MA 02453
>>> > >
>>> >
>>> > all the best, Don
>>> > --
>>> > Don Brutzman  Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>>  +1.831.656.2149
>>> > X3D graphics, virtual worlds, navy robotics
>>> http://faculty.nps.edu/brutzman
>>>
>>>
>>>
>>> --
>>> Andreas Plesch
>>> Waltham, MA 02453
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200420/02fdecee/attachment-0001.html>

More information about the x3d-public mailing list