[x3d-public] file converter based on pythonocc

[John Carlson]

what I do plan on doing is generating an XHTML package around my
VRMLScript.  Then I will go headless if necessary--But I'm not clear at all
how to get data out of chrome --headless (docs seem to be wrong). There was
a previous email, not sure.

On Mon, Apr 20, 2020 at 6:44 AM John Carlson <yottzumm at gmail.com> wrote:

> I did not "go with" headless chrome.  Node.js yes.  That's different from
> headless chrome.   Chrome is a full browser environment. I currently am
> using your suggestion of pasting VRMLScript into the headed browser.
>
> I am supporting Nashorn, Node.js, Python, Java on server side, XML, JSON,
> EXI, PLY (not done) and STL (not done), and soon (hopefully, still thinking
> of how I want to handle MF nodes), VRMLScript.
>
> Thanks for the link on tessellation, I will look into it.   By chance is
> there a JavaScript tessellator?
>
> John
>
> On Sun, Apr 19, 2020 at 8:15 PM Andreas Plesch <andreasplesch at gmail.com>
> wrote:
>
>> Thanks for the response, see below for more.
>>
>> On Sun, Apr 19, 2020 at 7:29 PM Don Brutzman <brutzman at nps.edu> wrote:
>> >
>> > cooler and cooler!
>> >
>> > On 4/19/2020 1:54 PM, Andreas Plesch wrote:
>> > > I am happy to report that the core OCCT vrml writer works fine with
>> > > pythonocc. It generates more optimized vrml than the pythonocc x3d
>> > > exporter.
>> >
>> > obvious statement: if the vrml exporter works well, it might be a
>> simple variation to copy/refactor the output-producing sections of that
>> code to output XML instead of WRL.
>> >
>> > ... reading on ...
>> >
>> > > I updated https://github.com/andreasplesch/OCCToX3D with example vrml
>> > > output, and used tovrmlx3d to convert to xml which is also available
>> > > in the repo.
>> >
>> > very pretty outputs.  is it using LineProperties to give colored
>> borders along mesh edges?
>>
>> There is an option to generate wireframe output in addition to what
>> they call shaded shape output. The edges are IndexedLineSets but the
>> Appearance does not have a LineProperties. This is something which
>> x3d.py could presumably do in postprocessing.
>>
>> Another goal would be to add better Metadata since it should be
>> possible to get those - probably prettyy easily - with pythonocc from
>> the original source.
>>
>> > > I could add tovrmlx3d to the docker image, and then call it from
>> > > python. Are there other options to convert from vrml2 to xml x3d,
>> > > using python ? x_ite could do it but requires a browser.
>> >
>> > i suspect that Holger knows how to run x_ite headless without a browser.
>>
>> John went with headless chrome which is an option but it seems heavier
>> than using castle tovrmlx3d.
>>
>> >
>> > > Enhancing c++ OCCT itself with a x3d writer, parallel to the vrml
>> > > writer is in principle possible but pretty daunting.
>> >
>> > this is what i was trying to express above... don't rewrite the whole
>> functionality, just look for output-producing code and put a selector on
>> those code blocks for one X3D encoding or another - .wrl for VRML97, .x3dv
>> for ClassicVRML (differs only in header line), then .x3d for XML.
>>
>> Yes but even with this strategy in mind it seems a major effort:
>>
>> mostly in:
>>
>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/Vrml;h=d24d7db215fb0c4f26ac73af52fe68735bf8adee;hb=master
>>
>> but also in:
>>
>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlAPI;h=bacc62597d161a9cb978386b13cb8ba8f072d6e4;hb=master
>>
>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlConverter;h=79507809ac27c8b4ed3674ede2c3198b06b352e1;hb=master\
>>
>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlData;h=18683ffff1dac6a17cd9eef9a3e37b25e642e50b;hb=master
>>
>> This would make x3d available to all OCCT users.
>>
>> Another option is to improve x3d generation by pythonocc:
>>
>>
>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/WebGl/x3dom_renderer.py
>> which uses
>>
>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Tesselator/ShapeTesselator.cpp#L427
>>
>> > > The pythonocc generated vrml and x3d does not include a nice default
>> > > viewpoint. If x3d.py would have a createX3DfromString() method, it may
>> > > be possible to add a viewpoint and then create xml with toXML.
>> >
>> > somebody with better python skills than me should write that parser.
>> shouldn't be hard for a python guru since all of the X3D nodes have Python
>> classes with essentially the same name.
>>
>> Yes, I think that would make x3d.py a lot more useful. In addition a
>> createVRMLfromString() would help with legacy content, and conversion.
>> x_ite has good parsers in js which could serve as a template for
>> python parsers. titanias parsers are probably similar.
>>
>> > > John suggested to use the stylesheet to convert from xml to Python
>> > > code. Saxon is probably the only processor which can handle the
>> > > stylesheet ? There is saxon-c/he which is free and has apparently a
>> > > python wrapper: https://www.saxonica.com/saxon-c/doc/html/saxonc.html
>> >
>> > I'm using Saxon Home Edition jar in all of the ant files for builds and
>> regression testing.  Also embedded in X3DJSAIL.  Even checked in to help
>> builders avoid versionitis.
>> >
>> > Better to find and use a Python package for XSLT 2 (not 1).
>>
>> Agree but there seem to be no native Python packages for XSLT 2. The
>> Python recommendation seems to be to use saxon he. In addition to the
>> java version, there is the c version which has an official Python
>> wrapper. Do you want to give it a quick try ?
>>
>> > We would not want to hardwire such a package into x3d.py since it might
>> collide with other uses.  am trying to fit x3d.py well in the python
>> ecology.
>> >
>> > meanwhile I believe there is a python incantation to check at run time
>> and load such-and-such a package (if necessary) if a specialty method gets
>> invoked.  we might also create a x3d-tools.py package for simplfying
>> addition of commonly used packages (such as XSLT or even pythonocc).
>>
>> pythonocc is big, since it depends on c++ occ.
>>
>> There is a pip package:
>> https://pypi.org/project/pythonocc-core/#description
>> but it has not been updated to 7.4.0, the current version.
>>
>> But there is also a conda package
>> https://anaconda.org/pythonocc/pythonocc-core
>> which is uptodate. Hm, I wonder why this is not used for the binderhub
>> pythonocc docker image. It is compiled from scratch in the Dockerfile,
>> instead. Perhaps the package does not work well.
>>
>> > > Did anybody try saxon-c/he with the python api ? It may work.
>> > >
>> > > If that is succesful, it would be probably possible to eval the
>> > > resulting python code in python and get a x3d document/object. This
>> > > could then be manipulated with x3d.py.
>> >
>> > with enough coercion many things can be forced to work... but
>> cross-language solutions are notoriously difficult to port and maintain.
>> also worth noting is that 'eval()' opens up an endless security hole,
>> usually frowned upon...
>>
>> I do not think there would a lot of coercion sicne there is an
>> official Python API for saxon.
>>
>> >
>> > don't want to sound discouraging!  a build workflow can work even if
>> cross-language.  so those might establish initial capability, but we are
>> likely best served by seeking general Pythonic solutions when
>> possible/practical and then folding them into x3d.py itself.
>>
>> I agree, this should stay Python as much as possible.
>>
>> Cheers,  -Andreas
>>
>> > > So think the plan would be for now:
>> > >
>> > > - create a Dockerfile which includes also view3dscene for tovrmlx3d
>> > > - try Jupyter viola and/or appmode, there is an example with upload
>> of a file
>> > > - use binder to host for now, get a viable service
>> > > - try saxon-c/he python api, by installing in dockerfile and then
>> > > trying it in Jupyter.
>> > >
>> > > -Andreas
>> >
>> > good luck out there!  8)
>> >
>> >
>> > > On Sun, Apr 19, 2020 at 12:35 PM Andreas Plesch <
>> andreasplesch at gmail.com> wrote:
>> > >>
>> > >> OpenCascade is the only realistic, open-source option to digest step
>> > >> format. It is a c++ library, so there is a need for backend/server
>> > >> side processing. The easiest option for that is docker and there is a
>> > >> preconfigured, working docker image with both, a working OpenCascade
>> > >> installation, and the pythonocc wrapper which also adds some
>> > >> functionality such as x3d export. Also, python is faster to iterate
>> > >> than c++, and jupyter gets you even more convenience.
>> > >>
>> > >> I added a small example jupyter notebook which work on top of
>> > >> https://mybinder.org/v2/gh/tpaviot/pythonocc-binderhub/master to
>> > >> https://github.com/andreasplesch/OCCToX3D, and an example x3d
>> > >> conversion as proof of concept.
>> > >>
>> > >> I also discovered that OpenCascade supports generally compilation to
>> > >> wasm using emscripten. So longer term it may be possible to compile
>> > >> the OCC libraries required for loading, triangulation and exporting
>> > >> (to VRML) to wasm/ecmascript, and get a completely client based
>> > >> solution. However, the content typically is large and complex, and
>> > >> there is a lot of computation involved. So in practice it may not
>> work
>> > >> out. A main advantage would be that users would not have to trust a
>> > >> remote server with their valuable content.
>> > >>
>> > >> I think next I will try for a bit if is feasible to use the OCC VRML
>> > >> writer, rather than the relatively simple pythonocc x3d exporter. The
>> > >> python wrapper seems to exist but I do not think there is an example.
>> > >>
>> > >> -Andreas
>> > >>
>> > >> On Sat, Apr 18, 2020 at 9:47 PM John Carlson <yottzumm at gmail.com>
>> wrote:
>> > >>>
>> > >>> X3DJSAIL has
>> www.web3d.org/x3d/stylesheets/java/src/org/web3d/x3d/jsail/X3DLoaderDOM.java
>> which supports DOM scenegraphs.
>> > >>>
>> > >>> Examples of usage are here:
>> > >>>
>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Json.js
>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Nashorn.js
>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/X3d.js
>> > >>>
>> > >>> Basically, the examples parse JSON into a document using
>> X3DJSONLD.java then convert DOM into a X3DJSAIL scenegraph.
>> > >>>
>> > >>> So to answer your createX3DFromString() question, if you start with
>> JSON, I think you could get a X3DJSAIL scenegraph from a JSON string using
>> the X3DJSAIL-like python API (the one we got to work with Jupyter).
>> > >>>
>> > >>> Otherwise, you probably have to wait for enhancements to x3d.py.
>> It looks like toXML is supported in x3d.py currently.  One could use a
>> stylesheet X3dToPython.xslt to convert XML to Python.
>> > >>>
>> > >>> So one could use the stylesheet in a web program (GL with that!) to
>> output python.  Then you would have to use something like eval (ugh!).
>> > >>>
>> > >>> John
>> > >>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> On Sat, Apr 18, 2020 at 5:20 PM Andreas Plesch <
>> andreasplesch at gmail.com> wrote:
>> > >>>>
>> > >>>> Thanks for the interest. I initialized a project at
>> > >>>>
>> > >>>> https://github.com/andreasplesch/OCCToX3D
>> > >>>>
>> > >>>> The wiki has some collected info and a short outline. The wiki is
>> > >>>> editable by anyone.
>> > >>>>
>> > >>>> pythonocc has a method to generate X3D xml. It is a simple Shape
>> with
>> > >>>> a TriangleSet, nothing sophisticated. If x3d.py would have  a
>> > >>>> createX3DFromString() function, it would be possible to use x3d.py
>> to
>> > >>>> enhance the generated X3D. But I do not think x3d.py has that. But
>> > >>>> x3d.py may come in handy in other ways.
>> > >>>>
>> > >>>> The first goal is to add a simple jupyter notebook to the repo
>> based
>> > >>>> on the available pythonocc docker image which demonstrate how to
>> load
>> > >>>> step and output X3D.
>> > >>>>
>> > >>>> Any feedback or ideas welcome, using Issues on the project or here,
>> > >>>>
>> > >>>> -Andreas
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> On Sat, Apr 18, 2020 at 1:27 PM Don Brutzman <brutzman at nps.edu>
>> wrote:
>> > >>>>>
>> > >>>>> Hi Andreas, sounds like a great idea to me.  Here are some
>> peripheral points of support.
>> > >>>>>
>> > >>>>> Last weekend i updated and deployed the x3d.py package (sometimes
>> called X3DPSAIL) at
>> > >>>>>
>> > >>>>> [1]     Python Package Index (PyPi) x3d 0.0.25
>> > >>>>>          Package support for Extensible 3D (X3D) Graphics
>> International Standard (IS)
>> > >>>>>          https://pypi.org/project/x3d/
>> > >>>>>
>> > >>>>> [2]     Python X3D Package x3d.py,X3D Python Scene Access
>> Interface Library (X3DPSAIL)
>> > >>>>>          https://www.web3d.org/x3d/stylesheets/python/python.html
>> > >>>>>
>> > >>>>> "The x3d.py Python X3D Package supports programmers with Python
>> interfaces and objects for standards-based X3D programming, all as open
>> source.
>> > >>>>> The presentation *Python X3D Package Implementation* provides an
>> overview and shows examples."
>> > >>>>>
>> > >>>>> [2.1]   Python X3D Package Implementation
>> > >>>>>
>> https://www.web3d.org/x3d/stylesheets/python/documents/PythonX3dPackageImplementation.pdf
>> > >>>>>
>> > >>>>> So you would use this library for model creation and data-driven
>> improvement, no need to reinvent X3D data structures or methods.
>> > >>>>>
>> > >>>>> This is autogenerated from X3DUOM so it will always stay in sync
>> with latest/greatest X3D4.
>> > >>>>>
>> > >>>>> We've done some preliminary work with Jupyter notebook, listed
>> there.  Lots of potential projects in TODO section.
>> > >>>>>
>> > >>>>> [2.2]   X3DPSAIL: Jupyter notebook, TODO
>> > >>>>>
>> https://www.web3d.org/x3d/stylesheets/python/python.html#Jupyter
>> > >>>>>
>> https://www.web3d.org/x3d/stylesheets/python/python.html#TODO
>> > >>>>>
>> > >>>>> For pythonocc conversion efforts, will be happy to ensure that
>> > >>>>> a. all improvements to API are applied,
>> > >>>>> b. ensure that validation of all scene-graph inputs remains
>> rigorous,
>> > >>>>> b. utility methods are added for easy use together,
>> > >>>>> d. examplar conversion models are properly saved in X3D Examples
>> Archive,
>> > >>>>> e. Whatever else!
>> > >>>>>
>> > >>>>> Looks like there is vast potential here.  Hoping folks start
>> picking it up and applying capabilities.
>> > >>>>>
>> > >>>>> Have fun with X3D Python!  8)
>> > >>>>>
>> > >>>>>
>> > >>>>> On 4/17/2020 5:15 PM, Andreas Plesch wrote:
>> > >>>>>> Did somebody put together an x3d converter based on pythonocc ?
>> > >>>>>>
>> > >>>>>> Here is an example:
>> > >>>>>>
>> > >>>>>> https://senties-martinelli.com/software/cad-viewer
>> > >>>>>>
>> > >>>>>> It could convert  step, stp, igs, iges, obj and stl formats.
>> Format
>> > >>>>>> import and export is only a small subset of pythonocc
>> > >>>>>> (https://github.com/tpaviot/pythonocc-core) but I think just
>> making
>> > >>>>>> this functionality easy to use with a dedicated converter could
>> be
>> > >>>>>> really valuable.
>> > >>>>>>
>> > >>>>>> In the end, I think a  web service just for conversion would be
>> most useful.
>> > >>>>>>
>> > >>>>>> I played with the Jupyter notebook and could generate with only
>> a few lines x3d.
>> > >>>>>>
>> > >>>>>> And there are ways to generate a standalone webapp from a
>> notebook:
>> > >>>>>>
>> https://discourse.jupyter.org/t/how-to-transform-a-jupyter-notebook-into-a-webapp/758/6
>> > >>>>>>
>> > >>>>>> That could be quick way to offer a web service.
>> > >>>>>>
>> > >>>>>> Have there been attempts of this kind ? Would there be interest
>> in a
>> > >>>>>> open source github hosted project ?
>> > >>>>>>
>> > >>>>>> -Andreas
>> > >>>>>
>> > >>>>> all the best, Don
>> > >>>>> --
>> > >>>>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> > >>>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>  +1.831.656.2149
>> > >>>>> X3D graphics, virtual worlds, navy robotics
>> http://faculty.nps.edu/brutzman
>> > >>>>
>> > >>>>
>> > >>>>
>> > >>>> --
>> > >>>> Andreas Plesch
>> > >>>> Waltham, MA 02453
>> > >>>>
>> > >>>> _______________________________________________
>> > >>>> x3d-public mailing list
>> > >>>> x3d-public at web3d.org
>> > >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Andreas Plesch
>> > >> Waltham, MA 02453
>> > >
>> > >
>> > >
>> > > --
>> > > Andreas Plesch
>> > > Waltham, MA 02453
>> > >
>> >
>> > all the best, Don
>> > --
>> > Don Brutzman  Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>  +1.831.656.2149
>> > X3D graphics, virtual worlds, navy robotics
>> http://faculty.nps.edu/brutzman
>>
>>
>>
>> --
>> Andreas Plesch
>> Waltham, MA 02453
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200420/46670ecd/attachment-0001.html>