[x3d-public] file converter based on pythonocc

John Carlson yottzumm at gmail.com
Mon Apr 20 05:48:55 PDT 2020 

While you are at it, add an X3D JSON -> X3D XML C++-based converter.  I
have the starts of one using Xerces/Gason.  It works with one or two files
right now.
http://web3d.org/pipermail/x3d-public_web3d.org/2018-March/008344.html

https://github.com/coderextreme/X3DJSONLD/tree/master/src/main/cplusplus

https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/cplusplus/src/X3DJSONLD.cpp


Thanks a bunch for including this.  This may help browser vendors adapt
their code to X3DJSON.



On Mon, Apr 20, 2020 at 7:00 AM John Carlson <yottzumm at gmail.com> wrote:

> If someone knows a browser testing package that will traverse a collection
> of pages, let me know.
>
> Maybe puppeteer or CEFpython?   We were talking about using python.  I
> just got CEFpython working the other day for our RPG product.
>
> Will be very helpful!
>
> John
>
> On Mon, Apr 20, 2020 at 6:57 AM John Carlson <yottzumm at gmail.com> wrote:
>
>> One can get a snapshot out of chrome --headless, or perhaps use
>> puppeteer, but I'm not pursuing that until I have an X_ITE SAI HelloWorld
>> file example that passes through the browser in normal mode.
>>
>> I spent the last couple of days polishing up X3DJSONLD and pythonSAI.
>>
>> John
>>
>> On Mon, Apr 20, 2020 at 6:48 AM John Carlson <yottzumm at gmail.com> wrote:
>>
>>> what I do plan on doing is generating an XHTML package around my
>>> VRMLScript.  Then I will go headless if necessary--But I'm not clear at all
>>> how to get data out of chrome --headless (docs seem to be wrong). There was
>>> a previous email, not sure.
>>>
>>> On Mon, Apr 20, 2020 at 6:44 AM John Carlson <yottzumm at gmail.com> wrote:
>>>
>>>> I did not "go with" headless chrome.  Node.js yes.  That's different
>>>> from headless chrome.   Chrome is a full browser environment. I currently
>>>> am using your suggestion of pasting VRMLScript into the headed browser.
>>>>
>>>> I am supporting Nashorn, Node.js, Python, Java on server side, XML,
>>>> JSON, EXI, PLY (not done) and STL (not done), and soon (hopefully, still
>>>> thinking of how I want to handle MF nodes), VRMLScript.
>>>>
>>>> Thanks for the link on tessellation, I will look into it.   By chance
>>>> is there a JavaScript tessellator?
>>>>
>>>> John
>>>>
>>>> On Sun, Apr 19, 2020 at 8:15 PM Andreas Plesch <andreasplesch at gmail.com>
>>>> wrote:
>>>>
>>>>> Thanks for the response, see below for more.
>>>>>
>>>>> On Sun, Apr 19, 2020 at 7:29 PM Don Brutzman <brutzman at nps.edu> wrote:
>>>>> >
>>>>> > cooler and cooler!
>>>>> >
>>>>> > On 4/19/2020 1:54 PM, Andreas Plesch wrote:
>>>>> > > I am happy to report that the core OCCT vrml writer works fine with
>>>>> > > pythonocc. It generates more optimized vrml than the pythonocc x3d
>>>>> > > exporter.
>>>>> >
>>>>> > obvious statement: if the vrml exporter works well, it might be a
>>>>> simple variation to copy/refactor the output-producing sections of that
>>>>> code to output XML instead of WRL.
>>>>> >
>>>>> > ... reading on ...
>>>>> >
>>>>> > > I updated https://github.com/andreasplesch/OCCToX3D with example
>>>>> vrml
>>>>> > > output, and used tovrmlx3d to convert to xml which is also
>>>>> available
>>>>> > > in the repo.
>>>>> >
>>>>> > very pretty outputs.  is it using LineProperties to give colored
>>>>> borders along mesh edges?
>>>>>
>>>>> There is an option to generate wireframe output in addition to what
>>>>> they call shaded shape output. The edges are IndexedLineSets but the
>>>>> Appearance does not have a LineProperties. This is something which
>>>>> x3d.py could presumably do in postprocessing.
>>>>>
>>>>> Another goal would be to add better Metadata since it should be
>>>>> possible to get those - probably prettyy easily - with pythonocc from
>>>>> the original source.
>>>>>
>>>>> > > I could add tovrmlx3d to the docker image, and then call it from
>>>>> > > python. Are there other options to convert from vrml2 to xml x3d,
>>>>> > > using python ? x_ite could do it but requires a browser.
>>>>> >
>>>>> > i suspect that Holger knows how to run x_ite headless without a
>>>>> browser.
>>>>>
>>>>> John went with headless chrome which is an option but it seems heavier
>>>>> than using castle tovrmlx3d.
>>>>>
>>>>> >
>>>>> > > Enhancing c++ OCCT itself with a x3d writer, parallel to the vrml
>>>>> > > writer is in principle possible but pretty daunting.
>>>>> >
>>>>> > this is what i was trying to express above... don't rewrite the
>>>>> whole functionality, just look for output-producing code and put a selector
>>>>> on those code blocks for one X3D encoding or another - .wrl for VRML97,
>>>>> .x3dv for ClassicVRML (differs only in header line), then .x3d for XML.
>>>>>
>>>>> Yes but even with this strategy in mind it seems a major effort:
>>>>>
>>>>> mostly in:
>>>>>
>>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/Vrml;h=d24d7db215fb0c4f26ac73af52fe68735bf8adee;hb=master
>>>>>
>>>>> but also in:
>>>>>
>>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlAPI;h=bacc62597d161a9cb978386b13cb8ba8f072d6e4;hb=master
>>>>>
>>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlConverter;h=79507809ac27c8b4ed3674ede2c3198b06b352e1;hb=master\
>>>>>
>>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlData;h=18683ffff1dac6a17cd9eef9a3e37b25e642e50b;hb=master
>>>>>
>>>>> This would make x3d available to all OCCT users.
>>>>>
>>>>> Another option is to improve x3d generation by pythonocc:
>>>>>
>>>>>
>>>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/WebGl/x3dom_renderer.py
>>>>> which uses
>>>>>
>>>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Tesselator/ShapeTesselator.cpp#L427
>>>>>
>>>>> > > The pythonocc generated vrml and x3d does not include a nice
>>>>> default
>>>>> > > viewpoint. If x3d.py would have a createX3DfromString() method, it
>>>>> may
>>>>> > > be possible to add a viewpoint and then create xml with toXML.
>>>>> >
>>>>> > somebody with better python skills than me should write that
>>>>> parser.  shouldn't be hard for a python guru since all of the X3D nodes
>>>>> have Python classes with essentially the same name.
>>>>>
>>>>> Yes, I think that would make x3d.py a lot more useful. In addition a
>>>>> createVRMLfromString() would help with legacy content, and conversion.
>>>>> x_ite has good parsers in js which could serve as a template for
>>>>> python parsers. titanias parsers are probably similar.
>>>>>
>>>>> > > John suggested to use the stylesheet to convert from xml to Python
>>>>> > > code. Saxon is probably the only processor which can handle the
>>>>> > > stylesheet ? There is saxon-c/he which is free and has apparently a
>>>>> > > python wrapper:
>>>>> https://www.saxonica.com/saxon-c/doc/html/saxonc.html
>>>>> >
>>>>> > I'm using Saxon Home Edition jar in all of the ant files for builds
>>>>> and regression testing.  Also embedded in X3DJSAIL.  Even checked in to
>>>>> help builders avoid versionitis.
>>>>> >
>>>>> > Better to find and use a Python package for XSLT 2 (not 1).
>>>>>
>>>>> Agree but there seem to be no native Python packages for XSLT 2. The
>>>>> Python recommendation seems to be to use saxon he. In addition to the
>>>>> java version, there is the c version which has an official Python
>>>>> wrapper. Do you want to give it a quick try ?
>>>>>
>>>>> > We would not want to hardwire such a package into x3d.py since it
>>>>> might collide with other uses.  am trying to fit x3d.py well in the python
>>>>> ecology.
>>>>> >
>>>>> > meanwhile I believe there is a python incantation to check at run
>>>>> time and load such-and-such a package (if necessary) if a specialty method
>>>>> gets invoked.  we might also create a x3d-tools.py package for simplfying
>>>>> addition of commonly used packages (such as XSLT or even pythonocc).
>>>>>
>>>>> pythonocc is big, since it depends on c++ occ.
>>>>>
>>>>> There is a pip package:
>>>>> https://pypi.org/project/pythonocc-core/#description
>>>>> but it has not been updated to 7.4.0, the current version.
>>>>>
>>>>> But there is also a conda package
>>>>> https://anaconda.org/pythonocc/pythonocc-core
>>>>> which is uptodate. Hm, I wonder why this is not used for the binderhub
>>>>> pythonocc docker image. It is compiled from scratch in the Dockerfile,
>>>>> instead. Perhaps the package does not work well.
>>>>>
>>>>> > > Did anybody try saxon-c/he with the python api ? It may work.
>>>>> > >
>>>>> > > If that is succesful, it would be probably possible to eval the
>>>>> > > resulting python code in python and get a x3d document/object. This
>>>>> > > could then be manipulated with x3d.py.
>>>>> >
>>>>> > with enough coercion many things can be forced to work... but
>>>>> cross-language solutions are notoriously difficult to port and maintain.
>>>>> also worth noting is that 'eval()' opens up an endless security hole,
>>>>> usually frowned upon...
>>>>>
>>>>> I do not think there would a lot of coercion sicne there is an
>>>>> official Python API for saxon.
>>>>>
>>>>> >
>>>>> > don't want to sound discouraging!  a build workflow can work even if
>>>>> cross-language.  so those might establish initial capability, but we are
>>>>> likely best served by seeking general Pythonic solutions when
>>>>> possible/practical and then folding them into x3d.py itself.
>>>>>
>>>>> I agree, this should stay Python as much as possible.
>>>>>
>>>>> Cheers,  -Andreas
>>>>>
>>>>> > > So think the plan would be for now:
>>>>> > >
>>>>> > > - create a Dockerfile which includes also view3dscene for tovrmlx3d
>>>>> > > - try Jupyter viola and/or appmode, there is an example with
>>>>> upload of a file
>>>>> > > - use binder to host for now, get a viable service
>>>>> > > - try saxon-c/he python api, by installing in dockerfile and then
>>>>> > > trying it in Jupyter.
>>>>> > >
>>>>> > > -Andreas
>>>>> >
>>>>> > good luck out there!  8)
>>>>> >
>>>>> >
>>>>> > > On Sun, Apr 19, 2020 at 12:35 PM Andreas Plesch <
>>>>> andreasplesch at gmail.com> wrote:
>>>>> > >>
>>>>> > >> OpenCascade is the only realistic, open-source option to digest
>>>>> step
>>>>> > >> format. It is a c++ library, so there is a need for backend/server
>>>>> > >> side processing. The easiest option for that is docker and there
>>>>> is a
>>>>> > >> preconfigured, working docker image with both, a working
>>>>> OpenCascade
>>>>> > >> installation, and the pythonocc wrapper which also adds some
>>>>> > >> functionality such as x3d export. Also, python is faster to
>>>>> iterate
>>>>> > >> than c++, and jupyter gets you even more convenience.
>>>>> > >>
>>>>> > >> I added a small example jupyter notebook which work on top of
>>>>> > >> https://mybinder.org/v2/gh/tpaviot/pythonocc-binderhub/master to
>>>>> > >> https://github.com/andreasplesch/OCCToX3D, and an example x3d
>>>>> > >> conversion as proof of concept.
>>>>> > >>
>>>>> > >> I also discovered that OpenCascade supports generally compilation
>>>>> to
>>>>> > >> wasm using emscripten. So longer term it may be possible to
>>>>> compile
>>>>> > >> the OCC libraries required for loading, triangulation and
>>>>> exporting
>>>>> > >> (to VRML) to wasm/ecmascript, and get a completely client based
>>>>> > >> solution. However, the content typically is large and complex, and
>>>>> > >> there is a lot of computation involved. So in practice it may not
>>>>> work
>>>>> > >> out. A main advantage would be that users would not have to trust
>>>>> a
>>>>> > >> remote server with their valuable content.
>>>>> > >>
>>>>> > >> I think next I will try for a bit if is feasible to use the OCC
>>>>> VRML
>>>>> > >> writer, rather than the relatively simple pythonocc x3d exporter.
>>>>> The
>>>>> > >> python wrapper seems to exist but I do not think there is an
>>>>> example.
>>>>> > >>
>>>>> > >> -Andreas
>>>>> > >>
>>>>> > >> On Sat, Apr 18, 2020 at 9:47 PM John Carlson <yottzumm at gmail.com>
>>>>> wrote:
>>>>> > >>>
>>>>> > >>> X3DJSAIL has
>>>>> www.web3d.org/x3d/stylesheets/java/src/org/web3d/x3d/jsail/X3DLoaderDOM.java
>>>>> which supports DOM scenegraphs.
>>>>> > >>>
>>>>> > >>> Examples of usage are here:
>>>>> > >>>
>>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Json.js
>>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Nashorn.js
>>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/X3d.js
>>>>> > >>>
>>>>> > >>> Basically, the examples parse JSON into a document using
>>>>> X3DJSONLD.java then convert DOM into a X3DJSAIL scenegraph.
>>>>> > >>>
>>>>> > >>> So to answer your createX3DFromString() question, if you start
>>>>> with JSON, I think you could get a X3DJSAIL scenegraph from a JSON string
>>>>> using the X3DJSAIL-like python API (the one we got to work with Jupyter).
>>>>> > >>>
>>>>> > >>> Otherwise, you probably have to wait for enhancements to
>>>>> x3d.py.  It looks like toXML is supported in x3d.py currently.  One could
>>>>> use a stylesheet X3dToPython.xslt to convert XML to Python.
>>>>> > >>>
>>>>> > >>> So one could use the stylesheet in a web program (GL with that!)
>>>>> to output python.  Then you would have to use something like eval (ugh!).
>>>>> > >>>
>>>>> > >>> John
>>>>> > >>>
>>>>> > >>>
>>>>> > >>>
>>>>> > >>>
>>>>> > >>> On Sat, Apr 18, 2020 at 5:20 PM Andreas Plesch <
>>>>> andreasplesch at gmail.com> wrote:
>>>>> > >>>>
>>>>> > >>>> Thanks for the interest. I initialized a project at
>>>>> > >>>>
>>>>> > >>>> https://github.com/andreasplesch/OCCToX3D
>>>>> > >>>>
>>>>> > >>>> The wiki has some collected info and a short outline. The wiki
>>>>> is
>>>>> > >>>> editable by anyone.
>>>>> > >>>>
>>>>> > >>>> pythonocc has a method to generate X3D xml. It is a simple
>>>>> Shape with
>>>>> > >>>> a TriangleSet, nothing sophisticated. If x3d.py would have  a
>>>>> > >>>> createX3DFromString() function, it would be possible to use
>>>>> x3d.py to
>>>>> > >>>> enhance the generated X3D. But I do not think x3d.py has that.
>>>>> But
>>>>> > >>>> x3d.py may come in handy in other ways.
>>>>> > >>>>
>>>>> > >>>> The first goal is to add a simple jupyter notebook to the repo
>>>>> based
>>>>> > >>>> on the available pythonocc docker image which demonstrate how
>>>>> to load
>>>>> > >>>> step and output X3D.
>>>>> > >>>>
>>>>> > >>>> Any feedback or ideas welcome, using Issues on the project or
>>>>> here,
>>>>> > >>>>
>>>>> > >>>> -Andreas
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>> On Sat, Apr 18, 2020 at 1:27 PM Don Brutzman <brutzman at nps.edu>
>>>>> wrote:
>>>>> > >>>>>
>>>>> > >>>>> Hi Andreas, sounds like a great idea to me.  Here are some
>>>>> peripheral points of support.
>>>>> > >>>>>
>>>>> > >>>>> Last weekend i updated and deployed the x3d.py package
>>>>> (sometimes called X3DPSAIL) at
>>>>> > >>>>>
>>>>> > >>>>> [1]     Python Package Index (PyPi) x3d 0.0.25
>>>>> > >>>>>          Package support for Extensible 3D (X3D) Graphics
>>>>> International Standard (IS)
>>>>> > >>>>>          https://pypi.org/project/x3d/
>>>>> > >>>>>
>>>>> > >>>>> [2]     Python X3D Package x3d.py,X3D Python Scene Access
>>>>> Interface Library (X3DPSAIL)
>>>>> > >>>>>
>>>>> https://www.web3d.org/x3d/stylesheets/python/python.html
>>>>> > >>>>>
>>>>> > >>>>> "The x3d.py Python X3D Package supports programmers with
>>>>> Python interfaces and objects for standards-based X3D programming, all as
>>>>> open source.
>>>>> > >>>>> The presentation *Python X3D Package Implementation* provides
>>>>> an overview and shows examples."
>>>>> > >>>>>
>>>>> > >>>>> [2.1]   Python X3D Package Implementation
>>>>> > >>>>>
>>>>> https://www.web3d.org/x3d/stylesheets/python/documents/PythonX3dPackageImplementation.pdf
>>>>> > >>>>>
>>>>> > >>>>> So you would use this library for model creation and
>>>>> data-driven improvement, no need to reinvent X3D data structures or methods.
>>>>> > >>>>>
>>>>> > >>>>> This is autogenerated from X3DUOM so it will always stay in
>>>>> sync with latest/greatest X3D4.
>>>>> > >>>>>
>>>>> > >>>>> We've done some preliminary work with Jupyter notebook, listed
>>>>> there.  Lots of potential projects in TODO section.
>>>>> > >>>>>
>>>>> > >>>>> [2.2]   X3DPSAIL: Jupyter notebook, TODO
>>>>> > >>>>>
>>>>> https://www.web3d.org/x3d/stylesheets/python/python.html#Jupyter
>>>>> > >>>>>
>>>>> https://www.web3d.org/x3d/stylesheets/python/python.html#TODO
>>>>> > >>>>>
>>>>> > >>>>> For pythonocc conversion efforts, will be happy to ensure that
>>>>> > >>>>> a. all improvements to API are applied,
>>>>> > >>>>> b. ensure that validation of all scene-graph inputs remains
>>>>> rigorous,
>>>>> > >>>>> b. utility methods are added for easy use together,
>>>>> > >>>>> d. examplar conversion models are properly saved in X3D
>>>>> Examples Archive,
>>>>> > >>>>> e. Whatever else!
>>>>> > >>>>>
>>>>> > >>>>> Looks like there is vast potential here.  Hoping folks start
>>>>> picking it up and applying capabilities.
>>>>> > >>>>>
>>>>> > >>>>> Have fun with X3D Python!  8)
>>>>> > >>>>>
>>>>> > >>>>>
>>>>> > >>>>> On 4/17/2020 5:15 PM, Andreas Plesch wrote:
>>>>> > >>>>>> Did somebody put together an x3d converter based on pythonocc
>>>>> ?
>>>>> > >>>>>>
>>>>> > >>>>>> Here is an example:
>>>>> > >>>>>>
>>>>> > >>>>>> https://senties-martinelli.com/software/cad-viewer
>>>>> > >>>>>>
>>>>> > >>>>>> It could convert  step, stp, igs, iges, obj and stl formats.
>>>>> Format
>>>>> > >>>>>> import and export is only a small subset of pythonocc
>>>>> > >>>>>> (https://github.com/tpaviot/pythonocc-core) but I think just
>>>>> making
>>>>> > >>>>>> this functionality easy to use with a dedicated converter
>>>>> could be
>>>>> > >>>>>> really valuable.
>>>>> > >>>>>>
>>>>> > >>>>>> In the end, I think a  web service just for conversion would
>>>>> be most useful.
>>>>> > >>>>>>
>>>>> > >>>>>> I played with the Jupyter notebook and could generate with
>>>>> only a few lines x3d.
>>>>> > >>>>>>
>>>>> > >>>>>> And there are ways to generate a standalone webapp from a
>>>>> notebook:
>>>>> > >>>>>>
>>>>> https://discourse.jupyter.org/t/how-to-transform-a-jupyter-notebook-into-a-webapp/758/6
>>>>> > >>>>>>
>>>>> > >>>>>> That could be quick way to offer a web service.
>>>>> > >>>>>>
>>>>> > >>>>>> Have there been attempts of this kind ? Would there be
>>>>> interest in a
>>>>> > >>>>>> open source github hosted project ?
>>>>> > >>>>>>
>>>>> > >>>>>> -Andreas
>>>>> > >>>>>
>>>>> > >>>>> all the best, Don
>>>>> > >>>>> --
>>>>> > >>>>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>>>>> brutzman at nps.edu
>>>>> > >>>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>>>>  +1.831.656.2149
>>>>> > >>>>> X3D graphics, virtual worlds, navy robotics
>>>>> http://faculty.nps.edu/brutzman
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>>
>>>>> > >>>> --
>>>>> > >>>> Andreas Plesch
>>>>> > >>>> Waltham, MA 02453
>>>>> > >>>>
>>>>> > >>>> _______________________________________________
>>>>> > >>>> x3d-public mailing list
>>>>> > >>>> x3d-public at web3d.org
>>>>> > >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>> > >>
>>>>> > >>
>>>>> > >>
>>>>> > >> --
>>>>> > >> Andreas Plesch
>>>>> > >> Waltham, MA 02453
>>>>> > >
>>>>> > >
>>>>> > >
>>>>> > > --
>>>>> > > Andreas Plesch
>>>>> > > Waltham, MA 02453
>>>>> > >
>>>>> >
>>>>> > all the best, Don
>>>>> > --
>>>>> > Don Brutzman  Naval Postgraduate School, Code USW/Br
>>>>> brutzman at nps.edu
>>>>> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>>>>  +1.831.656.2149
>>>>> > X3D graphics, virtual worlds, navy robotics
>>>>> http://faculty.nps.edu/brutzman
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andreas Plesch
>>>>> Waltham, MA 02453
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200420/45ba7140/attachment-0001.html>

More information about the x3d-public mailing list