[x3d-public] file converter based on pythonocc
John Carlson
yottzumm at gmail.com
Mon Apr 20 05:00:04 PDT 2020
If someone knows a browser testing package that will traverse a collection
of pages, let me know.
Maybe puppeteer or CEFpython? We were talking about using python. I just
got CEFpython working the other day for our RPG product.
Will be very helpful!
John
On Mon, Apr 20, 2020 at 6:57 AM John Carlson <yottzumm at gmail.com> wrote:
> One can get a snapshot out of chrome --headless, or perhaps use puppeteer,
> but I'm not pursuing that until I have an X_ITE SAI HelloWorld file example
> that passes through the browser in normal mode.
>
> I spent the last couple of days polishing up X3DJSONLD and pythonSAI.
>
> John
>
> On Mon, Apr 20, 2020 at 6:48 AM John Carlson <yottzumm at gmail.com> wrote:
>
>> what I do plan on doing is generating an XHTML package around my
>> VRMLScript. Then I will go headless if necessary--But I'm not clear at all
>> how to get data out of chrome --headless (docs seem to be wrong). There was
>> a previous email, not sure.
>>
>> On Mon, Apr 20, 2020 at 6:44 AM John Carlson <yottzumm at gmail.com> wrote:
>>
>>> I did not "go with" headless chrome. Node.js yes. That's different
>>> from headless chrome. Chrome is a full browser environment. I currently
>>> am using your suggestion of pasting VRMLScript into the headed browser.
>>>
>>> I am supporting Nashorn, Node.js, Python, Java on server side, XML,
>>> JSON, EXI, PLY (not done) and STL (not done), and soon (hopefully, still
>>> thinking of how I want to handle MF nodes), VRMLScript.
>>>
>>> Thanks for the link on tessellation, I will look into it. By chance is
>>> there a JavaScript tessellator?
>>>
>>> John
>>>
>>> On Sun, Apr 19, 2020 at 8:15 PM Andreas Plesch <andreasplesch at gmail.com>
>>> wrote:
>>>
>>>> Thanks for the response, see below for more.
>>>>
>>>> On Sun, Apr 19, 2020 at 7:29 PM Don Brutzman <brutzman at nps.edu> wrote:
>>>> >
>>>> > cooler and cooler!
>>>> >
>>>> > On 4/19/2020 1:54 PM, Andreas Plesch wrote:
>>>> > > I am happy to report that the core OCCT vrml writer works fine with
>>>> > > pythonocc. It generates more optimized vrml than the pythonocc x3d
>>>> > > exporter.
>>>> >
>>>> > obvious statement: if the vrml exporter works well, it might be a
>>>> simple variation to copy/refactor the output-producing sections of that
>>>> code to output XML instead of WRL.
>>>> >
>>>> > ... reading on ...
>>>> >
>>>> > > I updated https://github.com/andreasplesch/OCCToX3D with example
>>>> vrml
>>>> > > output, and used tovrmlx3d to convert to xml which is also available
>>>> > > in the repo.
>>>> >
>>>> > very pretty outputs. is it using LineProperties to give colored
>>>> borders along mesh edges?
>>>>
>>>> There is an option to generate wireframe output in addition to what
>>>> they call shaded shape output. The edges are IndexedLineSets but the
>>>> Appearance does not have a LineProperties. This is something which
>>>> x3d.py could presumably do in postprocessing.
>>>>
>>>> Another goal would be to add better Metadata since it should be
>>>> possible to get those - probably prettyy easily - with pythonocc from
>>>> the original source.
>>>>
>>>> > > I could add tovrmlx3d to the docker image, and then call it from
>>>> > > python. Are there other options to convert from vrml2 to xml x3d,
>>>> > > using python ? x_ite could do it but requires a browser.
>>>> >
>>>> > i suspect that Holger knows how to run x_ite headless without a
>>>> browser.
>>>>
>>>> John went with headless chrome which is an option but it seems heavier
>>>> than using castle tovrmlx3d.
>>>>
>>>> >
>>>> > > Enhancing c++ OCCT itself with a x3d writer, parallel to the vrml
>>>> > > writer is in principle possible but pretty daunting.
>>>> >
>>>> > this is what i was trying to express above... don't rewrite the whole
>>>> functionality, just look for output-producing code and put a selector on
>>>> those code blocks for one X3D encoding or another - .wrl for VRML97, .x3dv
>>>> for ClassicVRML (differs only in header line), then .x3d for XML.
>>>>
>>>> Yes but even with this strategy in mind it seems a major effort:
>>>>
>>>> mostly in:
>>>>
>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/Vrml;h=d24d7db215fb0c4f26ac73af52fe68735bf8adee;hb=master
>>>>
>>>> but also in:
>>>>
>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlAPI;h=bacc62597d161a9cb978386b13cb8ba8f072d6e4;hb=master
>>>>
>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlConverter;h=79507809ac27c8b4ed3674ede2c3198b06b352e1;hb=master\
>>>>
>>>> http://git.dev.opencascade.org/gitweb/?p=occt.git;a=tree;f=src/VrmlData;h=18683ffff1dac6a17cd9eef9a3e37b25e642e50b;hb=master
>>>>
>>>> This would make x3d available to all OCCT users.
>>>>
>>>> Another option is to improve x3d generation by pythonocc:
>>>>
>>>>
>>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Display/WebGl/x3dom_renderer.py
>>>> which uses
>>>>
>>>> https://github.com/tpaviot/pythonocc-core/blob/master/src/Tesselator/ShapeTesselator.cpp#L427
>>>>
>>>> > > The pythonocc generated vrml and x3d does not include a nice default
>>>> > > viewpoint. If x3d.py would have a createX3DfromString() method, it
>>>> may
>>>> > > be possible to add a viewpoint and then create xml with toXML.
>>>> >
>>>> > somebody with better python skills than me should write that parser.
>>>> shouldn't be hard for a python guru since all of the X3D nodes have Python
>>>> classes with essentially the same name.
>>>>
>>>> Yes, I think that would make x3d.py a lot more useful. In addition a
>>>> createVRMLfromString() would help with legacy content, and conversion.
>>>> x_ite has good parsers in js which could serve as a template for
>>>> python parsers. titanias parsers are probably similar.
>>>>
>>>> > > John suggested to use the stylesheet to convert from xml to Python
>>>> > > code. Saxon is probably the only processor which can handle the
>>>> > > stylesheet ? There is saxon-c/he which is free and has apparently a
>>>> > > python wrapper:
>>>> https://www.saxonica.com/saxon-c/doc/html/saxonc.html
>>>> >
>>>> > I'm using Saxon Home Edition jar in all of the ant files for builds
>>>> and regression testing. Also embedded in X3DJSAIL. Even checked in to
>>>> help builders avoid versionitis.
>>>> >
>>>> > Better to find and use a Python package for XSLT 2 (not 1).
>>>>
>>>> Agree but there seem to be no native Python packages for XSLT 2. The
>>>> Python recommendation seems to be to use saxon he. In addition to the
>>>> java version, there is the c version which has an official Python
>>>> wrapper. Do you want to give it a quick try ?
>>>>
>>>> > We would not want to hardwire such a package into x3d.py since it
>>>> might collide with other uses. am trying to fit x3d.py well in the python
>>>> ecology.
>>>> >
>>>> > meanwhile I believe there is a python incantation to check at run
>>>> time and load such-and-such a package (if necessary) if a specialty method
>>>> gets invoked. we might also create a x3d-tools.py package for simplfying
>>>> addition of commonly used packages (such as XSLT or even pythonocc).
>>>>
>>>> pythonocc is big, since it depends on c++ occ.
>>>>
>>>> There is a pip package:
>>>> https://pypi.org/project/pythonocc-core/#description
>>>> but it has not been updated to 7.4.0, the current version.
>>>>
>>>> But there is also a conda package
>>>> https://anaconda.org/pythonocc/pythonocc-core
>>>> which is uptodate. Hm, I wonder why this is not used for the binderhub
>>>> pythonocc docker image. It is compiled from scratch in the Dockerfile,
>>>> instead. Perhaps the package does not work well.
>>>>
>>>> > > Did anybody try saxon-c/he with the python api ? It may work.
>>>> > >
>>>> > > If that is succesful, it would be probably possible to eval the
>>>> > > resulting python code in python and get a x3d document/object. This
>>>> > > could then be manipulated with x3d.py.
>>>> >
>>>> > with enough coercion many things can be forced to work... but
>>>> cross-language solutions are notoriously difficult to port and maintain.
>>>> also worth noting is that 'eval()' opens up an endless security hole,
>>>> usually frowned upon...
>>>>
>>>> I do not think there would a lot of coercion sicne there is an
>>>> official Python API for saxon.
>>>>
>>>> >
>>>> > don't want to sound discouraging! a build workflow can work even if
>>>> cross-language. so those might establish initial capability, but we are
>>>> likely best served by seeking general Pythonic solutions when
>>>> possible/practical and then folding them into x3d.py itself.
>>>>
>>>> I agree, this should stay Python as much as possible.
>>>>
>>>> Cheers, -Andreas
>>>>
>>>> > > So think the plan would be for now:
>>>> > >
>>>> > > - create a Dockerfile which includes also view3dscene for tovrmlx3d
>>>> > > - try Jupyter viola and/or appmode, there is an example with upload
>>>> of a file
>>>> > > - use binder to host for now, get a viable service
>>>> > > - try saxon-c/he python api, by installing in dockerfile and then
>>>> > > trying it in Jupyter.
>>>> > >
>>>> > > -Andreas
>>>> >
>>>> > good luck out there! 8)
>>>> >
>>>> >
>>>> > > On Sun, Apr 19, 2020 at 12:35 PM Andreas Plesch <
>>>> andreasplesch at gmail.com> wrote:
>>>> > >>
>>>> > >> OpenCascade is the only realistic, open-source option to digest
>>>> step
>>>> > >> format. It is a c++ library, so there is a need for backend/server
>>>> > >> side processing. The easiest option for that is docker and there
>>>> is a
>>>> > >> preconfigured, working docker image with both, a working
>>>> OpenCascade
>>>> > >> installation, and the pythonocc wrapper which also adds some
>>>> > >> functionality such as x3d export. Also, python is faster to iterate
>>>> > >> than c++, and jupyter gets you even more convenience.
>>>> > >>
>>>> > >> I added a small example jupyter notebook which work on top of
>>>> > >> https://mybinder.org/v2/gh/tpaviot/pythonocc-binderhub/master to
>>>> > >> https://github.com/andreasplesch/OCCToX3D, and an example x3d
>>>> > >> conversion as proof of concept.
>>>> > >>
>>>> > >> I also discovered that OpenCascade supports generally compilation
>>>> to
>>>> > >> wasm using emscripten. So longer term it may be possible to compile
>>>> > >> the OCC libraries required for loading, triangulation and exporting
>>>> > >> (to VRML) to wasm/ecmascript, and get a completely client based
>>>> > >> solution. However, the content typically is large and complex, and
>>>> > >> there is a lot of computation involved. So in practice it may not
>>>> work
>>>> > >> out. A main advantage would be that users would not have to trust a
>>>> > >> remote server with their valuable content.
>>>> > >>
>>>> > >> I think next I will try for a bit if is feasible to use the OCC
>>>> VRML
>>>> > >> writer, rather than the relatively simple pythonocc x3d exporter.
>>>> The
>>>> > >> python wrapper seems to exist but I do not think there is an
>>>> example.
>>>> > >>
>>>> > >> -Andreas
>>>> > >>
>>>> > >> On Sat, Apr 18, 2020 at 9:47 PM John Carlson <yottzumm at gmail.com>
>>>> wrote:
>>>> > >>>
>>>> > >>> X3DJSAIL has
>>>> www.web3d.org/x3d/stylesheets/java/src/org/web3d/x3d/jsail/X3DLoaderDOM.java
>>>> which supports DOM scenegraphs.
>>>> > >>>
>>>> > >>> Examples of usage are here:
>>>> > >>>
>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Json.js
>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/Nashorn.js
>>>> > >>> www.web3d.org/x3d/stylesheets/java/nashorn/examples/X3d.js
>>>> > >>>
>>>> > >>> Basically, the examples parse JSON into a document using
>>>> X3DJSONLD.java then convert DOM into a X3DJSAIL scenegraph.
>>>> > >>>
>>>> > >>> So to answer your createX3DFromString() question, if you start
>>>> with JSON, I think you could get a X3DJSAIL scenegraph from a JSON string
>>>> using the X3DJSAIL-like python API (the one we got to work with Jupyter).
>>>> > >>>
>>>> > >>> Otherwise, you probably have to wait for enhancements to x3d.py.
>>>> It looks like toXML is supported in x3d.py currently. One could use a
>>>> stylesheet X3dToPython.xslt to convert XML to Python.
>>>> > >>>
>>>> > >>> So one could use the stylesheet in a web program (GL with that!)
>>>> to output python. Then you would have to use something like eval (ugh!).
>>>> > >>>
>>>> > >>> John
>>>> > >>>
>>>> > >>>
>>>> > >>>
>>>> > >>>
>>>> > >>> On Sat, Apr 18, 2020 at 5:20 PM Andreas Plesch <
>>>> andreasplesch at gmail.com> wrote:
>>>> > >>>>
>>>> > >>>> Thanks for the interest. I initialized a project at
>>>> > >>>>
>>>> > >>>> https://github.com/andreasplesch/OCCToX3D
>>>> > >>>>
>>>> > >>>> The wiki has some collected info and a short outline. The wiki is
>>>> > >>>> editable by anyone.
>>>> > >>>>
>>>> > >>>> pythonocc has a method to generate X3D xml. It is a simple Shape
>>>> with
>>>> > >>>> a TriangleSet, nothing sophisticated. If x3d.py would have a
>>>> > >>>> createX3DFromString() function, it would be possible to use
>>>> x3d.py to
>>>> > >>>> enhance the generated X3D. But I do not think x3d.py has that.
>>>> But
>>>> > >>>> x3d.py may come in handy in other ways.
>>>> > >>>>
>>>> > >>>> The first goal is to add a simple jupyter notebook to the repo
>>>> based
>>>> > >>>> on the available pythonocc docker image which demonstrate how to
>>>> load
>>>> > >>>> step and output X3D.
>>>> > >>>>
>>>> > >>>> Any feedback or ideas welcome, using Issues on the project or
>>>> here,
>>>> > >>>>
>>>> > >>>> -Andreas
>>>> > >>>>
>>>> > >>>>
>>>> > >>>>
>>>> > >>>>
>>>> > >>>> On Sat, Apr 18, 2020 at 1:27 PM Don Brutzman <brutzman at nps.edu>
>>>> wrote:
>>>> > >>>>>
>>>> > >>>>> Hi Andreas, sounds like a great idea to me. Here are some
>>>> peripheral points of support.
>>>> > >>>>>
>>>> > >>>>> Last weekend i updated and deployed the x3d.py package
>>>> (sometimes called X3DPSAIL) at
>>>> > >>>>>
>>>> > >>>>> [1] Python Package Index (PyPi) x3d 0.0.25
>>>> > >>>>> Package support for Extensible 3D (X3D) Graphics
>>>> International Standard (IS)
>>>> > >>>>> https://pypi.org/project/x3d/
>>>> > >>>>>
>>>> > >>>>> [2] Python X3D Package x3d.py,X3D Python Scene Access
>>>> Interface Library (X3DPSAIL)
>>>> > >>>>>
>>>> https://www.web3d.org/x3d/stylesheets/python/python.html
>>>> > >>>>>
>>>> > >>>>> "The x3d.py Python X3D Package supports programmers with Python
>>>> interfaces and objects for standards-based X3D programming, all as open
>>>> source.
>>>> > >>>>> The presentation *Python X3D Package Implementation* provides
>>>> an overview and shows examples."
>>>> > >>>>>
>>>> > >>>>> [2.1] Python X3D Package Implementation
>>>> > >>>>>
>>>> https://www.web3d.org/x3d/stylesheets/python/documents/PythonX3dPackageImplementation.pdf
>>>> > >>>>>
>>>> > >>>>> So you would use this library for model creation and
>>>> data-driven improvement, no need to reinvent X3D data structures or methods.
>>>> > >>>>>
>>>> > >>>>> This is autogenerated from X3DUOM so it will always stay in
>>>> sync with latest/greatest X3D4.
>>>> > >>>>>
>>>> > >>>>> We've done some preliminary work with Jupyter notebook, listed
>>>> there. Lots of potential projects in TODO section.
>>>> > >>>>>
>>>> > >>>>> [2.2] X3DPSAIL: Jupyter notebook, TODO
>>>> > >>>>>
>>>> https://www.web3d.org/x3d/stylesheets/python/python.html#Jupyter
>>>> > >>>>>
>>>> https://www.web3d.org/x3d/stylesheets/python/python.html#TODO
>>>> > >>>>>
>>>> > >>>>> For pythonocc conversion efforts, will be happy to ensure that
>>>> > >>>>> a. all improvements to API are applied,
>>>> > >>>>> b. ensure that validation of all scene-graph inputs remains
>>>> rigorous,
>>>> > >>>>> b. utility methods are added for easy use together,
>>>> > >>>>> d. examplar conversion models are properly saved in X3D
>>>> Examples Archive,
>>>> > >>>>> e. Whatever else!
>>>> > >>>>>
>>>> > >>>>> Looks like there is vast potential here. Hoping folks start
>>>> picking it up and applying capabilities.
>>>> > >>>>>
>>>> > >>>>> Have fun with X3D Python! 8)
>>>> > >>>>>
>>>> > >>>>>
>>>> > >>>>> On 4/17/2020 5:15 PM, Andreas Plesch wrote:
>>>> > >>>>>> Did somebody put together an x3d converter based on pythonocc ?
>>>> > >>>>>>
>>>> > >>>>>> Here is an example:
>>>> > >>>>>>
>>>> > >>>>>> https://senties-martinelli.com/software/cad-viewer
>>>> > >>>>>>
>>>> > >>>>>> It could convert step, stp, igs, iges, obj and stl formats.
>>>> Format
>>>> > >>>>>> import and export is only a small subset of pythonocc
>>>> > >>>>>> (https://github.com/tpaviot/pythonocc-core) but I think just
>>>> making
>>>> > >>>>>> this functionality easy to use with a dedicated converter
>>>> could be
>>>> > >>>>>> really valuable.
>>>> > >>>>>>
>>>> > >>>>>> In the end, I think a web service just for conversion would
>>>> be most useful.
>>>> > >>>>>>
>>>> > >>>>>> I played with the Jupyter notebook and could generate with
>>>> only a few lines x3d.
>>>> > >>>>>>
>>>> > >>>>>> And there are ways to generate a standalone webapp from a
>>>> notebook:
>>>> > >>>>>>
>>>> https://discourse.jupyter.org/t/how-to-transform-a-jupyter-notebook-into-a-webapp/758/6
>>>> > >>>>>>
>>>> > >>>>>> That could be quick way to offer a web service.
>>>> > >>>>>>
>>>> > >>>>>> Have there been attempts of this kind ? Would there be
>>>> interest in a
>>>> > >>>>>> open source github hosted project ?
>>>> > >>>>>>
>>>> > >>>>>> -Andreas
>>>> > >>>>>
>>>> > >>>>> all the best, Don
>>>> > >>>>> --
>>>> > >>>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>>> brutzman at nps.edu
>>>> > >>>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>>> +1.831.656.2149
>>>> > >>>>> X3D graphics, virtual worlds, navy robotics
>>>> http://faculty.nps.edu/brutzman
>>>> > >>>>
>>>> > >>>>
>>>> > >>>>
>>>> > >>>> --
>>>> > >>>> Andreas Plesch
>>>> > >>>> Waltham, MA 02453
>>>> > >>>>
>>>> > >>>> _______________________________________________
>>>> > >>>> x3d-public mailing list
>>>> > >>>> x3d-public at web3d.org
>>>> > >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>> > >>
>>>> > >>
>>>> > >>
>>>> > >> --
>>>> > >> Andreas Plesch
>>>> > >> Waltham, MA 02453
>>>> > >
>>>> > >
>>>> > >
>>>> > > --
>>>> > > Andreas Plesch
>>>> > > Waltham, MA 02453
>>>> > >
>>>> >
>>>> > all the best, Don
>>>> > --
>>>> > Don Brutzman Naval Postgraduate School, Code USW/Br
>>>> brutzman at nps.edu
>>>> > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>>> +1.831.656.2149
>>>> > X3D graphics, virtual worlds, navy robotics
>>>> http://faculty.nps.edu/brutzman
>>>>
>>>>
>>>>
>>>> --
>>>> Andreas Plesch
>>>> Waltham, MA 02453
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200420/3c36575d/attachment-0001.html>
More information about the x3d-public
mailing list