[x3d-public] was: X3D C/C++/C# language API bindings, now: conversions and X3D machine learning, ANTLR [ what John is thinking about conversions ], design for test cases for full code coverage for grammars and schemas, designing X3D virtual worlds from generators

[John Carlson]

[ Only subject retained from previous message to private list, moved to public list]

[ Long message, you may have to take a break.  This mail is mainly to provide Terence a reference point for X3D/VRML and to let people know my current thinking, Re: X3D conversions and APIs ]

NetworkSensor folks may be interested in this for creating a front-end VRML parser that contacts a back-end as part of parsing VRML.

Don:  I think it would be fairly simple to write X3D (VRML, Java, Python and JavaScript) program parsers with ANTLR, given existing grammars, or even new grammars. Not that we need translators for those things. It would be good to have a single grammar (maybe VRML) that we can convert to other grammars for other languages.  Probably X3DUOM is our best bet at this point, if XML+DSL is supported in ANTLR.  How shall we generate various document grammars (I’m still in generation mode) and schemas (not needed because we now have grammars) from X3DUOM? Basically, I’d like to provide parsers as adjunct libraries to X3DJSAIL, so we can use any encoding or well formatted program as input with X3DJSAIL (in addition to using SAI from the program in the compiled version). We only have limited inputs right now I think (XML and JSON).  Let me know of other inputs.

Video introducing ANTLR 4: https://vimeo.com/59285751 It’s not your 1989 parser-generator any more, baby! Awesome!

Turing/Enigma/Digitization/Symbol Processing [ Parsing and Machine Learning ] [ Second in a Series ]

For non-spam, skip the next paragraph. For those reading my “Rise and Demise of Command Computing” series, continue here.

The gist of this is, can I use continuous thought, voice, emotion, senses and action to create a virtual world on digital systems?  Has digitization (CCDs and LEDs for example) and symbols led us down a path of using commands (including mouse “commands”, finger presses, swipes, searches that resolve to commands instead of ecosystems) to address the computer, much to the detriment of our humanity?  Do we need to revisit analog computing to achieve new advances?  I take alleys discussing parsers, and how they are turning into mere command generators (listener and visitor method calls, “spawn”, “exec” and friends), instead of achieving more glorious things, like dance -> choreographer notation or choreographer notation -> dance -> predicted audience response (domain specific visual languages—no, not visual studio! Arg! BTW: A computer scientist’s primary architectural visual language is often collections of commands—UML class diagrams).  One bright spot has been the development of sign language processing on computers (but I don’t understand sign language). For a little homework before my next installment, try to think of a continuous command (I’ve thought of a continuous verb, but not a continuous command).   Then try to thing of a continuous subject or object.  See the difference?  Commands are likely the most transient and least kept track of thing in computer systems.  What if you had a system which kept track of all your commands so you could undo or redo them?  Why don’t we have that?  What if we didn’t have commands at all? What is the minimal set of commands we could use?  Message continues below (search for spam).

I certainly hope people enjoy my emails!  Please send me updates!

One goal of this email is to try to open a discussion for the future of our X3D (web3d) conversion and encoding efforts, even by perhaps applying machine learning (for regression testing speed) on our corpus of X3D XML (and other encodings) “mostly-structured” documents to do conversions/formatting on them.   Since we have thousands of “good” translations to Python, Java, JavaScript, XML, JSON, VRML and EXI, it would be useful to compare the results of machine learning conversions to that of an translation designer doing the same.  I did have a proposal to take an XML document and make it structured (CSV, on the order of 6 columns), if people think that’s useful to resurrect, but it may be worth using EXI “text binary” encoding for machine learning (neural networks) instead.  For examples of some conversions and X3D documents, visit this website: https://coderextreme.net/X3DJSONLD/src/main/html/index.html (EXI is on far top right, you need to click on left top “here” link to initiate current conversions.)  For downloads of more X3D examples and more, go here: http://www.web3d.org/x3d/content/examples/X3dResources.html#Examples [ Thanks, Don and all! ]

In other words, I’m looking for machine learning or the below to create a massive performance conversion win once it’s trained (you can store and restore neural networks using ONNX), and running regression tests will be much quicker than it currently is.

Terence Parr (ANTLR author) cc’ed is into machine learning again, https://explained.ai/. He is cc’ed for both his experience with parser-generators and machine learning.

In my web searches, I discovered that ANTLR 4.5+ has a JavaScript version https://www.antlr.org/download/index.html in addition to C++, C# and Java versions (soon TypeScript I’ve heard), so I think that deserves some investigation for our X3D conversions.  https://github.com/antlr/grammars-v4 -- maybe we could offer ANTLR a VRML grammar?  Do our encodings offer enough in the way of parsing, or can we use full parsers to good use—beyond mere syntax validation to evaluation perhaps?  I for one, would like to plug a JavaScript parser into the embedded VRML Script node to help clear up potential cross-site scripting issues. It’s very nice to see that the parsing problem has been separated from the evaluation problem, too, in ANTLR 4, so your grammar is useful cross language (compared to something like bison or yacc).  ANTLR also has advantages for grammar understandability and backtracking, including left recursion. Andreas, with ANTLR 4.5+ JavaScript, could we write a VRML parser for X3DOM, X3DV4 or contribute back to ANTLR?  Could we create an encoding independent ProtoExpander with ANTLR 4?

What I am trying to say is, ANTLR probably has superiority over XSLT 2.0 for portability, recursion, and is likely gaining ground in regards to programming language support, especially to the web (I haven’t looked at XSLT 3.0 much),  ANTLR now has SAX-like parsing and visitor patterns. Is there potential for XSLT written in ANTLR? What are the considerations for ANTLR being proprietary versus the supposed “openness” of XSLT 3.0? Java and thus JavaScript (not ECMAScript) also have a lot of potential for going proprietary, as seen by recent moves by Oracle, plus java is being fragmented now.   Java is not a standard as far as I know, so ECMAScript and C++ might be our most “standard, freely available” options.   I have not heard that Python, Perl or PHP is a standard yet.

X3D has a long history of “separating” content from syntax.  Let’s keep it that way and enhance it!

Terence, I seem to recall some work in ANTLR 3 toward parsing objects (an AST to generate text in this case) instead of text.  Is that in ANTLR 4?  Can I build a language parser on top of a parser for XML or JSON (XML and JSON-based DSLs or POLs)?  I would be willing to work on a X3D JSON Schema generated from http://www.web3d.org/specifications/X3DUOM.html with ANTLR in a desired language (one that ANTLR is written in), essentially rewriting the Python version of the JSON schema generator, if I can parse both the XML and the domain specific language written in XML-encoding (say with listener objects for both XML and the DSL, or the combination, perhaps for ALL encodings if we do it clever enough). Thus we will have a grammar for X3DUOM in addition to the schema (that we could validate and verify, beyond what X3DJSAIL stylesheet and my python provide). Once we have a grammar for X3DUOM (beyond XML and JSON), there will be a LOT more freedom of expression. If we agree on a standard X3D grammar (in addition to standard schemas and object models)—say VRML, which can be plugged into any popular language without concern for doing parsing of JSON, XML, DOM or VRML parsing specifics, and we just have to program parser listeners and or visitors in our language to generate a scenegraph with SAI, I think we will have gained something. Your opinion is requested.  Don, do you consider set* and add* methods of http://www.web3d.org/specifications/java/X3DJSAIL.html listener or visitor methods in some language parsing project?  Could we easily adapt the X3DJSAIL methods to say, a SAX parser or ANTLR 4 parser (SAX-like)?  We currently have a DOM-parser in X3DJSAIL, correct? Could we now with ANTLR 4 easily write a VRML parser on top of X3DJSAIL (instead of in X3DJSAIL, as I did with X3DJSONLD.java and JSON) using ANTLR 4 Java version?

I’d also like to see some of the trees found in the video built in d3.js or vega.  For example, here is the X3D JSON schema as a force directed graph in three.js, I think:  https://coderextreme.net/x3dschema/ Cool!

As an aside, Terence I have an idea for generating X3D (and other) test cases that involves an inverted (bottom up) grammar, very similar to take all paths through the grammar maze mentioned in the video (I like the relation to quantum mechanics), but more like planting generator seeds all over the maze to grow in all directions (potentially using bottom up Monte-Carlo search as a second choice) in the maze, until the whole maze has been discovered (full code coverage), if you’re still into grammars.  Send me a note if you’re interested or know anything about this!  If it’s a unique idea, using inverted grammars and inverted schemas to generate test cases, bottom up, can I get a remote PhD or Master’s? I’m pretty much stuck in Iowa right now.

I also thing there’s potential for generating real X3D scenes with XML with the above!  If we consider the domain-specific grammar of an X3D encoding DSL (not the specific encoding, but something more like the abstract X3D standard), we can create or use computer science generators in the visitors (as above for test cases). There are many types of generators: L-system, procedural, loops, stochastic, chaotic, video, audio, shape, hyperdimensional, quantum, binary, floating point, matrix, neural network (NEAT),  document, building, system, network, ecosystem, meta etc. [ and only 3 types of LED/print media, hypervoxel, hypertext and hypervolume—particle-wave trinity ] the output of which might be inserted into a scene via the same mechanism as generating test cases.  If not already there, I might write a VRML Script which generates weather for a virtual world using an Lorenz attractor, and “outputs” tuples like lat,long,weather.

Then there’s translating web3d XSLT to other languages. I could go on and on.  Plenty of work!

ANTLR 4 is good at parallel parsing of a single threaded computer document (let’s call this script parsing).  Let’s extend this beyond scripts (1970s AI technology which failed, seemingly) to streams, systems, networks (actors I guess, which hasn’t really caught on, AFAICT. Maybe singularity.net and RoboEarth are examples) and planets.   How do we integrate ANTLR with a hypervideo machine learning system or even the futuristic all seeing “1984” internet, which is massively parallel?  Imagine ANTLR supervisor host downloading grammars, etc.  into neural network based video devices and collecting parse events (listener and visitor calls) from all over the planet.  Then you can zoom into the grammar for the exact parse tree, state variables, to find out what’s happening in the video etc.  Terence Parr for dictator! But first he has to reach stage 16 of this video game: https://en.wikipedia.org/wiki/Model_of_hierarchical_complexity#Stages_of_hierarchical_complexity (has he?)

Can you see X3D/VRML as a compression format?  Has anyone done this?

Terence, can I update the Java or JavaScript listeners after a grammar modification automatically, or does it require more intervention for a version upgrade?

Can we open up VRML/X3D beyond standard encodings to “build your own encoding” with ANTLR. Then we will have a negotiable protocol pretty much?  Could we use the listener/visitor interface in ANTLR (with buffering) as a protocol?   A new X3D API?   Terence, how good is ANTLR at buffering up listener/visitor calls for efficient networking?  Can we make it work with an “infinite” undo/redo?  Can we dump calls in a message queue?

“To Infinity! and Beyond!” – Buzz Lightyear in Toy Story

Spam continues below.

Next discussion topic, GUIfied or “syntax free” operations or “operations almost anywhere” in language (probably similar to FORTH or PDF), AKA very high level reversible programming. After that, quantum programming by demonstration. In the continuing series on the rise and demise of command line interpreters and command GUIs.  Did Alan Kay get rid of command languages just to invent a command GUI?  Is HTML, XML, JSON, and CSS “command-free”?  What about HTTP? Why did Alan Kay come up with new networking for Croquet? Why isn’t there a “command-free” interface built with hypertext:  Think (oh shucks, this is a command) of a subject and the relevant material will popup on your screen—the system will web crawl the current screen and referred screens looking for related information).  Running a computer with continuous thoughts, continuous haptics, continuous voices, continuous scribbling, drawing, painting, dancing and pottery making, etc.  Have I reinvented DynamicLand or 1984 yet? Did Turing lead us down the wrong path with symbol processing?  Or was it the Germans? What exactly is an analog computer?

I supposed I really should create a blog for these.  Coffee time!

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190428/6212c62d/attachment-0001.html>