[semantics-public] interpreter -> compiler -> Humanoid Animation

Tue Jul 21 00:43:27 PDT 2020

I didn't quite follow all that, Joseph, perhaps mostly because I'm
pretty ignorant when it comes to whatever magical powers X3D might
have.

If it helps, skip all my meandering philosophy in this essay ("Natural
Semantic Programming") by searching on Towers of Hanoi.

  https://docs.google.com/document/d/1CWpals3OBWgAC8OcWuWBhLoxjgpE-khxxT33NnFnzm8/edit

There, I hypothesize a situation where a blind child prodigy listens
over the phone to the NSP program being read, a version that doesn't
have "Towers of Hanoi" named as such in the code, and realizes what
game it is -- and writes an animation into the code. I suppose that,
with enough NSM definition of what a human is, and how a human body is
constrained, the animation could include a human figure actually
moving the disks from peg to peg.

To be sure, I have no complete parser for what I wrote there -- it's
an NSM-constrained pseudocode exercise. That said, NSM-DALIA

https://github.com/Yakushima/NSM-DALIA

might not be too far from parsing something very close to this
pseudocode and executing a specialized parse tree as Prolog. I got
NSM-DALIA working again (John helped a little at one point) and as far
as I know, it's the only working software that can parse NSM and
translate NSM to other natural-language NSM descriptions.

My goals are two-fold:

-- a programming language that uses natural language in a common-sense
way (though allowing for the more-or-less universal syntax of
mathematics where things truly need to be mathematically expressed)

-- a system that depends on no /particular/ natural language. Programs
are represented as meanings, and are parsed into that semantic
representation and re-generated with appropriate vocabulary and
"natural-language syntactic sugar" as required, depending on the users
preferred or native language.

I'm not at all sure what this could have to do with X3D. I think the
more natural first area of commercial application of Natural Semantic
Programming would be as a kind of "meta-COBOL" constraint-oriented
programming language, for enterprises that need to operate more
smoothly across language barriers. For 3D animation, I suppose that
internationally distributed machine repair would bring in some need
for graphics with constraint programming. For example, animate "here's
how you swap the old part out and put in a new one", with an animation
that allows the trainee to zoom and pan and even to focus in so far
that the he can see slow-motion/stop-motion finger-level details of a
repair or maintenance procedure.

Regards,
Michael Turner
Executive Director
Project Persephone
1-25-33 Takadanobaba
Shinjuku-ku Tokyo 169-0075
Mobile: +81 (90) 5203-8682
turner at projectpersephone.org

Understand - http://www.projectpersephone.org/
Join - http://www.facebook.com/groups/ProjectPersephone/
Donate - http://www.patreon.com/ProjectPersephone
Volunteer - https://github.com/ProjectPersephone

"Love does not consist in gazing at each other, but in looking outward
together in the same direction." -- Antoine de Saint-Exupéry

On Mon, Jul 20, 2020 at 10:42 PM Joseph D Williams
<joedwil at earthlink.net> wrote:
>
>
>
> it's hard for me to understand how this would
> apply immediately to X3D
>
>
>
> However you can specify actual content by an author producing some combinations of abstract and strictly defined indication of what visualization is desired, then sooner or later we want some human and machine readable user code to examine, validate, and execute. If the author’s intent is to produce an interactive environment then first the author needs to understand available objects and interactions. If we place an interactive humanoid in the environment, then we need to be able to define its features. The application to x3d happens when we form the links between aspects of the environment and the humanoid the author wishes to define using some interaction with some authoring system then the authoring system delivers some realtime simulation of the author’s intent. The x3d is in the parts where something relates intent to some constructed object somewhere and transportable user code. Where else than x3d can we find such a fine linkage between the concrete or abstract object the author wishes to produce and actual user code representation of some device or interaction?
>
> For humanoids, he author might need some hints to begin to define the character, then some more to define some standard behaviors. Where else than x3d is there the possibility to, for example, interrogate the use code in order to find usage of certain objects and observed events? When categorizing scene contents by examination of included hierarchies and events, where else than x3d would you find the base user code generator vocabulary with such a direct connection with actual graphics elements, whatever you might be searching for, and whatever author actions might be used to specify content and interactions?
>
>
>
> interpreter -> compiler -> Humanoid Animation
>
>
>
> For hanim, of course we ought be to be able to find ”kicking ball with skin” for example and find something, bring it to the system, and see if it can be used directly or modified. This is the part where there is data out there that must be captured and turned into some usable information, then analyzed and categorized into some knowledge, so we can find examples of usable humanoids that can be studied until they are wise enough for actual application of generating some more data. T every step of this our x3d basic stuff and hanim offers a link between what the author or character might be looking for and what is available. Whatever quack the author might use, there is probably a semantic method for finding it using terms for items and interactions, even though connections may depend upon commentary and metadata in the accessible catalog of connections.
>
> Thanks,
>
> Joe
>
>
>
>
>
> From: John Carlson
> Sent: Friday, July 17, 2020 2:19 PM
> To: Michael Turner
> Cc: Joe D Williams; X3D Semantic Web Working Group
> Subject: Re: interpreter -> compiler -> Humanoid Animation
>
>
>
> •step up on stump•
>
> I think what I was proposing was NSM for a semantic language of X3D.   I don’t think I made that clear.   Again, movies, symbols and shapes; what languages or mishmashes can handle all of them without a serious amount of work?   No, I don’t want to go backwards like the web did.   Now that the web has begun to catch up, what can we do with it?   What steps can we take to improve it?
>
>
>
> My goal here is to integrate vector, character and raster, among other things, or at least raise awareness.   PostScript/PDF has done it.   3D graphics has not typically supported text as a first class citizen, and the web and NSM have not typically supported 3D graphics as a first class citizen.   I don’t want to see further “mistakes” like that.   I want to lower barriers, not raise them.
>
>
>
> I believe that one of the things Bruce Garner, Nik Mitschkowitz, Don Vickers and Carolyn Wimple were working on was the integration of STEP and CGM with SGML.   I do not know why that was not made widespread.   Perhaps there was competition with HP and Adobe?   I am sorry I was not more clued into their project.   I understand now!
>
>
>
> NSM has more history than OWL/RDF/Turtle and should be considered!
>
>
>
> John
>
>
>
>
>
> On Fri, Jul 17, 2020 at 1:17 PM Michael Turner <michael.eugene.turner at gmail.com> wrote:
>
> So I skipped to the last paragraph, as John suggested. To the rest of
> you: John's understanding of Natural Semantic Metalanguage (NSM) is
> limited. From my more informed stance on NSM (and from some very
> limited acquaintance with X3D) can say categorically: X3D is not "the
> NSM of graphics."
>
> While I'm interested in attempting what I call Natural Semantic
> Programming (that is, programming built up in a declarative style, in
> natural language, and executed with constraint solvers, perhaps Prolog
> or ECLiPSe to start), it's hard for me to understand how this would
> apply immediately to X3D. Nor can I understand why any X3D standards
> developer or user would care.
>
> Any NSM-based computational framework that's capable of 3D graphical
> animation would probably need start with "what is a bit?" and build up
> from there, through floating point computation and matrix math. That's
> hardly a good starting point for anything like humanoid animation at
> this point. The main advantage of NSM would be to enable interlingual
> programming that's not just highly accessible not only to
> non-programmers, but interlingually so -- programs described in one
> natural language could be readily translatable to another.
>
> John tried to interest me in sign language, and fair enough: sign
> languages are natural languages. But to go from (say) English straight
> to some sign language seemed to freight my Natural Semantic
> Programming agenda with an elaborate graphical aspect at a time when
> that agenda was still just a twinkle in my (inner) eye.
>
> Long story short: you can safely ignore all this.
>
> Regards,
> Michael Turner
> Executive Director
> Project Persephone
> 1-25-33 Takadanobaba
> Shinjuku-ku Tokyo 169-0075
> Mobile: +81 (90) 5203-8682
> turner at projectpersephone.org
>
> Understand - http://www.projectpersephone.org/
> Join - http://www.facebook.com/groups/ProjectPersephone/
> Donate - http://www.patreon.com/ProjectPersephone
> Volunteer - https://github.com/ProjectPersephone
>
> "Love does not consist in gazing at each other, but in looking outward
> together in the same direction." -- Antoine de Saint-Exupéry
>
> On Thu, Jul 16, 2020 at 11:01 PM John Carlson <yottzumm at gmail.com> wrote:
> >
> >
> > Don, all, would it be possible to get the X3DUOM as RDF/Turtle (just as an interesting exercise)?  Thanks!
> >
> > Michael,
> >
> > TL;DR;  Just read last paragraph if you like.
> >
> > The subject basically describes all the technology I've been describing to you.  I have never achieved a complete compiler to machine code for a general language. It's possible that I've created a bytecode interpreter using the translator flat file format converted to compileable/interpretable C++ variable declarations. I did at one point, expand branches into the steps a branch would take as a C++ function, but went no further and backed out, primarily because I wanted to decompile the code.  I was able to compare the flat file to the decompiled code by re-persisting the programs, to verify the source code generator.
> >
> > This is how I am a languages guy, I guess.  I am not a big Natural Semantic Metalanguage (NSM) fan, but if NSM statements can be couched in the terms of persistent objects or class grammar (even list of lists...of primes--the obvious choice), and can fulfill the last paragraph, I would be interested in exploring the NSM concept for whatever purpose you want.
> >
> > As a side note, I state that I've been able to convert documents to lists of lists of words.  I merely used something like PDF->HTML, and converted HTML div's to JSON (JavaScript Object Notation) arrays. I wanted to do JSON translation-by-demonstration and I had an existing data source.  We can have a very large supply of documents represented as JSON arrays, if we need it. I realize that NSM-DALIA takes English.
> >
> > I guess we're dealing with a *semantic* object model and *semantic* graph with NSM.  The Web3D consortium is currently working with RDF/Turtle I believe.  There's also OWL/OWL2.  I should be able to provide you with around 3000 .ttl (RDF/Turtle) files in a single domain (X3D) translated from XML: https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples (try Online link, .ttl is on right for individual scenes).  Did you send me an NSM Bible at one point?  Can we translate RDF/Turtle to NSM?
> >
> > Ah yes! https://openresearch-repository.anu.edu.au/bitstream/1885/155252/2/What%20Christians%20Believe%20for%20Open%20Research%2020190204.pdf
> >
> > I'm not saying that RDF/Turtle is even desirable.  I just have a bunch of XML files (X3D scenes--non-RDF) I'd like to convert to their processable semantics.  We may have to improvise some of the semantics, that is, someone converts animations to semantics, such as "walk."  (I know this isn't an NSM prime.  I want something like "move from point to point upright at normal speed using legs") It's very likely we would have to have thousands of walk examples, and many more with "not walk", and do some kind of supervised learning with current technology that I've seen (I need to think about adversarial networks, here).  It may be possible to convert each NSM prime to an animation, IDK.  I tried both the dictionary approach (word->video) and the SignWriting approach (word->icons)
> >
> > I know NSM is about breaking down larger structures into simpler ones.  Say I'd like to reduce the X3D to VR/AR animations (assume for conversion to a 4D printer/animatronics).  If NSM-DALIA would help with that, I'm all ears.  If we can even create animations from NSM phrases (skipping the X3D), that would be awesome!  Can we create an animation of Towers of Hanoi from the NSM code in your Natural Semantic Programming paper?
> >
> > Yes, I realize you assigned me that very goal with translating a single word to an animation, which I have not achieved yet.
> >
> > In other words, we need to extend NSM to handle virtual worlds, or satellite worlds, not only human worlds, right? How does one describe a virtual world in NSM?  We have Towers of Hanoi.   We can obviously use any language we like as long as it can be reduced to NSM primes.
> >
> > The HAnim (Humanoid Animation) ISO standard: https://www.web3d.org/documents/specifications/19774/V2.0/index.html  has been ratified.  We now need examples of HAnimMotion (.bvh) elements.  This is my other job.
> >
> > The difficulty in all of this is translating from spatiotemporal semantics (geometry, coordinates, etc.) to/from NSM semantics. The NSM primes for this are:
> >
> > Time:  WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR SOME TIME, MOMENT
> >
> > Space: WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE, TOUCH (CONTACT)
> >
> > Thus, there are three kinds of "output," movies, symbols, and shapes. NSM handles symbols.   What handles movies and shapes?  X3D!  I don't care if it's VRML, XML, JSON, Turtle, Python, JavaScript, Java ... X3D is the NSM of graphics!  Now, how can NSM and X3D work together?
> >
> > John
>
>