[semantics-public] interpreter -> compiler -> Humanoid Animation

[Joseph D Williams]

➢ it's hard for me to understand how this would
apply immediately to X3D

However you can specify actual content by an author producing some combinations of abstract and strictly defined indication of what visualization is desired, then sooner or later we want some human and machine readable user code to examine, validate, and execute. If the author’s intent is to produce an interactive environment then first the author needs to understand available objects and interactions. If we place an interactive humanoid in the environment, then we need to be able to define its features. The application to x3d happens when we form the links between aspects of the environment and the humanoid the author wishes to define using some interaction with some authoring system then the authoring system delivers some realtime simulation of the author’s intent. The x3d is in the parts where something relates intent to some constructed object somewhere and transportable user code. Where else than x3d can we find such a fine linkage between the concrete or abstract object the author wishes to produce and actual user code representation of some device or interaction? 
For humanoids, he author might need some hints to begin to define the character, then some more to define some standard behaviors. Where else than x3d is there the possibility to, for example, interrogate the use code in order to find usage of certain objects and observed events? When categorizing scene contents by examination of included hierarchies and events, where else than x3d would you find the base user code generator vocabulary with such a direct connection with actual graphics elements, whatever you might be searching for, and whatever author actions might be used to specify content and interactions? 

➢ interpreter -> compiler -> Humanoid Animation

For hanim, of course we ought be to be able to find ”kicking ball with skin” for example and find something, bring it to the system, and see if it can be used directly or modified. This is the part where there is data out there that must be captured and turned into some usable information, then analyzed and categorized into some knowledge, so we can find examples of usable humanoids that can be studied until they are wise enough for actual application of generating some more data. T every step of this our x3d basic stuff and hanim offers a link between what the author or character might be looking for and what is available. Whatever quack the author might use, there is probably a semantic method for finding it using terms for items and interactions, even though connections may depend upon commentary and metadata in the accessible catalog of connections.
Thanks, 
Joe 


From: John Carlson
Sent: Friday, July 17, 2020 2:19 PM
To: Michael Turner
Cc: Joe D Williams; X3D Semantic Web Working Group
Subject: Re: interpreter -> compiler -> Humanoid Animation

•step up on stump•
I think what I was proposing was NSM for a semantic language of X3D.   I don’t think I made that clear.   Again, movies, symbols and shapes; what languages or mishmashes can handle all of them without a serious amount of work?   No, I don’t want to go backwards like the web did.   Now that the web has begun to catch up, what can we do with it?   What steps can we take to improve it?

My goal here is to integrate vector, character and raster, among other things, or at least raise awareness.   PostScript/PDF has done it.   3D graphics has not typically supported text as a first class citizen, and the web and NSM have not typically supported 3D graphics as a first class citizen.   I don’t want to see further “mistakes” like that.   I want to lower barriers, not raise them.

I believe that one of the things Bruce Garner, Nik Mitschkowitz, Don Vickers and Carolyn Wimple were working on was the integration of STEP and CGM with SGML.   I do not know why that was not made widespread.   Perhaps there was competition with HP and Adobe?   I am sorry I was not more clued into their project.   I understand now!

NSM has more history than OWL/RDF/Turtle and should be considered!

John


On Fri, Jul 17, 2020 at 1:17 PM Michael Turner <michael.eugene.turner at gmail.com> wrote:
So I skipped to the last paragraph, as John suggested. To the rest of
you: John's understanding of Natural Semantic Metalanguage (NSM) is
limited. From my more informed stance on NSM (and from some very
limited acquaintance with X3D) can say categorically: X3D is not "the
NSM of graphics."

While I'm interested in attempting what I call Natural Semantic
Programming (that is, programming built up in a declarative style, in
natural language, and executed with constraint solvers, perhaps Prolog
or ECLiPSe to start), it's hard for me to understand how this would
apply immediately to X3D. Nor can I understand why any X3D standards
developer or user would care.

Any NSM-based computational framework that's capable of 3D graphical
animation would probably need start with "what is a bit?" and build up
from there, through floating point computation and matrix math. That's
hardly a good starting point for anything like humanoid animation at
this point. The main advantage of NSM would be to enable interlingual
programming that's not just highly accessible not only to
non-programmers, but interlingually so -- programs described in one
natural language could be readily translatable to another.

John tried to interest me in sign language, and fair enough: sign
languages are natural languages. But to go from (say) English straight
to some sign language seemed to freight my Natural Semantic
Programming agenda with an elaborate graphical aspect at a time when
that agenda was still just a twinkle in my (inner) eye.

Long story short: you can safely ignore all this.

Regards,
Michael Turner
Executive Director
Project Persephone
1-25-33 Takadanobaba
Shinjuku-ku Tokyo 169-0075
Mobile: +81 (90) 5203-8682
turner at projectpersephone.org

Understand - http://www.projectpersephone.org/
Join - http://www.facebook.com/groups/ProjectPersephone/
Donate - http://www.patreon.com/ProjectPersephone
Volunteer - https://github.com/ProjectPersephone

"Love does not consist in gazing at each other, but in looking outward
together in the same direction." -- Antoine de Saint-Exupéry

On Thu, Jul 16, 2020 at 11:01 PM John Carlson <yottzumm at gmail.com> wrote:
>
>
> Don, all, would it be possible to get the X3DUOM as RDF/Turtle (just as an interesting exercise)?  Thanks!
>
> Michael,
>
> TL;DR;  Just read last paragraph if you like.
>
> The subject basically describes all the technology I've been describing to you.  I have never achieved a complete compiler to machine code for a general language. It's possible that I've created a bytecode interpreter using the translator flat file format converted to compileable/interpretable C++ variable declarations. I did at one point, expand branches into the steps a branch would take as a C++ function, but went no further and backed out, primarily because I wanted to decompile the code.  I was able to compare the flat file to the decompiled code by re-persisting the programs, to verify the source code generator.
>
> This is how I am a languages guy, I guess.  I am not a big Natural Semantic Metalanguage (NSM) fan, but if NSM statements can be couched in the terms of persistent objects or class grammar (even list of lists...of primes--the obvious choice), and can fulfill the last paragraph, I would be interested in exploring the NSM concept for whatever purpose you want.
>
> As a side note, I state that I've been able to convert documents to lists of lists of words.  I merely used something like PDF->HTML, and converted HTML div's to JSON (JavaScript Object Notation) arrays. I wanted to do JSON translation-by-demonstration and I had an existing data source.  We can have a very large supply of documents represented as JSON arrays, if we need it. I realize that NSM-DALIA takes English.
>
> I guess we're dealing with a *semantic* object model and *semantic* graph with NSM.  The Web3D consortium is currently working with RDF/Turtle I believe.  There's also OWL/OWL2.  I should be able to provide you with around 3000 .ttl (RDF/Turtle) files in a single domain (X3D) translated from XML: https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples (try Online link, .ttl is on right for individual scenes).  Did you send me an NSM Bible at one point?  Can we translate RDF/Turtle to NSM?
>
> Ah yes! https://openresearch-repository.anu.edu.au/bitstream/1885/155252/2/What%20Christians%20Believe%20for%20Open%20Research%2020190204.pdf
>
> I'm not saying that RDF/Turtle is even desirable.  I just have a bunch of XML files (X3D scenes--non-RDF) I'd like to convert to their processable semantics.  We may have to improvise some of the semantics, that is, someone converts animations to semantics, such as "walk."  (I know this isn't an NSM prime.  I want something like "move from point to point upright at normal speed using legs") It's very likely we would have to have thousands of walk examples, and many more with "not walk", and do some kind of supervised learning with current technology that I've seen (I need to think about adversarial networks, here).  It may be possible to convert each NSM prime to an animation, IDK.  I tried both the dictionary approach (word->video) and the SignWriting approach (word->icons)
>
> I know NSM is about breaking down larger structures into simpler ones.  Say I'd like to reduce the X3D to VR/AR animations (assume for conversion to a 4D printer/animatronics).  If NSM-DALIA would help with that, I'm all ears.  If we can even create animations from NSM phrases (skipping the X3D), that would be awesome!  Can we create an animation of Towers of Hanoi from the NSM code in your Natural Semantic Programming paper?
>
> Yes, I realize you assigned me that very goal with translating a single word to an animation, which I have not achieved yet.
>
> In other words, we need to extend NSM to handle virtual worlds, or satellite worlds, not only human worlds, right? How does one describe a virtual world in NSM?  We have Towers of Hanoi.   We can obviously use any language we like as long as it can be reduced to NSM primes.
>
> The HAnim (Humanoid Animation) ISO standard: https://www.web3d.org/documents/specifications/19774/V2.0/index.html  has been ratified.  We now need examples of HAnimMotion (.bvh) elements.  This is my other job.
>
> The difficulty in all of this is translating from spatiotemporal semantics (geometry, coordinates, etc.) to/from NSM semantics. The NSM primes for this are:
>
> Time:  WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR SOME TIME, MOMENT
>
> Space: WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE, TOUCH (CONTACT)
>
> Thus, there are three kinds of "output," movies, symbols, and shapes. NSM handles symbols.   What handles movies and shapes?  X3D!  I don't care if it's VRML, XML, JSON, Turtle, Python, JavaScript, Java ... X3D is the NSM of graphics!  Now, how can NSM and X3D work together?
>
> John

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/semantics-public_web3d.org/attachments/20200720/f05ed14d/attachment-0001.html>