[semantics-public] interpreter -> compiler -> Humanoid Animation

[John Carlson]

Along the lines of 3D environments for programming:
https://en.wikipedia.org/wiki/ToonTalk (I'm not sure if it's true 3D).

I don't know why I research this stuff, but it may come in handy!


On Tue, Jul 21, 2020 at 8:54 AM John Carlson <yottzumm at gmail.com> wrote:

> https://forums.livecode.com/viewtopic.php?t=21079
>
> One possibility.
>
> On Tue, Jul 21, 2020 at 8:42 AM John Carlson <yottzumm at gmail.com> wrote:
>
>> Might I suggest one convert the X3DUOM and scene files to Prolog or
>> LiveCode?   Does Prolog have a 3D extension?   Does LiveCode have a 3D
>> extension?
>>
>> I do not know these suggestions are reasonable or not.   I’m just opening
>> a door of possibility.
>>
>> Don, I found the X3DUOM as Turtle.
>>
>> John
>>
>> On Mon, Jul 20, 2020 at 3:42 PM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>>
>>>
>>>    - it's hard for me to understand how this would
>>>    apply immediately to X3D
>>>
>>>
>>>
>>> However you can specify actual content by an author producing some
>>> combinations of abstract and strictly defined indication of what
>>> visualization is desired, then sooner or later we want some human and
>>> machine readable user code to examine, validate, and execute. If the
>>> author’s intent is to produce an interactive environment then first the
>>> author needs to understand available objects and interactions. If we place
>>> an interactive humanoid in the environment, then we need to be able to
>>> define its features. The application to x3d happens when we form the links
>>> between aspects of the environment and the humanoid the author wishes to
>>> define using some interaction with some authoring system then the authoring
>>> system delivers some realtime simulation of the author’s intent. The x3d is
>>> in the parts where something relates intent to some constructed object
>>> somewhere and transportable user code. Where else than x3d can we find such
>>> a fine linkage between the concrete or abstract object the author wishes to
>>> produce and actual user code representation of some device or interaction?
>>>
>>> For humanoids, he author might need some hints to begin to define the
>>> character, then some more to define some standard behaviors. Where else
>>> than x3d is there the possibility to, for example, interrogate the use code
>>> in order to find usage of certain objects and observed events? When
>>> categorizing scene contents by examination of included hierarchies and
>>> events, where else than x3d would you find the base user code generator
>>> vocabulary with such a direct connection with actual graphics elements,
>>> whatever you might be searching for, and whatever author actions might be
>>> used to specify content and interactions?
>>>
>>>
>>>
>>>    - interpreter -> compiler -> Humanoid Animation
>>>
>>>
>>>
>>> For hanim, of course we ought be to be able to find ”kicking ball with
>>> skin” for example and find something, bring it to the system, and see if it
>>> can be used directly or modified. This is the part where there is data out
>>> there that must be captured and turned into some usable information, then
>>> analyzed and categorized into some knowledge, so we can find examples of
>>> usable humanoids that can be studied until they are wise enough for actual
>>> application of generating some more data. T every step of this our x3d
>>> basic stuff and hanim offers a link between what the author or character
>>> might be looking for and what is available. Whatever quack the author might
>>> use, there is probably a semantic method for finding it using terms for
>>> items and interactions, even though connections may depend upon commentary
>>> and metadata in the accessible catalog of connections.
>>>
>>> Thanks,
>>>
>>> Joe
>>>
>>>
>>>
>>>
>>>
>>> *From: *John Carlson <yottzumm at gmail.com>
>>> *Sent: *Friday, July 17, 2020 2:19 PM
>>> *To: *Michael Turner <michael.eugene.turner at gmail.com>
>>> *Cc: *Joe D Williams <joedwil at earthlink.net>; X3D Semantic Web Working
>>> Group <semantics-public at web3d.org>
>>> *Subject: *Re: interpreter -> compiler -> Humanoid Animation
>>>
>>>
>>>
>>> •step up on stump•
>>>
>>> I think what I was proposing was NSM for a semantic language of X3D.   I
>>> don’t think I made that clear.   Again, movies, symbols and shapes; what
>>> languages or mishmashes can handle all of them without a serious amount of
>>> work?   No, I don’t want to go backwards like the web did.   Now that the
>>> web has begun to catch up, what can we do with it?   What steps can we take
>>> to improve it?
>>>
>>>
>>>
>>> My goal here is to integrate vector, character and raster, among other
>>> things, or at least raise awareness.   PostScript/PDF has done it.   3D
>>> graphics has not typically supported text as a first class citizen, and the
>>> web and NSM have not typically supported 3D graphics as a first class
>>> citizen.   I don’t want to see further “mistakes” like that.   I want to
>>> lower barriers, not raise them.
>>>
>>>
>>>
>>> I believe that one of the things Bruce Garner, Nik Mitschkowitz, Don
>>> Vickers and Carolyn Wimple were working on was the integration of STEP and
>>> CGM with SGML.   I do not know why that was not made widespread.   Perhaps
>>> there was competition with HP and Adobe?   I am sorry I was not more clued
>>> into their project.   I understand now!
>>>
>>>
>>>
>>> NSM has more history than OWL/RDF/Turtle and should be considered!
>>>
>>>
>>>
>>> John
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jul 17, 2020 at 1:17 PM Michael Turner <
>>> michael.eugene.turner at gmail.com> wrote:
>>>
>>> So I skipped to the last paragraph, as John suggested. To the rest of
>>> you: John's understanding of Natural Semantic Metalanguage (NSM) is
>>> limited. From my more informed stance on NSM (and from some very
>>> limited acquaintance with X3D) can say categorically: X3D is not "the
>>> NSM of graphics."
>>>
>>> While I'm interested in attempting what I call Natural Semantic
>>> Programming (that is, programming built up in a declarative style, in
>>> natural language, and executed with constraint solvers, perhaps Prolog
>>> or ECLiPSe to start), it's hard for me to understand how this would
>>> apply immediately to X3D. Nor can I understand why any X3D standards
>>> developer or user would care.
>>>
>>> Any NSM-based computational framework that's capable of 3D graphical
>>> animation would probably need start with "what is a bit?" and build up
>>> from there, through floating point computation and matrix math. That's
>>> hardly a good starting point for anything like humanoid animation at
>>> this point. The main advantage of NSM would be to enable interlingual
>>> programming that's not just highly accessible not only to
>>> non-programmers, but interlingually so -- programs described in one
>>> natural language could be readily translatable to another.
>>>
>>> John tried to interest me in sign language, and fair enough: sign
>>> languages are natural languages. But to go from (say) English straight
>>> to some sign language seemed to freight my Natural Semantic
>>> Programming agenda with an elaborate graphical aspect at a time when
>>> that agenda was still just a twinkle in my (inner) eye.
>>>
>>> Long story short: you can safely ignore all this.
>>>
>>> Regards,
>>> Michael Turner
>>> Executive Director
>>> Project Persephone
>>> 1-25-33 Takadanobaba
>>> Shinjuku-ku Tokyo 169-0075
>>> Mobile: +81 (90) 5203-8682
>>> turner at projectpersephone.org
>>>
>>> Understand - http://www.projectpersephone.org/
>>> Join - http://www.facebook.com/groups/ProjectPersephone/
>>> Donate - http://www.patreon.com/ProjectPersephone
>>> Volunteer - https://github.com/ProjectPersephone
>>>
>>> "Love does not consist in gazing at each other, but in looking outward
>>> together in the same direction." -- Antoine de Saint-Exupéry
>>>
>>> On Thu, Jul 16, 2020 at 11:01 PM John Carlson <yottzumm at gmail.com>
>>> wrote:
>>> >
>>> >
>>> > Don, all, would it be possible to get the X3DUOM as RDF/Turtle (just
>>> as an interesting exercise)?  Thanks!
>>> >
>>> > Michael,
>>> >
>>> > TL;DR;  Just read last paragraph if you like.
>>> >
>>> > The subject basically describes all the technology I've been
>>> describing to you.  I have never achieved a complete compiler to machine
>>> code for a general language. It's possible that I've created a bytecode
>>> interpreter using the translator flat file format converted to
>>> compileable/interpretable C++ variable declarations. I did at one point,
>>> expand branches into the steps a branch would take as a C++ function, but
>>> went no further and backed out, primarily because I wanted to decompile the
>>> code.  I was able to compare the flat file to the decompiled code by
>>> re-persisting the programs, to verify the source code generator.
>>> >
>>> > This is how I am a languages guy, I guess.  I am not a big Natural
>>> Semantic Metalanguage (NSM) fan, but if NSM statements can be couched in
>>> the terms of persistent objects or class grammar (even list of lists...of
>>> primes--the obvious choice), and can fulfill the last paragraph, I would be
>>> interested in exploring the NSM concept for whatever purpose you want.
>>> >
>>> > As a side note, I state that I've been able to convert documents to
>>> lists of lists of words.  I merely used something like PDF->HTML, and
>>> converted HTML div's to JSON (JavaScript Object Notation) arrays. I wanted
>>> to do JSON translation-by-demonstration and I had an existing data source.
>>> We can have a very large supply of documents represented as JSON arrays, if
>>> we need it. I realize that NSM-DALIA takes English.
>>> >
>>> > I guess we're dealing with a *semantic* object model and *semantic*
>>> graph with NSM.  The Web3D consortium is currently working with RDF/Turtle
>>> I believe.  There's also OWL/OWL2.  I should be able to provide you with
>>> around 3000 .ttl (RDF/Turtle) files in a single domain (X3D) translated
>>> from XML:
>>> https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>>> (try Online link, .ttl is on right for individual scenes).  Did you send me
>>> an NSM Bible at one point?  Can we translate RDF/Turtle to NSM?
>>> >
>>> > Ah yes!
>>> https://openresearch-repository.anu.edu.au/bitstream/1885/155252/2/What%20Christians%20Believe%20for%20Open%20Research%2020190204.pdf
>>> >
>>> > I'm not saying that RDF/Turtle is even desirable.  I just have a bunch
>>> of XML files (X3D scenes--non-RDF) I'd like to convert to their processable
>>> semantics.  We may have to improvise some of the semantics, that is,
>>> someone converts animations to semantics, such as "walk."  (I know this
>>> isn't an NSM prime.  I want something like "move from point to point
>>> upright at normal speed using legs") It's very likely we would have to have
>>> thousands of walk examples, and many more with "not walk", and do some kind
>>> of supervised learning with current technology that I've seen (I need to
>>> think about adversarial networks, here).  It may be possible to convert
>>> each NSM prime to an animation, IDK.  I tried both the dictionary approach
>>> (word->video) and the SignWriting approach (word->icons)
>>> >
>>> > I know NSM is about breaking down larger structures into simpler
>>> ones.  Say I'd like to reduce the X3D to VR/AR animations (assume for
>>> conversion to a 4D printer/animatronics).  If NSM-DALIA would help with
>>> that, I'm all ears.  If we can even create animations from NSM phrases
>>> (skipping the X3D), that would be awesome!  Can we create an animation of
>>> Towers of Hanoi from the NSM code in your Natural Semantic Programming
>>> paper?
>>> >
>>> > Yes, I realize you assigned me that very goal with translating a
>>> single word to an animation, which I have not achieved yet.
>>> >
>>> > In other words, we need to extend NSM to handle virtual worlds, or
>>> satellite worlds, not only human worlds, right? How does one describe a
>>> virtual world in NSM?  We have Towers of Hanoi.   We can obviously use any
>>> language we like as long as it can be reduced to NSM primes.
>>> >
>>> > The HAnim (Humanoid Animation) ISO standard:
>>> https://www.web3d.org/documents/specifications/19774/V2.0/index.html
>>> has been ratified.  We now need examples of HAnimMotion (.bvh) elements.
>>> This is my other job.
>>> >
>>> > The difficulty in all of this is translating from spatiotemporal
>>> semantics (geometry, coordinates, etc.) to/from NSM semantics. The NSM
>>> primes for this are:
>>> >
>>> > Time:  WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR
>>> SOME TIME, MOMENT
>>> >
>>> > Space: WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE, TOUCH
>>> (CONTACT)
>>> >
>>> > Thus, there are three kinds of "output," movies, symbols, and shapes.
>>> NSM handles symbols.   What handles movies and shapes?  X3D!  I don't care
>>> if it's VRML, XML, JSON, Turtle, Python, JavaScript, Java ... X3D is the
>>> NSM of graphics!  Now, how can NSM and X3D work together?
>>> >
>>> > John
>>>
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/semantics-public_web3d.org/attachments/20200721/7c44198a/attachment-0001.html>