[semantics-public] interpreter -> compiler -> Humanoid Animation

Tue Jul 21 06:30:50 PDT 2020

Repaired text below.

On Tue, Jul 21, 2020 at 8:29 AM John Carlson <yottzumm at gmail.com> wrote:

> Michael, have you looked recently at LiveCode script?   I believe it has
> an export to HTML5 now, besides running on practically every platform on
> the planet.   LiveCode is the descendant of
> HyperCard/HyperTalk/MetaCard/MetaTalk/xTalk.
>
> John
>
> On Tue, Jul 21, 2020 at 2:44 AM Michael Turner <
> michael.eugene.turner at gmail.com> wrote:
>
>> I didn't quite follow all that, Joseph, perhaps mostly because I'm
>> pretty ignorant when it comes to whatever magical powers X3D might
>> have.
>>
>> If it helps, skip all my meandering philosophy in this essay ("Natural
>> Semantic Programming") by searching on Towers of Hanoi.
>>
>>
>> https://docs.google.com/document/d/1CWpals3OBWgAC8OcWuWBhLoxjgpE-khxxT33NnFnzm8/edit
>>
>> There, I hypothesize a situation where a blind child prodigy listens
>> over the phone to the NSP program being read, a version that doesn't
>> have "Towers of Hanoi" named as such in the code, and realizes what
>> game it is -- and writes an animation into the code. I suppose that,
>> with enough NSM definition of what a human is, and how a human body is
>> constrained, the animation could include a human figure actually
>> moving the disks from peg to peg.
>>
>> To be sure, I have no complete parser for what I wrote there -- it's
>> an NSM-constrained pseudocode exercise. That said, NSM-DALIA
>>
>> https://github.com/Yakushima/NSM-DALIA
>>
>> might not be too far from parsing something very close to this
>> pseudocode and executing a specialized parse tree as Prolog. I got
>> NSM-DALIA working again (John helped a little at one point) and as far
>> as I know, it's the only working software that can parse NSM and
>> translate NSM to other natural-language NSM descriptions.
>>
>> My goals are two-fold:
>>
>> -- a programming language that uses natural language in a common-sense
>> way (though allowing for the more-or-less universal syntax of
>> mathematics where things truly need to be mathematically expressed)
>>
>> -- a system that depends on no /particular/ natural language. Programs
>> are represented as meanings, and are parsed into that semantic
>> representation and re-generated with appropriate vocabulary and
>> "natural-language syntactic sugar" as required, depending on the users
>> preferred or native language.
>>
>> I'm not at all sure what this could have to do with X3D. I think the
>> more natural first area of commercial application of Natural Semantic
>> Programming would be as a kind of "meta-COBOL" constraint-oriented
>> programming language, for enterprises that need to operate more
>> smoothly across language barriers. For 3D animation, I suppose that
>> internationally distributed machine repair would bring in some need
>> for graphics with constraint programming. For example, animate "here's
>> how you swap the old part out and put in a new one", with an animation
>> that allows the trainee to zoom and pan and even to focus in so far
>> that the he can see slow-motion/stop-motion finger-level details of a
>> repair or maintenance procedure.
>>
>> Regards,
>> Michael Turner
>> Executive Director
>> Project Persephone
>> 1-25-33 Takadanobaba
>> Shinjuku-ku Tokyo 169-0075
>> Mobile: +81 (90) 5203-8682
>> turner at projectpersephone.org
>>
>> Understand - http://www.projectpersephone.org/
>> Join - http://www.facebook.com/groups/ProjectPersephone/
>> Donate - http://www.patreon.com/ProjectPersephone
>> Volunteer - https://github.com/ProjectPersephone
>>
>> "Love does not consist in gazing at each other, but in looking outward
>> together in the same direction." -- Antoine de Saint-Exupéry
>>
>> On Mon, Jul 20, 2020 at 10:42 PM Joseph D Williams
>> <joedwil at earthlink.net> wrote:
>> >
>> >
>> >
>> > it's hard for me to understand how this would
>> > apply immediately to X3D
>> >
>> >
>> >
>> > However you can specify actual content by an author producing some
>> combinations of abstract and strictly defined indication of what
>> visualization is desired, then sooner or later we want some human and
>> machine readable user code to examine, validate, and execute. If the
>> author’s intent is to produce an interactive environment then first the
>> author needs to understand available objects and interactions. If we place
>> an interactive humanoid in the environment, then we need to be able to
>> define its features. The application to x3d happens when we form the links
>> between aspects of the environment and the humanoid the author wishes to
>> define using some interaction with some authoring system then the authoring
>> system delivers some realtime simulation of the author’s intent. The x3d is
>> in the parts where something relates intent to some constructed object
>> somewhere and transportable user code. Where else than x3d can we find such
>> a fine linkage between the concrete or abstract object the author wishes to
>> produce and actual user code representation of some device or interaction?
>> >
>> > For humanoids, he author might need some hints to begin to define the
>> character, then some more to define some standard behaviors. Where else
>> than x3d is there the possibility to, for example, interrogate the use code
>> in order to find usage of certain objects and observed events? When
>> categorizing scene contents by examination of included hierarchies and
>> events, where else than x3d would you find the base user code generator
>> vocabulary with such a direct connection with actual graphics elements,
>> whatever you might be searching for, and whatever author actions might be
>> used to specify content and interactions?
>> >
>> >
>> >
>> > interpreter -> compiler -> Humanoid Animation
>> >
>> >
>> >
>> > For hanim, of course we ought be to be able to find ”kicking ball with
>> skin” for example and find something, bring it to the system, and see if it
>> can be used directly or modified. This is the part where there is data out
>> there that must be captured and turned into some usable information, then
>> analyzed and categorized into some knowledge, so we can find examples of
>> usable humanoids that can be studied until they are wise enough for actual
>> application of generating some more data. T every step of this our x3d
>> basic stuff and hanim offers a link between what the author or character
>> might be looking for and what is available. Whatever quack the author might
>> use, there is probably a semantic method for finding it using terms for
>> items and interactions, even though connections may depend upon commentary
>> and metadata in the accessible catalog of connections.
>> >
>> > Thanks,
>> >
>> > Joe
>> >
>> >
>> >
>> >
>> >
>> > From: John Carlson
>> > Sent: Friday, July 17, 2020 2:19 PM
>> > To: Michael Turner
>> > Cc: Joe D Williams; X3D Semantic Web Working Group
>> > Subject: Re: interpreter -> compiler -> Humanoid Animation
>> >
>> >
>> >
>> > •step up on stump•
>> >
>> > I think what I was proposing was NSM for a semantic language of X3D.
>>  I don’t think I made that clear.   Again, movies, symbols and shapes; what
>> languages or mishmashes can handle all of them without a serious amount of
>> work?   No, I don’t want to go backwards like the web did.   Now that the
>> web has begun to catch up, what can we do with it?   What steps can we take
>> to improve it?
>> >
>> >
>> >
>> > My goal here is to integrate vector, character and raster, among other
>> things, or at least raise awareness.   PostScript/PDF has done it.   3D
>> graphics has not typically supported text as a first class citizen, and the
>> web and NSM have not typically supported 3D graphics as a first class
>> citizen.   I don’t want to see further “mistakes” like that.   I want to
>> lower barriers, not raise them.
>> >
>> >
>> >
>> > I believe that one of the things Bruce Garner, Nik Mitschkowitz, Don
>> Vickers and Carolyn Wimple were working on was the integration of STEP and
>> CGM with SGML.   I do not know why that was not made widespread.   Perhaps
>> there was competition with HP and Adobe?   I am sorry I was not more clued
>> into their project.   I understand now!
>> >
>> >
>> >
>> > NSM has more history than OWL/RDF/Turtle and should be considered!
>> >
>> >
>> >
>> > John
>> >
>> >
>> >
>> >
>> >
>> > On Fri, Jul 17, 2020 at 1:17 PM Michael Turner <
>> michael.eugene.turner at gmail.com> wrote:
>> >
>> > So I skipped to the last paragraph, as John suggested. To the rest of
>> > you: John's understanding of Natural Semantic Metalanguage (NSM) is
>> > limited. From my more informed stance on NSM (and from some very
>> > limited acquaintance with X3D) can say categorically: X3D is not "the
>> > NSM of graphics."
>> >
>> > While I'm interested in attempting what I call Natural Semantic
>> > Programming (that is, programming built up in a declarative style, in
>> > natural language, and executed with constraint solvers, perhaps Prolog
>> > or ECLiPSe to start), it's hard for me to understand how this would
>> > apply immediately to X3D. Nor can I understand why any X3D standards
>> > developer or user would care.
>> >
>> > Any NSM-based computational framework that's capable of 3D graphical
>> > animation would probably need start with "what is a bit?" and build up
>> > from there, through floating point computation and matrix math. That's
>> > hardly a good starting point for anything like humanoid animation at
>> > this point. The main advantage of NSM would be to enable interlingual
>> > programming that's not just highly accessible not only to
>> > non-programmers, but interlingually so -- programs described in one
>> > natural language could be readily translatable to another.
>> >
>> > John tried to interest me in sign language, and fair enough: sign
>> > languages are natural languages. But to go from (say) English straight
>> > to some sign language seemed to freight my Natural Semantic
>> > Programming agenda with an elaborate graphical aspect at a time when
>> > that agenda was still just a twinkle in my (inner) eye.
>> >
>> > Long story short: you can safely ignore all this.
>> >
>> > Regards,
>> > Michael Turner
>> > Executive Director
>> > Project Persephone
>> > 1-25-33 Takadanobaba
>> > Shinjuku-ku Tokyo 169-0075
>> > Mobile: +81 (90) 5203-8682
>> > turner at projectpersephone.org
>> >
>> > Understand - http://www.projectpersephone.org/
>> > Join - http://www.facebook.com/groups/ProjectPersephone/
>> > Donate - http://www.patreon.com/ProjectPersephone
>> > Volunteer - https://github.com/ProjectPersephone
>> >
>> > "Love does not consist in gazing at each other, but in looking outward
>> > together in the same direction." -- Antoine de Saint-Exupéry
>> >
>> > On Thu, Jul 16, 2020 at 11:01 PM John Carlson <yottzumm at gmail.com>
>> wrote:
>> > >
>> > >
>> > > Don, all, would it be possible to get the X3DUOM as RDF/Turtle (just
>> as an interesting exercise)?  Thanks!
>> > >
>> > > Michael,
>> > >
>> > > TL;DR;  Just read last paragraph if you like.
>> > >
>> > > The subject basically describes all the technology I've been
>> describing to you.  I have never achieved a complete compiler to machine
>> code for a general language. It's possible that I've created a bytecode
>> interpreter using the translator flat file format converted to
>> compileable/interpretable C++ variable declarations. I did at one point,
>> expand branches into the steps a branch would take as a C++ function, but
>> went no further and backed out, primarily because I wanted to decompile the
>> code.  I was able to compare the flat file to the decompiled code by
>> re-persisting the programs, to verify the source code generator.
>> > >
>> > > This is how I am a languages guy, I guess.  I am not a big Natural
>> Semantic Metalanguage (NSM) fan, but if NSM statements can be couched in
>> the terms of persistent objects or class grammar (even list of lists...of
>> primes--the obvious choice), and can fulfill the last paragraph, I would be
>> interested in exploring the NSM concept for whatever purpose you want.
>> > >
>> > > As a side note, I state that I've been able to convert documents to
>> lists of lists of words.  I merely used something like PDF->HTML, and
>> converted HTML div's to JSON (JavaScript Object Notation) arrays. I wanted
>> to do JSON translation-by-demonstration and I had an existing data source.
>> We can have a very large supply of documents represented as JSON arrays, if
>> we need it. I realize that NSM-DALIA takes English.
>> > >
>> > > I guess we're dealing with a *semantic* object model and *semantic*
>> graph with NSM.  The Web3D consortium is currently working with RDF/Turtle
>> I believe.  There's also OWL/OWL2.  I should be able to provide you with
>> around 3000 .ttl (RDF/Turtle) files in a single domain (X3D) translated
>> from XML:
>> https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>> (try Online link, .ttl is on right for individual scenes).  Did you send me
>> an NSM Bible at one point?  Can we translate RDF/Turtle to NSM?
>> > >
>> > > Ah yes!
>> https://openresearch-repository.anu.edu.au/bitstream/1885/155252/2/What%20Christians%20Believe%20for%20Open%20Research%2020190204.pdf
>> > >
>> > > I'm not saying that RDF/Turtle is even desirable.  I just have a
>> bunch of XML files (X3D scenes--non-RDF) I'd like to convert to their
>> processable semantics.  We may have to improvise some of the semantics,
>> that is, someone converts animations to semantics, such as "walk."  (I know
>> this isn't an NSM prime.  I want something like "move from point to point
>> upright at normal speed using legs") It's very likely we would have to have
>> thousands of walk examples, and many more with "not walk", and do some kind
>> of supervised learning with current technology that I've seen (I need to
>> think about adversarial networks, here).  It may be possible to convert
>> each NSM prime to an animation, IDK.  I tried both the dictionary approach
>> (word->video) and the SignWriting approach (word->icons)
>> > >
>> > > I know NSM is about breaking down larger structures into simpler
>> ones.  Say I'd like to reduce the X3D to VR/AR animations (assume for
>> conversion to a 4D printer/animatronics).  If NSM-DALIA would help with
>> that, I'm all ears.  If we can even create animations from NSM phrases
>> (skipping the X3D), that would be awesome!  Can we create an animation of
>> Towers of Hanoi from the NSM code in your Natural Semantic Programming
>> paper?
>> > >
>> > > Yes, I realize you assigned me that very goal with translating a
>> single word to an animation, which I have not achieved yet.
>> > >
>> > > In other words, we need to extend NSM to handle virtual worlds, or
>> satellite worlds, not only human worlds, right? How does one describe a
>> virtual world in NSM?  We have Towers of Hanoi.   We can obviously use any
>> language we like as long as it can be reduced to NSM primes.
>> > >
>> > > The HAnim (Humanoid Animation) ISO standard:
>> https://www.web3d.org/documents/specifications/19774/V2.0/index.html
>> has been ratified.  We now need examples of HAnimMotion (.bvh) elements.
>> This is my other job.
>> > >
>> > > The difficulty in all of this is translating from spatiotemporal
>> semantics (geometry, coordinates, etc.) to/from NSM semantics. The NSM
>> primes for this are:
>> > >
>> > > Time:  WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR
>> SOME TIME, MOMENT
>> > >
>> > > Space: WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE,
>> TOUCH (CONTACT)
>> > >
>> > > Thus, there are three kinds of "output," movies, symbols, and shapes.
>> NSM handles symbols.   What handles movies and shapes?  X3D!  I don't care
>> if it's VRML, XML, JSON, Turtle, Python, JavaScript, Java ... X3D is the
>> NSM of graphics!  Now, how can NSM and X3D work together?
>> > >
>> > > John
>> >
>> >
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/semantics-public_web3d.org/attachments/20200721/e7bf4521/attachment-0001.html>