[semantics-public] interpreter -> compiler -> Humanoid Animation

Tue Jul 21 06:29:10 PDT 2020

Michael, have you looked recently at LiveCode script?   I believe it has an
export to HTML5 now, besides running under practically every platform on
the planet.   Live code is the descendant of
HyperCard/HyperTalk/MetaCard/MetaTalk/xTalk.

John

On Tue, Jul 21, 2020 at 2:44 AM Michael Turner <
michael.eugene.turner at gmail.com> wrote:

> I didn't quite follow all that, Joseph, perhaps mostly because I'm
> pretty ignorant when it comes to whatever magical powers X3D might
> have.
>
> If it helps, skip all my meandering philosophy in this essay ("Natural
> Semantic Programming") by searching on Towers of Hanoi.
>
>
> https://docs.google.com/document/d/1CWpals3OBWgAC8OcWuWBhLoxjgpE-khxxT33NnFnzm8/edit
>
> There, I hypothesize a situation where a blind child prodigy listens
> over the phone to the NSP program being read, a version that doesn't
> have "Towers of Hanoi" named as such in the code, and realizes what
> game it is -- and writes an animation into the code. I suppose that,
> with enough NSM definition of what a human is, and how a human body is
> constrained, the animation could include a human figure actually
> moving the disks from peg to peg.
>
> To be sure, I have no complete parser for what I wrote there -- it's
> an NSM-constrained pseudocode exercise. That said, NSM-DALIA
>
> https://github.com/Yakushima/NSM-DALIA
>
> might not be too far from parsing something very close to this
> pseudocode and executing a specialized parse tree as Prolog. I got
> NSM-DALIA working again (John helped a little at one point) and as far
> as I know, it's the only working software that can parse NSM and
> translate NSM to other natural-language NSM descriptions.
>
> My goals are two-fold:
>
> -- a programming language that uses natural language in a common-sense
> way (though allowing for the more-or-less universal syntax of
> mathematics where things truly need to be mathematically expressed)
>
> -- a system that depends on no /particular/ natural language. Programs
> are represented as meanings, and are parsed into that semantic
> representation and re-generated with appropriate vocabulary and
> "natural-language syntactic sugar" as required, depending on the users
> preferred or native language.
>
> I'm not at all sure what this could have to do with X3D. I think the
> more natural first area of commercial application of Natural Semantic
> Programming would be as a kind of "meta-COBOL" constraint-oriented
> programming language, for enterprises that need to operate more
> smoothly across language barriers. For 3D animation, I suppose that
> internationally distributed machine repair would bring in some need
> for graphics with constraint programming. For example, animate "here's
> how you swap the old part out and put in a new one", with an animation
> that allows the trainee to zoom and pan and even to focus in so far
> that the he can see slow-motion/stop-motion finger-level details of a
> repair or maintenance procedure.
>
> Regards,
> Michael Turner
> Executive Director
> Project Persephone
> 1-25-33 Takadanobaba
> Shinjuku-ku Tokyo 169-0075
> Mobile: +81 (90) 5203-8682
> turner at projectpersephone.org
>
> Understand - http://www.projectpersephone.org/
> Join - http://www.facebook.com/groups/ProjectPersephone/
> Donate - http://www.patreon.com/ProjectPersephone
> Volunteer - https://github.com/ProjectPersephone
>
> "Love does not consist in gazing at each other, but in looking outward
> together in the same direction." -- Antoine de Saint-Exupéry
>
> On Mon, Jul 20, 2020 at 10:42 PM Joseph D Williams
> <joedwil at earthlink.net> wrote:
> >
> >
> >
> > it's hard for me to understand how this would
> > apply immediately to X3D
> >
> >
> >
> > However you can specify actual content by an author producing some
> combinations of abstract and strictly defined indication of what
> visualization is desired, then sooner or later we want some human and
> machine readable user code to examine, validate, and execute. If the
> author’s intent is to produce an interactive environment then first the
> author needs to understand available objects and interactions. If we place
> an interactive humanoid in the environment, then we need to be able to
> define its features. The application to x3d happens when we form the links
> between aspects of the environment and the humanoid the author wishes to
> define using some interaction with some authoring system then the authoring
> system delivers some realtime simulation of the author’s intent. The x3d is
> in the parts where something relates intent to some constructed object
> somewhere and transportable user code. Where else than x3d can we find such
> a fine linkage between the concrete or abstract object the author wishes to
> produce and actual user code representation of some device or interaction?
> >
> > For humanoids, he author might need some hints to begin to define the
> character, then some more to define some standard behaviors. Where else
> than x3d is there the possibility to, for example, interrogate the use code
> in order to find usage of certain objects and observed events? When
> categorizing scene contents by examination of included hierarchies and
> events, where else than x3d would you find the base user code generator
> vocabulary with such a direct connection with actual graphics elements,
> whatever you might be searching for, and whatever author actions might be
> used to specify content and interactions?
> >
> >
> >
> > interpreter -> compiler -> Humanoid Animation
> >
> >
> >
> > For hanim, of course we ought be to be able to find ”kicking ball with
> skin” for example and find something, bring it to the system, and see if it
> can be used directly or modified. This is the part where there is data out
> there that must be captured and turned into some usable information, then
> analyzed and categorized into some knowledge, so we can find examples of
> usable humanoids that can be studied until they are wise enough for actual
> application of generating some more data. T every step of this our x3d
> basic stuff and hanim offers a link between what the author or character
> might be looking for and what is available. Whatever quack the author might
> use, there is probably a semantic method for finding it using terms for
> items and interactions, even though connections may depend upon commentary
> and metadata in the accessible catalog of connections.
> >
> > Thanks,
> >
> > Joe
> >
> >
> >
> >
> >
> > From: John Carlson
> > Sent: Friday, July 17, 2020 2:19 PM
> > To: Michael Turner
> > Cc: Joe D Williams; X3D Semantic Web Working Group
> > Subject: Re: interpreter -> compiler -> Humanoid Animation
> >
> >
> >
> > •step up on stump•
> >
> > I think what I was proposing was NSM for a semantic language of X3D.   I
> don’t think I made that clear.   Again, movies, symbols and shapes; what
> languages or mishmashes can handle all of them without a serious amount of
> work?   No, I don’t want to go backwards like the web did.   Now that the
> web has begun to catch up, what can we do with it?   What steps can we take
> to improve it?
> >
> >
> >
> > My goal here is to integrate vector, character and raster, among other
> things, or at least raise awareness.   PostScript/PDF has done it.   3D
> graphics has not typically supported text as a first class citizen, and the
> web and NSM have not typically supported 3D graphics as a first class
> citizen.   I don’t want to see further “mistakes” like that.   I want to
> lower barriers, not raise them.
> >
> >
> >
> > I believe that one of the things Bruce Garner, Nik Mitschkowitz, Don
> Vickers and Carolyn Wimple were working on was the integration of STEP and
> CGM with SGML.   I do not know why that was not made widespread.   Perhaps
> there was competition with HP and Adobe?   I am sorry I was not more clued
> into their project.   I understand now!
> >
> >
> >
> > NSM has more history than OWL/RDF/Turtle and should be considered!
> >
> >
> >
> > John
> >
> >
> >
> >
> >
> > On Fri, Jul 17, 2020 at 1:17 PM Michael Turner <
> michael.eugene.turner at gmail.com> wrote:
> >
> > So I skipped to the last paragraph, as John suggested. To the rest of
> > you: John's understanding of Natural Semantic Metalanguage (NSM) is
> > limited. From my more informed stance on NSM (and from some very
> > limited acquaintance with X3D) can say categorically: X3D is not "the
> > NSM of graphics."
> >
> > While I'm interested in attempting what I call Natural Semantic
> > Programming (that is, programming built up in a declarative style, in
> > natural language, and executed with constraint solvers, perhaps Prolog
> > or ECLiPSe to start), it's hard for me to understand how this would
> > apply immediately to X3D. Nor can I understand why any X3D standards
> > developer or user would care.
> >
> > Any NSM-based computational framework that's capable of 3D graphical
> > animation would probably need start with "what is a bit?" and build up
> > from there, through floating point computation and matrix math. That's
> > hardly a good starting point for anything like humanoid animation at
> > this point. The main advantage of NSM would be to enable interlingual
> > programming that's not just highly accessible not only to
> > non-programmers, but interlingually so -- programs described in one
> > natural language could be readily translatable to another.
> >
> > John tried to interest me in sign language, and fair enough: sign
> > languages are natural languages. But to go from (say) English straight
> > to some sign language seemed to freight my Natural Semantic
> > Programming agenda with an elaborate graphical aspect at a time when
> > that agenda was still just a twinkle in my (inner) eye.
> >
> > Long story short: you can safely ignore all this.
> >
> > Regards,
> > Michael Turner
> > Executive Director
> > Project Persephone
> > 1-25-33 Takadanobaba
> > Shinjuku-ku Tokyo 169-0075
> > Mobile: +81 (90) 5203-8682
> > turner at projectpersephone.org
> >
> > Understand - http://www.projectpersephone.org/
> > Join - http://www.facebook.com/groups/ProjectPersephone/
> > Donate - http://www.patreon.com/ProjectPersephone
> > Volunteer - https://github.com/ProjectPersephone
> >
> > "Love does not consist in gazing at each other, but in looking outward
> > together in the same direction." -- Antoine de Saint-Exupéry
> >
> > On Thu, Jul 16, 2020 at 11:01 PM John Carlson <yottzumm at gmail.com>
> wrote:
> > >
> > >
> > > Don, all, would it be possible to get the X3DUOM as RDF/Turtle (just
> as an interesting exercise)?  Thanks!
> > >
> > > Michael,
> > >
> > > TL;DR;  Just read last paragraph if you like.
> > >
> > > The subject basically describes all the technology I've been
> describing to you.  I have never achieved a complete compiler to machine
> code for a general language. It's possible that I've created a bytecode
> interpreter using the translator flat file format converted to
> compileable/interpretable C++ variable declarations. I did at one point,
> expand branches into the steps a branch would take as a C++ function, but
> went no further and backed out, primarily because I wanted to decompile the
> code.  I was able to compare the flat file to the decompiled code by
> re-persisting the programs, to verify the source code generator.
> > >
> > > This is how I am a languages guy, I guess.  I am not a big Natural
> Semantic Metalanguage (NSM) fan, but if NSM statements can be couched in
> the terms of persistent objects or class grammar (even list of lists...of
> primes--the obvious choice), and can fulfill the last paragraph, I would be
> interested in exploring the NSM concept for whatever purpose you want.
> > >
> > > As a side note, I state that I've been able to convert documents to
> lists of lists of words.  I merely used something like PDF->HTML, and
> converted HTML div's to JSON (JavaScript Object Notation) arrays. I wanted
> to do JSON translation-by-demonstration and I had an existing data source.
> We can have a very large supply of documents represented as JSON arrays, if
> we need it. I realize that NSM-DALIA takes English.
> > >
> > > I guess we're dealing with a *semantic* object model and *semantic*
> graph with NSM.  The Web3D consortium is currently working with RDF/Turtle
> I believe.  There's also OWL/OWL2.  I should be able to provide you with
> around 3000 .ttl (RDF/Turtle) files in a single domain (X3D) translated
> from XML:
> https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
> (try Online link, .ttl is on right for individual scenes).  Did you send me
> an NSM Bible at one point?  Can we translate RDF/Turtle to NSM?
> > >
> > > Ah yes!
> https://openresearch-repository.anu.edu.au/bitstream/1885/155252/2/What%20Christians%20Believe%20for%20Open%20Research%2020190204.pdf
> > >
> > > I'm not saying that RDF/Turtle is even desirable.  I just have a bunch
> of XML files (X3D scenes--non-RDF) I'd like to convert to their processable
> semantics.  We may have to improvise some of the semantics, that is,
> someone converts animations to semantics, such as "walk."  (I know this
> isn't an NSM prime.  I want something like "move from point to point
> upright at normal speed using legs") It's very likely we would have to have
> thousands of walk examples, and many more with "not walk", and do some kind
> of supervised learning with current technology that I've seen (I need to
> think about adversarial networks, here).  It may be possible to convert
> each NSM prime to an animation, IDK.  I tried both the dictionary approach
> (word->video) and the SignWriting approach (word->icons)
> > >
> > > I know NSM is about breaking down larger structures into simpler
> ones.  Say I'd like to reduce the X3D to VR/AR animations (assume for
> conversion to a 4D printer/animatronics).  If NSM-DALIA would help with
> that, I'm all ears.  If we can even create animations from NSM phrases
> (skipping the X3D), that would be awesome!  Can we create an animation of
> Towers of Hanoi from the NSM code in your Natural Semantic Programming
> paper?
> > >
> > > Yes, I realize you assigned me that very goal with translating a
> single word to an animation, which I have not achieved yet.
> > >
> > > In other words, we need to extend NSM to handle virtual worlds, or
> satellite worlds, not only human worlds, right? How does one describe a
> virtual world in NSM?  We have Towers of Hanoi.   We can obviously use any
> language we like as long as it can be reduced to NSM primes.
> > >
> > > The HAnim (Humanoid Animation) ISO standard:
> https://www.web3d.org/documents/specifications/19774/V2.0/index.html  has
> been ratified.  We now need examples of HAnimMotion (.bvh) elements.  This
> is my other job.
> > >
> > > The difficulty in all of this is translating from spatiotemporal
> semantics (geometry, coordinates, etc.) to/from NSM semantics. The NSM
> primes for this are:
> > >
> > > Time:  WHEN/TIME, NOW, BEFORE, AFTER, A LONG TIME, A SHORT TIME, FOR
> SOME TIME, MOMENT
> > >
> > > Space: WHERE/PLACE, HERE, ABOVE, BELOW, FAR, NEAR, SIDE, INSIDE, TOUCH
> (CONTACT)
> > >
> > > Thus, there are three kinds of "output," movies, symbols, and shapes.
> NSM handles symbols.   What handles movies and shapes?  X3D!  I don't care
> if it's VRML, XML, JSON, Turtle, Python, JavaScript, Java ... X3D is the
> NSM of graphics!  Now, how can NSM and X3D work together?
> > >
> > > John
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/semantics-public_web3d.org/attachments/20200721/12aab2ea/attachment-0001.html>