[x3d-public] Story-to-Motion

Thu Nov 23 05:57:07 PST 2023

The interpolators python program also contains ROUTES.

John

On Thu, Nov 23, 2023 at 7:33 AM GPU Group <gpugroup at gmail.com> wrote:

> The exporter options panel can / could / should have an option (bullets or
> pulldown list) to export as a) OrientationInterpolators b) Motions, and
> perhaps in the future c) MotionOrientation -- a proposed new node type that
> would have an MFNode field for orientationInterpolators, MFString field for
> joint names, and be a bridge between the two methods.
>
> On Wed, Nov 22, 2023 at 9:27 PM John Carlson <yottzumm at gmail.com> wrote:
>
>> Doug, if you need to re-enable HAnimMotion (motions) in the exporter, go
>> ahead.  I couldn't get it working well, but we could take
>> export_interpolators.py and redo export_bvh.py.
>>
>> John
>>
>> On Wed, Nov 22, 2023 at 9:36 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>>> Looks interesting -- go for it.
>>> And maybe some of the papers referenced might have some interesting
>>> algorithms.
>>> Relating to HAnim, could there be a HyperMotion node, and .motions
>>> [Motion] would be the 'database of short animations' the article mentions,
>>> and with any attributes needed added to the Motion node, for example 'name'
>>> for a name lookup / matching of motion, a 'cycle duration' handy, distance
>>> per cycle etc.
>>> HyperMotion might compute or be programmed with a queue of (name,
>>> location_target(x,y), duration(time)) tuples, and when it gets to a target,
>>> it looks up the next motion 'name' from the .motions list, and does a
>>> transition blending for a fraction of a second, between old and new motion.
>>> But the story-to-motion or whatever else is generating the queue of
>>> tuples - that's the part that looks either laborious pre-programmed, or
>>> requires AI automation of some sort to generate the queue.
>>> And what about collision avoidance with other animated characters and
>>> static scene elements? What about switching to stair climbing animation
>>> when over stairs and other scene-responsiveness?
>>> Reminds me a bit of the Crowd Particles function map.
>>>
>>> https://freewrl.sourceforge.io/tests/40_Particle_systems/crowd/function_map.png
>>>
>>>
>>> https://drive.google.com/file/d/1krC0k2CKZDuOGgzlyaFosrFbjFRO2qzx/view?usp=drive_link
>>>
>>> which could be done other ways such as function bounding boxes /
>>> rectangles or as the paper suggested trajectory lines.
>>> So HyperMotion might have children field with some sort of (zone,
>>> function) tuple list it can use to switch between motions along a
>>> trajectory.
>>> -Doug
>>>
>>>
>>> On Wed, Nov 22, 2023 at 7:36 PM John Carlson <yottzumm at gmail.com> wrote:
>>>
>>>> Hot air:
>>>>
>>>> What may be more interesting to game developers is tracking many, many
>>>> players in a game, and then producing NPCs based on traces capable of
>>>> playing like humans.  It’s similar to “predict the next word” for popular
>>>> AIs.   Then add reinforcement learning to improve successful NPCs.  This is
>>>> where synthetic motion generation (ala Story-to-Motion) may come in handy
>>>> instead of relying solely on humans for training data.  The question is,
>>>> can story-to-motion provide enough creativity?
>>>>
>>>> On Wed, Nov 22, 2023 at 7:57 PM John Carlson <yottzumm at gmail.com>
>>>> wrote:
>>>>
>>>>> Please let us know of supporting authoring systems and players.
>>>>>
>>>>> From what I can see of the video and paper, the Story-To-Motion is
>>>>> more about position of the humanoid and rotation of joints.  If these are
>>>>> “Group”ed by keyword, and smooth transitions are accomplished by chaining
>>>>> animations, then some degree of automation can result.  If result numbers
>>>>> are available in a potential tool like JAminate,  Blender or a text editor,
>>>>> and accessible to human animators for optimization, all the better!
>>>>>
>>>>> I’m thinking more and more about parsing interpolators into JAminate,
>>>>> which has been a goal.  Using JAminate or a spreadsheet, one could
>>>>> potentially group key-fractions and keyValues, then output interpolators
>>>>> and routes could be published in addition to chaining of moves with Boolean
>>>>> Sequencers.
>>>>>
>>>>> John
>>>>>
>>>>> On Wed, Nov 22, 2023 at 7:18 PM Joe D Williams <joedwil at earthlink.net>
>>>>> wrote:
>>>>>
>>>>>> It might be that the closest x3d going on with this is in web3d x3d
>>>>>> HAnim WG facial animation standards-track. The facial mesh of various
>>>>>> densities are organized into regions activated in certain sets to perform a
>>>>>> certain expression.  So, standard keywords and parameter sets can activate
>>>>>> standard related areas of the mesh to allow use of standardized animations
>>>>>> as well as serve as a prime basis for the personalization of the Humanoid
>>>>>> that you really want to do. This is not a quick and simple effort. It is
>>>>>> very interesting, design-friendly, much simplified authoring, and much more
>>>>>> standardizable and demonstratable at all levels of expression using the
>>>>>> hanim displacer concept.
>>>>>>
>>>>>>
>>>>>>
>>>>>> Joe
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: John Carlson <yottzumm at gmail.com>
>>>>>> Sent: Nov 21, 2023 3:19 PM
>>>>>> To: Joe D Williams <joedwil at earthlink.net>, X3D Graphics public
>>>>>> mailing list <x3d-public at web3d.org>
>>>>>> Subject: Re: Story-to-Motion
>>>>>>
>>>>>>
>>>>>> Here’s a better link i hope.
>>>>>>
>>>>>>
>>>>>> https://arxiv.org/abs/2311.07446#:~:text=A%20new%20and%20challenging%20task,level%20control%20(motion%20semantics)
>>>>>> .
>>>>>>
>>>>>> This should be a better link (added f):
>>>>>>
>>>>>> https://arxiv.org/pdf/2311.07446.pdf
>>>>>>
>>>>>> Apologies for multiple posts.
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On Tue, Nov 21, 2023 at 5:08 PM John Carlson <yottzumm at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> https://arxiv.org/pdf/2311.07446.pd
>>>>>>>
>>>>>>> Has this research been peer-reviewed?
>>>>>>>
>>>>>>> It’s hard to read on my cell phone, I may get my computer and
>>>>>>> attempt to read it.
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>> x3d-public mailing list
>>>> x3d-public at web3d.org
>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20231123/09e19442/attachment-0001.html>