[x3d-public] Story-to-Motion

Thu Nov 23 06:02:42 PST 2023

MotionOrientation node would not need the ROUTEs going from
OrientationInterpolator.value_changed field to the Joints -- that would be
done by name lookup, and allows LOD skeleton+skin changes. The ROUTEs going
to the OrientationInterpolators would still be needed / no change.

On Thu, Nov 23, 2023 at 6:57 AM John Carlson <yottzumm at gmail.com> wrote:

> The interpolators python program also contains ROUTES.
>
> John
>
> On Thu, Nov 23, 2023 at 7:33 AM GPU Group <gpugroup at gmail.com> wrote:
>
>> The exporter options panel can / could / should have an option (bullets
>> or pulldown list) to export as a) OrientationInterpolators b) Motions, and
>> perhaps in the future c) MotionOrientation -- a proposed new node type that
>> would have an MFNode field for orientationInterpolators, MFString field for
>> joint names, and be a bridge between the two methods.
>>
>> On Wed, Nov 22, 2023 at 9:27 PM John Carlson <yottzumm at gmail.com> wrote:
>>
>>> Doug, if you need to re-enable HAnimMotion (motions) in the exporter, go
>>> ahead.  I couldn't get it working well, but we could take
>>> export_interpolators.py and redo export_bvh.py.
>>>
>>> John
>>>
>>> On Wed, Nov 22, 2023 at 9:36 PM GPU Group <gpugroup at gmail.com> wrote:
>>>
>>>> Looks interesting -- go for it.
>>>> And maybe some of the papers referenced might have some interesting
>>>> algorithms.
>>>> Relating to HAnim, could there be a HyperMotion node, and .motions
>>>> [Motion] would be the 'database of short animations' the article mentions,
>>>> and with any attributes needed added to the Motion node, for example 'name'
>>>> for a name lookup / matching of motion, a 'cycle duration' handy, distance
>>>> per cycle etc.
>>>> HyperMotion might compute or be programmed with a queue of (name,
>>>> location_target(x,y), duration(time)) tuples, and when it gets to a target,
>>>> it looks up the next motion 'name' from the .motions list, and does a
>>>> transition blending for a fraction of a second, between old and new motion.
>>>> But the story-to-motion or whatever else is generating the queue of
>>>> tuples - that's the part that looks either laborious pre-programmed, or
>>>> requires AI automation of some sort to generate the queue.
>>>> And what about collision avoidance with other animated characters and
>>>> static scene elements? What about switching to stair climbing animation
>>>> when over stairs and other scene-responsiveness?
>>>> Reminds me a bit of the Crowd Particles function map.
>>>>
>>>> https://freewrl.sourceforge.io/tests/40_Particle_systems/crowd/function_map.png
>>>>
>>>>
>>>> https://drive.google.com/file/d/1krC0k2CKZDuOGgzlyaFosrFbjFRO2qzx/view?usp=drive_link
>>>>
>>>> which could be done other ways such as function bounding boxes /
>>>> rectangles or as the paper suggested trajectory lines.
>>>> So HyperMotion might have children field with some sort of (zone,
>>>> function) tuple list it can use to switch between motions along a
>>>> trajectory.
>>>> -Doug
>>>>
>>>>
>>>> On Wed, Nov 22, 2023 at 7:36 PM John Carlson <yottzumm at gmail.com>
>>>> wrote:
>>>>
>>>>> Hot air:
>>>>>
>>>>> What may be more interesting to game developers is tracking many, many
>>>>> players in a game, and then producing NPCs based on traces capable of
>>>>> playing like humans.  It’s similar to “predict the next word” for popular
>>>>> AIs.   Then add reinforcement learning to improve successful NPCs.  This is
>>>>> where synthetic motion generation (ala Story-to-Motion) may come in handy
>>>>> instead of relying solely on humans for training data.  The question is,
>>>>> can story-to-motion provide enough creativity?
>>>>>
>>>>> On Wed, Nov 22, 2023 at 7:57 PM John Carlson <yottzumm at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Please let us know of supporting authoring systems and players.
>>>>>>
>>>>>> From what I can see of the video and paper, the Story-To-Motion is
>>>>>> more about position of the humanoid and rotation of joints.  If these are
>>>>>> “Group”ed by keyword, and smooth transitions are accomplished by chaining
>>>>>> animations, then some degree of automation can result.  If result numbers
>>>>>> are available in a potential tool like JAminate,  Blender or a text editor,
>>>>>> and accessible to human animators for optimization, all the better!
>>>>>>
>>>>>> I’m thinking more and more about parsing interpolators into JAminate,
>>>>>> which has been a goal.  Using JAminate or a spreadsheet, one could
>>>>>> potentially group key-fractions and keyValues, then output interpolators
>>>>>> and routes could be published in addition to chaining of moves with Boolean
>>>>>> Sequencers.
>>>>>>
>>>>>> John
>>>>>>
>>>>>> On Wed, Nov 22, 2023 at 7:18 PM Joe D Williams <joedwil at earthlink.net>
>>>>>> wrote:
>>>>>>
>>>>>>> It might be that the closest x3d going on with this is in web3d x3d
>>>>>>> HAnim WG facial animation standards-track. The facial mesh of various
>>>>>>> densities are organized into regions activated in certain sets to perform a
>>>>>>> certain expression.  So, standard keywords and parameter sets can activate
>>>>>>> standard related areas of the mesh to allow use of standardized animations
>>>>>>> as well as serve as a prime basis for the personalization of the Humanoid
>>>>>>> that you really want to do. This is not a quick and simple effort. It is
>>>>>>> very interesting, design-friendly, much simplified authoring, and much more
>>>>>>> standardizable and demonstratable at all levels of expression using the
>>>>>>> hanim displacer concept.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Joe
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> -----Original Message-----
>>>>>>> From: John Carlson <yottzumm at gmail.com>
>>>>>>> Sent: Nov 21, 2023 3:19 PM
>>>>>>> To: Joe D Williams <joedwil at earthlink.net>, X3D Graphics public
>>>>>>> mailing list <x3d-public at web3d.org>
>>>>>>> Subject: Re: Story-to-Motion
>>>>>>>
>>>>>>>
>>>>>>> Here’s a better link i hope.
>>>>>>>
>>>>>>>
>>>>>>> https://arxiv.org/abs/2311.07446#:~:text=A%20new%20and%20challenging%20task,level%20control%20(motion%20semantics)
>>>>>>> .
>>>>>>>
>>>>>>> This should be a better link (added f):
>>>>>>>
>>>>>>> https://arxiv.org/pdf/2311.07446.pdf
>>>>>>>
>>>>>>> Apologies for multiple posts.
>>>>>>>
>>>>>>> John
>>>>>>>
>>>>>>> On Tue, Nov 21, 2023 at 5:08 PM John Carlson <yottzumm at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> https://arxiv.org/pdf/2311.07446.pd
>>>>>>>>
>>>>>>>> Has this research been peer-reviewed?
>>>>>>>>
>>>>>>>> It’s hard to read on my cell phone, I may get my computer and
>>>>>>>> attempt to read it.
>>>>>>>>
>>>>>>>> John
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>> x3d-public mailing list
>>>>> x3d-public at web3d.org
>>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20231123/d3435cea/attachment.html>