[x3d-public] CALL FOR: HAnim Level of Detail and other speed optimizations

[GPU Group]

Example LOD Method 2: Multi-level contiguous skin, LOD node wraps 3
Humanoids, which share same Motion node, demonstrating animation state
carrying over to other LOD levels
(3 Boxman HAnimHumanoids, with yellow, cyan, magenta representing LOA3,
LOA2, LOA1)
https://drive.google.com/file/d/1qF-BsZoENe_x1sVRwbRFsFgE5lQr0P5k/view?usp=sharing
https://freewrl.sourceforge.io/tests/26_Humanoid_Animation/BoxmanBVH_LOD_playlib.x3d
-Doug

On Mon, Nov 20, 2023 at 5:36 PM John Carlson <yottzumm at gmail.com> wrote:

> Only issue i see is using same skin coord info for all skins.  If one
> disables the skinCoord, that might work.
>
> John
>
> On Mon, Nov 20, 2023 at 5:39 PM Joe D Williams <joedwil at earthlink.net>
> wrote:
>
>> This is a bit complex for me to consider much of for now, but I do want
>> to see the ability to use an lod style interface to be able to use lower
>> resolution geometries in certain situations. Please also consider the work
>> being done in HAnim WG LOE (levels of expression) to produce different
>> levels of feature point and geometry point density for facial animation.
>>
>>
>>
>> Please notice that for the skeleton, the same skeleton can be used for
>> any rendition. Only the skin and segment geometries really need to be
>> considered. It is likely that for distance LOD effects the skin would be
>> exchanged for simpler segment geometry, and the need for detailed
>> animations will be reduced. So, I think it makes best practice to first
>> consider using the same skeleton and just be concerned with changing
>> visible geometries and reduced complexity for the animations.
>>
>>
>>
>> For instance the full resolution character may use a skin plus various
>> segment geometry accessories, and the lower resolution versions drop to
>> just segment geometries. The simpler animations can use the  same
>> skeleton,just activate fewer joints.
>>
>>
>>
>> Also please consider the need for various LOD effects for various
>> elements of the humanoid. For example,the accessory watch does not need to
>> be active until the humanoid looks at it or shows it to an observer.
>>
>> Likewise, there is a use case where the character or an external observer
>> may want to see detail of certain area, such a a patch of skin, or a
>> display of some instrumentation.
>>
>>
>>
>> More later. Is it possible that it is time for a phone or zoom meeting to
>> discuss some of this?
>>
>>
>>
>> Thanks,
>>
>> Joe
>>
>>
>>
>>
>>
>> -----Original Message-----
>>
>> From: GPU Group <gpugroup at gmail.com>
>> Sent: Nov 19, 2023 12:47 PM
>> To: Humanoid Animation (H-Anim) Working Group <h-anim at web3d.org>
>> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>
>> Subject: Re: [x3d-public] CALL FOR: HAnim Level of Detail and other speed
>> optimizations
>>
>>
>> A slightly different design: use a conventional LOD node
>> - USE the same Motions in all levels, so routing to a Motion affects all
>> the humanoids, and Motion state -held in Motion node- is transferred to all
>> LOAs
>> - route to motionsEnabled in all humanoids (multiple route targets)
>> - using Motion nodes for animation allows the loose coupling / name
>> lookup approach
>> - proposed MotionOrientation node type would also allow loose coupling
>> for OrientationInterpolators so would work on all LOA skeletons
>> LOD {
>> range [ 1, 5, 20]
>> children [
>> DEF LOA3 HAnimHumanoid {
>> motionsEnabled [true false]
>> motions [
>>   DEF MA Motion {...}
>>   DEF MB Motion {...}
>>   ]
>> }
>> DEF LOA2 HAnimHumanoid {
>> motionsEnabled [true false]
>> motions [ USE MA, USE MB ]
>> }
>> DEF LOA1 HAnimHumanoid {
>> motionsEnabled [true false]
>> motions [ USE MA, USE MB ]
>> }
>> ]
>> }
>>
>>
>> On Sun, Nov 19, 2023 at 1:16 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>>> The browser would swap all related fields when swapping skeleton and
>>> skin. But wouldn't swap .motions or .motionsEnabled -- that way the scene
>>> only has to route to animations in the high LOA humanoid, and the state of
>>> animation would persist across levels.
>>> more..
>>> The challenge is to swap humanoids, but do it in such a way that
>>> a) routes from the scene don't need to change endpoints - such as
>>> MotionsEnabled or to Joints
>>> b) animations don't miss a step - animation state is carried over to the
>>> next LOA humanoid
>>> -Doug
>>>
>>> On Sun, Nov 19, 2023 at 12:21 PM John Carlson <yottzumm at gmail.com>
>>> wrote:
>>>
>>>> How would skinCoord and skinNormal work?
>>>>
>>>> On Sat, Nov 18, 2023 at 12:26 PM GPU Group <gpugroup at gmail.com> wrote:
>>>>
>>>>> LOD types > multi-level contiguous > extension to Humanoid > Proposed
>>>>> internal extension method
>>>>> - fields kept the same
>>>>> - internal processing changes to do something different when there's
>>>>> an LOD containing HAnimHumanoids in the skin field
>>>>> a) stores parent humanoid (level 0, LOA3) and LOD level 1,2 (LOA2,
>>>>> LOA1) humanoids in internal array
>>>>> b) doesn't render LOD as part of skin
>>>>> c) scrapes the SFInt32 [out] level_changed field on each frame to find
>>>>> which humanoid to render
>>>>> d) requires Motion or MotionOrientation methods of animation (loose
>>>>> coupling through name lookups)
>>>>> e) Motions and motionsEnabled are put in LOA3 parent humanoid, and
>>>>> applied to LOA2 and LOA1 via loose name-lookup
>>>>> Benefits:
>>>>> - no field changes, no new node types
>>>>> - no new nodes or fields added to x3d.py
>>>>> - exporter can do it with some changes, such as export panel option
>>>>> [x] package humanoids as LOD
>>>>> Disbenefits:
>>>>> - method not documented in specs and not obvious to scene designers
>>>>> - hard for new browser or tool entrants to figure it out from specs or
>>>>> sample scenes
>>>>> - would crash or confuse a browser that's not prepared for it
>>>>>
>>>>> Switch {
>>>>>   whichChoice -1
>>>>>   children [
>>>>>      DEF LOA2 HAnimHumanoid ...
>>>>>      DEF LOA1 HAnimHumanoid...
>>>>>   ]
>>>>> }
>>>>> DEF LOA3 HAnimHumanoid {
>>>>> skeleton HAnimJoint { ... }
>>>>> skin [
>>>>>   DEF LOA3SKIN Group {...}
>>>>>   LOD {
>>>>>     range [ 2, 5, 20]
>>>>>    children [
>>>>>       Group {} #empty placeholder for LOA3, the parent humanoid
>>>>>       USE LOA2
>>>>>       USE LOA1
>>>>>    ]
>>>>>   }
>>>>> ]
>>>>> }
>>>>>
>>>>> -Doug
>>>>>
>>>>> On Mon, Nov 13, 2023 at 6:44 AM GPU Group <gpugroup at gmail.com> wrote:
>>>>>
>>>>>> makehuman has topologies with different vertex counts:
>>>>>>
>>>>>> http://www.makehumancommunity.org/wiki/Documentation:Alternative_topologies
>>>>>>
>>>>>> - that's available in both standalone makehuman (geometries >
>>>>>> topologies tab) and in MPFB2 blender plugin makehuman (Apply Assets >
>>>>>> Topologies library)
>>>>>> - lowest poly male is 741 vertices
>>>>>> - highest is 13290
>>>>>> - there's a low and high poly eye or none
>>>>>> Options
>>>>>> - create and export each level separately from separate blender scene
>>>>>> files, and hand-merge in LOD web3d scene
>>>>>> - create each level in same blender scene file, export as scene with
>>>>>> multiple humanoids at same location and hand patch file to make LOD
>>>>>> - export panel option [x] merge humanoids as LOD
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 13, 2023 at 5:06 AM John Carlson <yottzumm at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Some can probably be post-processed.
>>>>>>>
>>>>>>> On Sun, Nov 12, 2023 at 9:41 PM GPU Group <gpugroup at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> A few things related to LOD level of detail for humanoids and
>>>>>>>> blender exporting
>>>>>>>> 1. how to convert a contiguous makehuman skin to segments for the
>>>>>>>> segment method
>>>>>>>> 2. how generate more than one level of skin and skeleton detail
>>>>>>>> with makehuman in blender for the multiple skin method
>>>>>>>> 3. how to export 1 and 2 / how to set up the results of multiple
>>>>>>>> levels for export so they come out in forms we can use
>>>>>>>> - multiple skin method might use a new makeHumanLOD node which will
>>>>>>>> have a field humanoids MFNode for each LOD as well as motions,
>>>>>>>> motionsEnabled fields and LOD-specific fields like MFFloat range []
>>>>>>>> 4. Apose vs Ipose - makehuman has a funny Apose for skin and
>>>>>>>> skeleton (rig). when exporting can the difference between Ipose and Apose
>>>>>>>> be written to the Joint.rotation fields? Can h-anim joint-named rig be
>>>>>>>> chosen from rig list, and moved to the makehuman A-pose?
>>>>>>>> -Doug
>>>>>>>>
>>>>>>>> On Sun, Nov 12, 2023 at 5:49 PM John Carlson <yottzumm at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>  Note, motionsEnabled may currently be disabled in HAnimHumanoid
>>>>>>>>> Blender export code.  We can add it back with some agreement.
>>>>>>>>>
>>>>>>>>> I believe motions is still exported, so adding motionsEnabled by
>>>>>>>>> hand is not difficult.
>>>>>>>>>
>>>>>>>>> I did get motions working for Pt 2 annex d example with some
>>>>>>>>> changes.
>>>>>>>>>
>>>>>>>>> John
>>>>>>>>> On Sat, Nov 11, 2023 at 9:15 AM GPU Group <gpugroup at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> LOD suggestions (all untested)
>>>>>>>>>>
>>>>>>>>>> LOD Requirements:
>>>>>>>>>>
>>>>>>>>>> a) route to persistant fields for motions because skeleton/joints
>>>>>>>>>> and skin are swapped
>>>>>>>>>>
>>>>>>>>>> b) motions include routing from orientationInterpolators,  and
>>>>>>>>>> routing to.motions and .motionsEnabled fields
>>>>>>>>>>
>>>>>>>>>> Main Options:
>>>>>>>>>>
>>>>>>>>>> A. external HAnimLOD node
>>>>>>>>>>
>>>>>>>>>> - with MFNode field of HanimHumanoid and range [] field from LOD
>>>>>>>>>> node
>>>>>>>>>>
>>>>>>>>>> - would expose .motions and .motionsEnabled fields and relay
>>>>>>>>>> their values to active LOD level Humanoid on each frame
>>>>>>>>>>
>>>>>>>>>> B. extension to HanimHumanoid
>>>>>>>>>>
>>>>>>>>>> - with MFFloat range [] and MFNode skeletons [] and MFNode skins
>>>>>>>>>> []
>>>>>>>>>>
>>>>>>>>>> - motions and motionsEnabled would be persistent
>>>>>>>>>>
>>>>>>>>>> - on each frame motions are applied to the currently selected LOD
>>>>>>>>>> level skeleton joints
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> OrientationInterpolator – options for persistent field targets >
>>>>>>>>>> new node types
>>>>>>>>>>
>>>>>>>>>> MotionRelay
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - MFString jointNames
>>>>>>>>>>
>>>>>>>>>> - MFRotation values
>>>>>>>>>>
>>>>>>>>>> – takes orientationInterpolator values and expresses them as
>>>>>>>>>> motions in Humanoid.motions field
>>>>>>>>>>
>>>>>>>>>> - so routing to multiple LOD levels is replaced by routing to a
>>>>>>>>>> relay
>>>>>>>>>>
>>>>>>>>>> - relay applies to LOD level skeleton joints by name lookup
>>>>>>>>>> during motion pass
>>>>>>>>>>
>>>>>>>>>> x no way to convert SFRotation routes to MFRotation in web3d
>>>>>>>>>>
>>>>>>>>>> MotionInterpolator
>>>>>>>>>>
>>>>>>>>>> - MFString jointNames
>>>>>>>>>>
>>>>>>>>>> - MFNode OrientationInterpolators
>>>>>>>>>>
>>>>>>>>>> - MFBool replace
>>>>>>>>>>
>>>>>>>>>> - no routing to skeleton joints needed, motion update code will
>>>>>>>>>> match jointname and scrape [out] value_changed
>>>>>>>>>>
>>>>>>>>>> Neither MotionRelay nor MotionInterpolator have frameIndex,
>>>>>>>>>> frameIncrement they aren’t fully derived from Motion abstract type, Motion
>>>>>>>>>> hierarchy would need to be refactored
>>>>>>>>>>
>>>>>>>>>> -Doug
>>>>>>>>>>
>>>>>>>>>> On Sat, Nov 11, 2023 at 8:05 AM GPU Group <gpugroup at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Issue: Unlike offline raytracing, realtime graphics looks best
>>>>>>>>>>> with fast frame rates. There is some desire to go beyond cartoon-like
>>>>>>>>>>> characters and achieve more realism in HAnim. Detailed HAnimHumanoid models
>>>>>>>>>>> designed for raytracing can slow frame rates. Scenes with multiple
>>>>>>>>>>> Humanoids and crowds with humanoids at different distances, and clothing
>>>>>>>>>>> details over skin on humanoids may benefit from speed optimizations.
>>>>>>>>>>> 1. Requirements for Level of Detail LOD type optimizations for
>>>>>>>>>>> humanoids:
>>>>>>>>>>> a) coordinating change of skeleton / joints with change of skin
>>>>>>>>>>> and Coordinate node
>>>>>>>>>>> b) persistent fields to route to, in particular motions,
>>>>>>>>>>> motionsEnabled when using Motion nodes, and method to use
>>>>>>>>>>> OrientationInterpolators on persistent fields as the skeleton / joint nodes
>>>>>>>>>>> change
>>>>>>>>>>> 2. other speed enhancements - methods to remove skin under
>>>>>>>>>>> clothing / swap clothing detail, use of buffers and GPU
>>>>>>>>>>>
>>>>>>>>>>> Please post suggestions, methods and state of trial / if it has
>>>>>>>>>>> been tested in a browser.
>>>>>>>>>>>
>>>>>>>>>>> -Doug
>>>>>>>>>>> (PS I have no authority to do a CALL FOR anything. But want to
>>>>>>>>>>> see if web3d and hanim working groups should be doing/ can benefit from
>>>>>>>>>>> calls for)
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> x3d-public mailing list
>>>>>>>>>> x3d-public at web3d.org
>>>>>>>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>> x3d-public mailing list
>>>>> x3d-public at web3d.org
>>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>>
>>>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20231121/713be109/attachment-0001.html>