[x3d-public] FW: FW: [x3d] Spec Comment by dougsanden on 19774-2: HAnimMotionCapture -V1.0

GPU Group gpugroup at gmail.com
Sat Jun 27 15:35:19 PDT 2020 

I don't remember doing any inverseBindingMatrices in HAnim.
Hypothesis: our (unsegmented) skin mesh is in HH space / root_joint space.
If so then the initial binding matrix for us would be the initial pose
inverse transform stack from HH space down jointX. .
For segmented, our segment meshes are typically already in joint space by
parenting to the joint transform, so inverseBindingMatrix for them would be
Identity.


On Sat, Jun 27, 2020 at 4:05 PM GPU Group <gpugroup at gmail.com> wrote:

> You mean like this:
>
> mesh in mesh-space => inverseBindingMatrixX => mesh in jointX space =>
> joint transform stack at joint X => mesh in global space.
>
> On Sat, Jun 27, 2020 at 3:14 PM Joseph D Williams <joedwil at earthlink.net>
> wrote:
>
>>
>>    - . Q, what are the gltf inverseBindingMatrices
>>
>> Notice the Joint node which lists the inverse binding matrices by listing
>> the vertex(s) controlled by that Joint and the weights. This tells the
>> system how to act when a joint is rotated. Move the vertex (linear along
>> radius) with Joint rotation according to the weight. For gltf, the binding
>> is shown for each vertex and each Joint that produces an effect on the
>> vertex. I think in general, this data represents a vertex and the joint
>> weights for each frame. So the data for that frame is telling the hardware
>> how to compute the vertex displacement for the frame.
>>
>> This is basically how, if someone wants to mostly hide the actual
>> animation routine, like a timer/interpolator/route system that wants to
>> remain fairly hidden, and to mostly hide the binding data, to stash the
>> binding and precomputed animation data for each frame in a multitude of
>> small files the represent what to compute for each vertex at each frame.
>>
>>
>>
>> Again, to me this is fine for collecting data to make the movie, but not
>> for realtime. For realtime, this data is basically computed the same way,
>> but in general, the gltf data is extracted from the runtime and saved. The
>> realtime system has to actually compute the data according to which joints
>> have moved for each frame.
>>
>>
>>
>> This is why some of those gltf file sets are so big, right. It is because
>> it does not represent actual user code (except for maybe the person who
>> lives in there) but precomputed stuff. With a file for each vertex for each
>> frame. From the gltf it is not that easy to reconstruct the actual
>> animation routine or even the bindings. All tyou get is basically
>> precomputed stuffs from whatever runtime generates the files. At least this
>> data is exposed for us to ponder over. In the old days, with tools
>> generally oriented towards making video or film, then when you asked for
>> the export, all you could get was a giant array, one set of actual rendered
>> data for each finished frame. If that is what you have, then Please notice
>> x3d includes an interpolator that allows you to increment through a stack
>> of completed frames.
>>
>>
>>
>> Thanks,
>>
>> Joe
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From: *GPU Group <gpugroup at gmail.com>
>> *Sent: *Saturday, June 27, 2020 5:39 AM
>> *To: *Joseph D Williams <joedwil at earthlink.net>
>> *Cc: *X3D Graphics public mailing list <x3d-public at web3d.org>
>> *Subject: *Re: FW: FW: [x3d] Spec Comment by dougsanden on 19774-2:
>> HAnimMotionCapture -V1.0
>>
>>
>>
>> Thanks for insight. Q, what are the gltf inverseBindingMatrices mentioned
>> in p.5, 6 https://www.khronos.org/files/gltf20-reference-guide.pdf in
>> HAnimn terminology?
>>
>>
>>
>> On Sat, Jun 27, 2020 at 12:33 AM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>
>>
>>
>> *From: *Joseph D Williams <joedwil at earthlink.net>
>> *Sent: *Friday, June 26, 2020 9:53 PM
>> *To: *GPU Group <gpugroup at gmail.com>
>> *Cc: *X3D Graphics public mailing list <x3d-public at web3d.org>
>> *Subject: *RE: FW: [x3d] Spec Comment by dougsanden on 19774-2: HAnim
>> MotionCapture -V1.0
>>
>>
>>
>>    - Getting past blockages in the HAnim project and on to node
>>    improvements might help later when harmonizing with glTF skinning.
>>
>>
>>
>> Any hanim blockages are just getting some really rather straightforward
>> simple updates (names and hierarchies) done.
>>
>>
>>
>> We are already harmonized with gltf, because it is the same data in json
>> form as we author in x3d human-readable form. Nothing magic or not
>> harmonious, just the gltf data is built for hardware instead of humans. If
>> you have figured out what the gltf stuff is and how to use it, then you
>> will have learned about x3d hanim.
>>
>> Likewise, the reverse is true but it is more difficult because the gltf
>>  data is so scattered around. For example, the skin deformation is indexed
>> by vertex usually with about 16 (for convenience) joints and weights, and
>> one of these for each vertex.  If you look at hanim Joint, the vertices and
>> weights associated with each joint are listed in the Joint node. So, what
>> would you do with hanim Joint data to create some gltf vertex objects?
>>
>> Joe
>>
>>
>>
>> *From: *GPU Group <gpugroup at gmail.com>
>> *Sent: *Friday, June 26, 2020 3:46 PM
>> *To: *Joseph D Williams <joedwil at earthlink.net>
>> *Cc: *X3D Graphics public mailing list <x3d-public at web3d.org>
>> *Subject: *Re: FW: [x3d] Spec Comment by dougsanden on 19774-2: HAnim
>> MotionCapture -V1.0
>>
>>
>>
>> Q. HAnim Examples > What does the to-do list look like?
>>
>> I ask because I got a sense that no new / modified HANim nodes would be
>> considered beyond official HAnim2 until examples cleaned up.
>>
>> Is there anything blocking that? I ask in case someone has something that
>> can help.
>>
>> One thing that popped to mind: is there a problem getting authors
>> permission? If so and an example was demonstrating a particular thing,
>> perhaps that scene can be dropped and a new one authored or volunteered by
>> others reading this who may be sitting on some assets or capabilities that
>> can volunteer equivalent scenery.
>>
>> -Doug
>>
>> more..
>>
>> On the horizon: glTF skinning which may have the effect of bypassing
>> HAnim, perhaps obsoleting HAnim in practice.
>>
>> https://www.khronos.org/files/gltf20-reference-guide.pdf
>>
>> - p.5,6,7
>>
>> Getting past blockages in the HAnim project and on to node improvements
>> might help later when harmonizing with glTF skinning.
>>
>>
>>
>> On Fri, Jun 26, 2020 at 3:52 PM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>
>>    - My conceptual model is a bit different.
>>
>>
>>
>> That is OK by me as long as it works. Whatever, Joint hierarchies will
>> operate just like practical hierarchies of standard x3d Transform nodes,
>> using center of rotation concept.
>>
>>
>>
>> For the Humanoid, if there are more joints in the capture than the
>> playback, then you “ignore” some sets of values. If there are more joints
>> in the playback than the capture, then, if the hierarchy is still ok, you
>> don’t send events to the extra joints, or just send the default.
>>
>> The x3d tool should be able to help an author by first giving a list of
>> what joints, segments, sites are available in the playback skeleton defined
>> In the Humanoid skeleton. All the author needs to do to use existing x3d
>> timer/interpolator/route animations is connect up the animation events to
>> the skeleton Joint DEFs. Multiple, selectable animation routines can reside
>> inside or outside the Humanoid, and prototype behaviors are very usable and
>> very sharable between similar characters. In general, these animations are
>> designed for realtime rather than frametime, but of course can be used to
>> produce a frame at any time. However, these animations by themselves may
>> not directly define the skeleton they apply to, or the initial pose.
>>
>>
>>
>> For x3d HAnim Humanoid Motion style, I think we are trying to access a
>> perhaps more public range of animations, by the easiest way possible. From
>> what I have seen these mostly use capture intervals related to video or
>> film and are typically oversampled for typical realtime needs, and also,
>> since mainly using xyz data, are subject to unpredictable animations,
>> particularly fast wide motions. However, we want to make it convenient for
>> an author, so we want to use the data in the bvh file to help connect the
>> skeletons with the data.
>>
>> Turns out this may be easy because the bvh file contains configuration
>> data for the capture skeleton as well as the animation data. Now all we
>> must do is connect the skeletons and then the data and events are all
>> handled under the covers. In this case, it is possible that a tool could
>> for example, display a model of the capture skeleton and the playback
>> skeleton to help the author decide how to proceed, or even, if the names
>> matched, do it automagically.
>>
>> A Humanoid Motion animation resides in the Humanoid object and for best
>> automation, should include enough information to reconstruct the capture
>> skeleton hierarchy. In order to establish this baseline, the existing
>> Motion node wants the author to transcribe important bvh data into x3d data
>> forms and place it within the Humanoid Motion node. This practice also
>> tends to help transportability between various typical capture skeletons
>> and x3d hanim ‘standard’ skeletons.
>>
>>
>>
>>    - Your loa4 idea makes sense if you don't want to ignore….
>>
>>
>>
>> I still think the idea behind ‘ignore’ is that if you have more joints
>> and data in for the capture skeleton than the playback skeleton can use.
>> The author has determined that some of the capture data has to be ignored
>> and not sent to playback skeleton, so those joints are labeled with
>> ‘ignore’ keyword in order to declare that the related data is not to be
>> used. Please tell me if this is not what is intended.
>>
>> If that is the case then the playback skeleton will never know because no
>> events are sent to joints that are not there.
>>
>> If you have more joints in the playback skeleton, then it is ok and no
>> capture data needs to be ignored and playback joints that don’t get data
>> don’t care and work as if not there.
>>
>>
>>
>> Thanks for thinking about this,
>>
>> Joe
>>
>>
>>
>> *From: *GPU Group <gpugroup at gmail.com>
>> *Sent: *Wednesday, June 24, 2020 4:33 PM
>> *To: *Joseph D Williams <joedwil at earthlink.net>
>> *Cc: *Spec Feedback <spec-comment at web3d.org>; x3d at web3d.org
>> *Subject: *Re: [x3d] Spec Comment by dougsanden on 19774-2: HAnim Motion
>> Capture -V1.0
>>
>>
>>
>> My conceptual model is a bit different.
>>
>> - skeletal joint asks for joint information from MotionData/MotionClip,
>> joint by joint when traversing the skeletal joint hierarchy.
>>
>> Your loa4 idea makes sense if you don't want to ignore.And -depending on
>> your implementation, like you say you'd get identity transforms -no joint
>> motion- for joints MotionClip doesn't have.
>>
>> -Doug
>>
>>
>>
>> On Wed, Jun 24, 2020 at 5:20 PM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>    - and they have a brief statement somewhere saying its ignored ie the
>>    values field has it, but the humanoid doesn;t.
>>    - But they didn't say mathematically what to do about it.
>>
>>
>>
>> OK, so there are Motion values for a joint in from the capture skeleton
>> but either they do not apply to a Joint in the playback skeleton, or there
>> is no corresponding joint in the playback skeleton. Thus, as an author, you
>> just want to ignore that set of values in the Motion values field.
>>
>>
>>
>> For a typical animation routine, if you send events to a node that is not
>> there, you get an error.
>>
>> If there is a Joint in the capture skeleton that does not appear in the
>> playback skeleton, then you don’t try to send events to that joint, because
>> it is not there. So, if the Joint is actually present, then that Joint will
>> remain in its default orientation.
>>
>> This is OK, and the skeleton does not mind at all, it is like the Joint
>> is not there except it is and may have child Joint hierarchy. Of course in
>> that case the parent of the ignored joint controls the child hierarchy. If
>> whatever animation device is under the covers thinks it must send data to
>> that joint, then don’t use the Motion values, just send  0 0 1 0 each cycle.
>>
>>
>>
>> Ignore, means Hey, my capture skeleton has a Joint and I have data for it
>> but my playback skeleton does not have that joint. I am trying to import,
>> for example, some LOA4 animations into my LOA3 playback skeleton. As an
>> author you are choosing to refuse to give your loa3 an update and you must
>> confess that fact by using IGNORE in your Motion node list.
>>
>>
>>
>> At least that has been my experience, when using animations aimed at a
>> higher or lower loa than the playback skeleton. For lower loa playback
>> skeleton, just comment out the interpolators and routes for the unused
>> Joint nodes. If higher loa playback skeleton, then problems because the
>> hierarchy does not care about ignoring intermediate joints. Just don’t send
>> data or if you must, just send 0 0 1 0 to the ignored joint.
>>
>>
>>
>> This is again why you want to just go ahead and start work with the loa4
>> playback skeleton. Because it is ok to ignore it if you don’t have data for
>> it. It is always not ok to not send data to something that is not there and
>> any browser should tell you and probably should fail if you try to do that.
>>
>> Thanks,.
>>
>> Joe
>>
>>
>>
>> *From: *GPU Group <gpugroup at gmail.com>
>> *Sent: *Wednesday, June 24, 2020 3:33 PM
>> *To: *Joseph D Williams <joedwil at earthlink.net>
>> *Cc: *Spec Feedback <spec-comment at web3d.org>; x3d at web3d.org
>> *Subject: *Re: [x3d] Spec Comment by dougsanden on 19774-2: HAnim Motion
>> Capture -V1.0
>>
>>
>>
>> The IGNORED is in the Motion AnnexD example scene:
>>
>>  <HAnimMotion frameCount="392" enabled='true' loop='true' frameTime =
>> "0.033333" frameDuration=".033333"
>>
>>              joints="humanoid_root, l_hip, l_knee, l_talocrural, r_hip,
>>
>>                    r_knee, r_talocrural, vl5, IGNORED, l_shoulder,
>>
>>                    l_elbow, l_radiocarpal, IGNORED, r_shoulder, r_elbow,
>>
>>                    r_radiocarpal, IGNORED, skullbase"
>>
>> and they have a brief statement somewhere saying its ignored ie the
>> values field has it, but the humanoid doesn;t.
>>
>> But they didn't say mathematically what to do about it. My interpretation
>> is that you would accumulate the ignored values as you go down the bvh limb
>> tree to get the next joint. If you're working call-by-call, to make sure
>> the transform fetcher understands what's been skipped, you woult tell it
>> the previous joint you asked for when asking for the next (and it would
>> look at the bvh and see you are ignoring some joints)
>>
>> getNextJointTransform(lastJoint,currentJoint,time,&transform)
>>
>> something like that. I'm not doing that yet, and haven't proven the
>> theory of joint transform accumulation.
>>
>> The likely reason the AnnexD model didn't seem to mind / looked like its
>> bvh animation in blender: the IGNORED values were zeros anyway. I was lucky,
>>
>> The nature of matching up free downloaded bvh with any loa hanim
>> character I have means we;ll be skipping and some and some joints never
>> captured by the mocap system / not in the bvh list. For freewrl users,
>> that's OK - doesn't need to be perfect, just needs to get us into the game
>> and having fun experimenting. .
>>
>> -Doug
>>
>>
>>
>> On Wed, Jun 24, 2020 at 4:18 PM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>    - - IGNORE - the transform fetcher needs to accumulate the ignored
>>    transforms to apply to the next non-ignored joint in the
>>
>>
>>
>> Not sure what the transform fetcher does but this is no different than
>> having an intermediate joint that is not animated.? In operation, not
>> animating a Joint does not cause any problems. That is why it is always
>> good to pick the loa4 skeleton for most fun. If a joint is ignored, it is
>> just not animated and remains in its default position. Its children may
>> still be animated.
>>
>> Anyway, am I missing something? Ignored just means there is no animation
>> driving that Joint. The bigger problem is when you have animation trying to
>> drive a joint that is not there. Mostly the browser will tell you that.
>>
>>
>>
>>    - No need for the software to complain if bvh doesn't match.
>>
>>
>>
>> I think if Motion node cannot find a matching Joint somewhere, then
>> failure until you get the list right and all named joints matched up[.
>>
>> If you look at some sample typical timer/interpolator/route setup and you
>> try to drive a joint that is not there or has the different name, then it
>> will usually tell you about missing sources or targets. Maybe I don’t
>> understand what ignore refers to? What does ignore refer to? A Joint in the
>> Humanoid skeleton, a joint in the capture skeleton,  or a set of data for a
>> joint in the bvh file?
>>
>> Thanks,
>>
>> Joe
>>
>>
>>
>>
>>
>>
>>
>> *From: *GPU Group <gpugroup at gmail.com>
>> *Sent: *Wednesday, June 24, 2020 5:56 AM
>> *To: *Joseph D Williams <joedwil at earthlink.net>
>> *Cc: *Spec Feedback <spec-comment at web3d.org>; x3d at web3d.org
>> *Subject: *Re: [x3d] Spec Comment by dougsanden on 19774-2: HAnim Motion
>> Capture -V1.0
>>
>>
>>
>> No need for the software to complain if bvh doesn't match.
>>
>> Already in the HAnim2 specification for Motion they use the IGNORE
>> keyword when when HH is skipping a joint the bvh has. Not shown in the
>> spec, but likely to happen is the bvh not having a joint that the HH has,
>> what I call NOT_IMPLEMENTED, to keep it separate from IGNORE
>>
>> - IGNORE - the transform fetcher needs to accumulate the ignored
>> transforms to apply to the next non-ignored joint in the limb tree I didn't
>> do this yet, but should have it somewhere, so it applies also in in the
>> original Motion node if there are IGNOREs.
>>
>> - NOT_IMPLEMENTED (bvh doesn't have the joint) - the HH would get an
>> identity transform back for that joint.
>>
>> -Doug
>>
>>
>>
>> On Tue, Jun 23, 2020 at 9:36 PM Joseph D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>    - Then one mapping node …
>>
>>
>>
>> Currently the Humanoid skeleton declares its active Joint nodes by naming
>> each Joint node and by including list of used Joints in the Humanoid joints
>> field.
>>
>> I think the Motion node also declares the joints it wants to use by
>> listing in the Motion node.
>>
>> To my knowledge there is no requirement that the Humanoid joints list be
>> in any specific order.
>>
>> I think the Humanoid Motion joints field needs the list in some order
>> relating to the data.
>>
>> The point is that for Motion to work, it must find the appropriate Joint
>> name in the Humanoid joints field.
>>
>>
>>
>> My only big point is that both of those lists should look about the same
>> (MF strings). If you want a skeleton that is matched to the bvh data, then
>> we change the names in the imported bvh to match the names used in the
>> skeleton field and enumerated in the Humanoid joints field. Some help could
>> be given to an author helping to match the bvh skeleton nomenclature with
>> the Humanoid joints. Or if you are really serious about using bvh data,
>> then gather enough examples to suggest a ‘typical standard’ bvh skeleton
>> and data, then compose a skeleton in the Humanoid skeleton field to match
>> and then you might use a ‘typical standard’ Motion fields.
>>
>>
>>
>> The main idea of the Humanoid joints field is to list the actual joints
>> that are available in the skeleton model. The purpose of the joints list in
>> the Motion node is to declare which of those joints will be active in the
>> current simulation. In this case, if the capture skeleton is the same as
>> the playback skeleton, no problem to just change the data.
>>
>>
>>
>> So, the only things the browser needs to figure out is: Do the Motion
>> joints match the skeleton joints and complain if not.
>>
>>
>>
>> Best Regards and Thanks for thoughts on this.
>>
>> Joe
>>
>>
>>
>> *From: *Spec Feedback <spec-comment at web3d.org>
>> *Sent: *Tuesday, June 23, 2020 6:02 AM
>> *To: *x3d at web3d.org
>> *Subject: *[x3d] Spec Comment by dougsanden on 19774-2: HAnim Motion
>> Capture -V1.0
>>
>>
>>
>> -- Submitter indicates that this comment may be public: *Yes* --
>>
>>
>>
>> Comment on 19774-2: HAnim Motion Capture - V1.0
>>
>>
>>
>>
>>
>>
>>
>> -----------------
>>
>> Proposed Explicit Name Mapping node
>>
>> NameMapping
>>
>> MFString A []
>>
>> MFString B []
>>
>>
>>
>> Esample:
>>
>>
>>
>>
>>
>> Discussion:
>>
>> not necessary to have IGNORED or NOT_IMPLEMENTED - that would be the
>> default
>>
>> if not found during lookup.
>>
>> Then one mapping node can be used for multiple .bvh from various sources.
>>
>> Putting a mapping node into both HAnimHumanoid HH and HAnimMotionClip/Data
>>
>> HMC would allow a 2-step lookup:
>>
>> - HH to loa, loa to HMC
>>
>> - A previous comment shows the math advantage of 2 step
>>
>> If no mapping nodes present, browser would assume names are same in HH and
>>
>> HMC
>>
>> - if one mapping node in either HH or HMC, then a 1-step lookup is done
>>
>>
>>
>>
>>
>> -----------------
>>
>>
>>
>> Submitted on Tuesday, 2020,  June 23 - 7:02am
>>
>> by dougsanden (dougsanden )
>>
>> IP: 75.159.18.239
>>
>>
>>
>> See: https://www.web3d.org/node/1694/submission/4051
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> x3d mailing list
>>
>> x3d at web3d.org
>>
>> http://web3d.org/mailman/listinfo/x3d_web3d.org
>>
>>
>>
>> _______________________________________________
>> x3d mailing list
>> x3d at web3d.org
>> http://web3d.org/mailman/listinfo/x3d_web3d.org
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200627/8b57b31b/attachment-0001.html>

More information about the x3d-public mailing list