[x3d-public] HAnim and glTF skins

[Joseph D Williams]

➢ Displacer could be a GPU friendly, simple Coordinate/NormalInterpolator.

It was, then somebody wanted to make it easier and more complact. To me, it worked. 

➢  In addition, morph targets also define normal and tangent 'displacements', so they do not need to be recomputed if available.

Right, hanim displacers are short on that. 

➢ I agree that it should be the job of the engine/browser to make authoring/editing easy while still providing 

Right, vrml/x3d think that is should be learnable and readable. 

➢ However, the tradeoff is that this makes implementations harder or perhaps completely unfeasible in some cases. 

Finding those cases is important. Adding skin seems to kill some implementations; Are we looking at such a tradeoff here? HAnim is tough to get complete especially when we start putting in restrictions like displacer is additive to deformations due to joint rotation.  

➢ I think, in glTF all vertices need a corresponding target. It is probably cheaper on the GPU just to displace all than doing the work necessary to skip some. 

As long as the stuff is done right, with best convenience to the author, then it is OK and up to the application. We do not wish the author to include the whole mesh if it is just a few points to be operated on. That’s why the displacer is great and once you get it, easier to use than an interpolator acting on the entire mesh just to move a couple of points. 

Thanks, 
Joe

From: Joseph D Williams
Sent: Saturday, December 1, 2018 8:44 AM
To: Andreas Plesch; X3D Graphics public mailing list
Subject: Re: [x3d-public] HAnim and glTF skins

➢ I do think glTF allows that mesh vertices are affected by both, morph targets and joints, at least nothing is precluding it. It does not seem explicit about what effect is applied first, perhaps it is implicit that morphing is first.

HAnim is clear: First find Joint(s), then find Displacer(s). What if multiple Displacer moves on same vertex, then maybe order of appearance in user code. Certainly the skin deformation must be applied in certain order - first parents, then child, down the line. Every cycle the vertex gets moved by new joint data and this file in its inial state would define a pose. I think in detailed surface animations, like maybe face, it is still mainly a simple set of basic joints, enhanced by various displacers and you want to be able to easily read the data you are authoring. . 
  
There is no reason to expect that any gltf would associate order about which effect(s) are applied first. 
That is for author and finally the runtime. Hey, it probably was a big deal with deep negotiations to define a gltf package that takes the step of being able to associate a vertex of a mesh with data set of joints and weights. There is a lot you need to know about the target to be animated in order to use this little package in realtime runtime. Read one for a piece of hair with 500 joints and 5000 vertices. A certain binding convenience for the runtime is provided vey expllicitly. 

There are other package(s) somewhere that associates the joint locations and hierarchies an others. In fact, this choice of contents is a most secure way of transporting carefully-guarded animation data, because unless you get more, exactly as expected, you know almost nothing. 

The author has to already understand what to do with the package he gets. For example, to import a certain style of gltf,  x3d importer uses those components of the hanim in place of expected user code. An author importing such a file would  know that he won’t need to supply skincordindex and skincoordweight for each Joint, and maybe skin field could be empty. This could be a real authoring convenience (but now don’t change the skeleton or nothing may work until bindings fixed). So you know what you are importing (like maybe you just got new skin with different res for your old skeleton so you need new bindings) and the sender is sure of what he is sending re: target skeleton and target skin.
If you know enough about the humanoid, you can look at this list either way or both ways, and in action. In fact, the only way you can do deep troubleshooting is look at the skin-skeleton binding(s) both ways!

For text-driven authortime, let’s give a list of joints that out front shows the joint to skin vertex bindings. Due to the hierarchy this is really the first and most important authoring step (of course some authoring views may show the vertex  association with a segment (or bone)). In reality, the joints mainly run the show. So, this is a way to structure a data package for one part of one step (now so simple) of best practice industry transport of one or more parts of the data set required to produce an animated humanoid? I’m very certain that for a certain set of transport operations, this a great way to save and transport a part of the show, maybe just a certain pose or the basis for further animation.  

➢ … what effect is applied first

Of course this is critical that there is an order. And this just gives one step in vertex operations - deforming the mesh any deformation order may be possible, maybe a switch somewhere, but ‘standard’ hanim is first you get the deformation due to joint(s), then you apply the displacer(s). 
As is necessary,  the author must be able to depend upon the order applied since in the final result all this ultimately, depending upon your persistence, needs to be carefully hacked in user code, or whatever is presented to you as user code in whatever tool you are driving to create your humanoid. I say just recall we are looking at stuff that should qualify as industry standard best practices for inclusion in x3d. Since all the data is the same for what hanim does, it is just the names and style of organizing the data, except in the case that hanim only knows about axis-angle and not quats, which we really ought to fix in x3d. If you have skin in the game of skeletal animation, you want to create, edit, document, and transport animation using quaternions. 

➢ It is possible to map many of the glTF data structures to X3D and HAnim data structures but this ability is unrelated to data types or binary encoding. X3D and HAnim provide enough flexibility to fit glTF. The reverse is not always the case.

Right, there is much ‘non-standard’ transport of certain elements of this highly prized professional exchange of skeleton and displacer style animations. This particular gltf example is just one that is actually one form of the most basic data you gotta send to a model you know very well to set a pose. All the bigs agree that this formulation is verifiable and secure and is practically worthless by itself. Really, to import a compete skin and skeleton and bindings and animations you need a grandlist of objects that the runtime knows it can just drop into certain buckets and go. In x3d the author defines those buckets in a certain hierarchy using certain objects with expectations of certain results. There is always some new stuff (for instance at one point there was discussion of defining some kind of event-driven runtime for an example gltf eater) and always new ways of looking at old stuff so let’s try to bring some of that into the hanim realm. 

All Best Wishes,
Joe



From: Andreas Plesch
Sent: Friday, November 30, 2018 6:20 AM
To: X3D Graphics public mailing list
Subject: Re: [x3d-public] HAnim and glTF skins

Joe, excellent points. See below.

> Date: Thu, 29 Nov 2018 11:47:34 -0800
> From: Joseph D Williams <joedwil at earthlink.net>
> To: John Carlson <yottzumm at gmail.com>, GPU Group <gpugroup at gmail.com>
> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>
> Subject: Re: [x3d-public] HAnim and glTF skins
>
> ? Andreas - Another feature in glTF are morph targets which are independent of skins. They are similar to HAnim displacers but I have not looked very closely if it is possible to define a mapping between them.
>
> The thing that maybe special in HAnim is that the displacers applied to skin are additive to other mesh animation. I think you will find that morph thingees will be used with other skin animation techniquess

I do think glTF allows that mesh vertices are affected by both, morph targets and joints, at least nothing is precluding it. It does not seem explicit about what effect is applied first, perhaps it is implicit that morphing is first.

>
> HAnimDisplacer
>  coordIndex    [  numerical index of each parent geometry vertex
>    to be animated in the same order as they appear
>    in the user code for the target geometry ]
>  displacements  [  x,y,z in skeleton space
>    maximum displacement of each vertex  ]
>  weight        0 to 1 animation control
>    (linear interpolation)
>    0 = same location as parent mesh initial vertex
>    1 = defined maximum displacement location of parent mesh vertex
>
> Parent geometry is either the Humanoid, if the Displacer is applied to the mesh in the skin node, or, the geometry of the Displacer node's parent Segment. The displacements xyz data gives a relative target maximum displacement, sort of like there is vector that points from the current vertex position to the location of maximum displacement and the current vertex is moved along that vector according to the weight input.

That is identical to morph targets in glTF. There, for some reason, the weights are defined in the Node (transform) around the morphed mesh.

In addition, morph targets also define normal and tangent 'displacements', so they do not need to be recomputed if available.

>
> Displacer nodes require detailed knowledge of the parent mesh. Specifically, the vertex order of the parent mesh must not change between authoring and rendering. Compared to CoordinateInterpolator, for example, since all vertices of the mesh may not need to be animated for a sequence, only the vertices to be moved are included. If the mesh is being animated by another process, then those results and all displacer(s) results must be applied to the entire mesh in the same frame.

I think, in glTF all vertices need a corresponding target. It is probably cheaper on the GPU just to displace all than doing the work necessary to skip some. 

>
> In fact, what would be the problem, since the name of the node in X3D is HAnimDisplacer, the name Displacer is available for x3d, Why not figure out how to do at least a simple one for V4, or change the name. Canvas the industry and see the various names they call these things, then pick something different. The data and execution of the displacer is like other things called various names that so the same thing, as it has always been.

Displacer could be a GPU friendly, simple Coordinate/NormalInterpolator.

>
> Also, if a good goal is to bring in some more simplified form of articulation-driven mesh animation, why not define a simple root-Joints-segments-skin hierarchy setup using some plain names for the animation nodes and skin bindings. I mean, whatever. we might as well call a joint a rotary, fixed center of rotation actuator. And, of course we need joint-like things, only the most perverse (mis)understanding of the fact that bone orientation the same as joint rotation wants to do away with joints.
>
> I can see why a gltf vertex contains a list of actuators and weights while hanim uses the actuator to hold the list of vertices and weights of the mesh it animates. Some gltf code may show a minimum of 16 actuaor realtionships per vertex is ?standard? with bunches of zeros, beca the hdwr wants it that way. That is wasteful and from what I have seen x3d would not make it the prime directive that the authortime  has to look like the runtime. That has always been. There is runtime and there is authortime. HAnim, is concerned with composability and clear documentation. Sure, it may depend upon how you learned it, but for authortime, if you take care of it at the jointss, then the mesh will be ok.

I agree that it should be the job of the engine/browser to make authoring/editing easy while still providing flexibility and power. However, the tradeoff is that this makes implementations harder or perhaps completely unfeasible in some cases.

>
> The gltf binary is for under the covers, more for rendering than for authoring. The runtimes want the data in gltf form and hanim says that is fine, take the way we do it and create your binary, maybe using a style sheet, or vice-versa. Every current x3d runtime converts the x3d utf-8 text to a target binary form at some point. That is why a set of properly structured binaries might be able to be imported rather easily because the runtimes probably use that form anyway, just input in different, more abstractly-connected containers than x3d hanim. The names and readability are different while the data, the data structures, and the necessary connections are the same under the covers. I hope all the gltf and x3d data types match. gltf is fine, as long as I can edit the source.

It is possible to map many of the glTF data structures to X3D and HAnim data structures but this ability is unrelated to data types or binary encoding. X3D and HAnim provide enough flexibility to fit glTF. The reverse is not always the case.

-Andreas


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20181201/431ba6c2/attachment-0001.html>