[x3d-public] HAnim and glTF skins

Thu Nov 29 17:51:53 PST 2018

➢ Joseph, are you basically saying skeletal animation can be done without vertex shaders and VRML scripes?

I don’t think I said that. This type of animation is the reason shaders are invented and of course you gotta keep the script kiddies happy all the time. 
Anything that can speedup the processing is helpful. Notice the Joint has a field for range of motion and stiffness. How will you finally do that even though it could be prototyped with a script? The progeres will generally be to develop it using a scipt and if it works then do the shader. 

How do you want to handle the problem of animating the skin? 
It is just a detail that each skin vertex is controlled by rotation of one or more joints in the hierarchy plus maybe a couple of other vertex controllers? 
The important item might be how you document what you want to happen in the authortime and then maintain and update that data in your runtime. I don’t know, just that, for instance, the humanoid with segment geometry can easily be animated with ‘standard’ long-standing features of vrml/x3d but when you incorporate realtime deformable mesh skin driven by skeleton and other types of mesh manipulators, then it seems to have needed a step in development of understanding and processing technique. Of course today all that is so common.  

> skeletal animation can be done without vertex shaders and VRML scripes

Who say that? Don’t let ‘em out the door. 
Well, figure out what is needed then please tell me. The skin is just a ‘continious’ closed surface that can move each frame. What does the vertex shader do if the vertex moves each frame? This is not new stuff and although the basics have not changed for years, the industry has obviously done everything possible to speed up everything possible. Even if you don’t need realtime because the scene will be put together with other stuff later, everyone needs to render the frame as fast as possible. Of course animation can be done by script, as in the original boxman example, but that won’t go too far for complex models even though scripts are a lot faster now than then, but anyway just try some of the stuff and see. Please, somebody simplify this stuff, but not just by building a black box around it. 

If you really want a lesson in skeleton animation, just look at the industry leaders for the last 20 years and look for a button that selects “Biped”, or like that and the thing will spit out essentially hanim LOA2 skeleton for you and help you bind a skin and get animations. There are attempts at tutorials from all majors. If it was easy, then anyone can do it. If it is hard, then make it easier, if you can. However, having built the thing in some system, also having the tool export the model to x3d with all the documentation and realtime animation code is another topic. x3d documents the practical interfaces and represents what is under the covers that you can carry with you. 

Anyway, speed is why the boxman example had about 50 or so points and the joekick skin example only has about 400 skin vertices. More points, more processing, more need for speed. That is why the gltf would apparently (pre)organize the data by vertex for the accelerated runtime, because they want to speed up skin updates and that is the way the hardware likes it. That probably helped speed a lot regardless of the way the data is actually organized internally by the authoring tool and allows a known path to transport the common and in this case very structured data between applications.  

So to pick your tools and optimize, when you think you have the ideal processing setup for full hanim, then think of adding the physics to the humanoid. Get the thing walking around by itself:). Since the deepest purpose of all of VRML and x3d is to benefit creation of the best transportable realistic hanim interacting in transportable augmented environments, then whatever you can do to add to realtime realism is great. 

John, have you run and read that last example I sent? 

All Best, 
Joe

From: John Carlson
Sent: Thursday, November 29, 2018 12:57 PM
To: Joseph D Williams; GPU Group
Cc: X3D Graphics public mailing list
Subject: RE: [x3d-public] HAnim and glTF skins

Joseph, are you basically saying skeletal animation can be done without vertex shaders and VRML scripes?

That’s good news.

John 

Sent from Mail for Windows 10

From: Joseph D Williams
Sent: Thursday, November 29, 2018 1:47 PM
To: John Carlson; GPU Group
Cc: X3D Graphics public mailing list
Subject: RE: [x3d-public] HAnim and glTF skins

➢ Andreas - Another feature in glTF are morph targets which are independent of skins. They are similar to HAnim displacers but I have not looked very closely if it is possible to define a mapping between them. 

The thing that maybe special in HAnim is that the displacers applied to skin are additive to other mesh animation. I think you will find that morph thingees will be used with other skin animation techniquess  

HAnimDisplacer
coordIndex    [  numerical index of each parent geometry vertex 
   to be animated in the same order as they appear 
   in the user code for the target geometry ]
displacements  [  x,y,z in skeleton space 
   maximum displacement of each vertex  ]
weight        0 to 1 animation control 
   (linear interpolation)
   0 = same location as parent mesh initial vertex 
   1 = defined maximum displacement location of parent mesh vertex 

Parent geometry is either the Humanoid, if the Displacer is applied to the mesh in the skin node, or, the geometry of the Displacer node's parent Segment. The displacements xyz data gives a relative target maximum displacement, sort of like there is vector that points from the current vertex position to the location of maximum displacement and the current vertex is moved along that vector according to the weight input. 

Displacer nodes require detailed knowledge of the parent mesh. Specifically, the vertex order of the parent mesh must not change between authoring and rendering. Compared to CoordinateInterpolator, for example, since all vertices of the mesh may not need to be animated for a sequence, only the vertices to be moved are included. If the mesh is being animated by another process, then those results and all displacer(s) results must be applied to the entire mesh in the same frame. 

In fact, what would be the problem, since the name of the node in X3D is HAnimDisplacer, the name Displacer is available for x3d, Why not figure out how to do at least a simple one for V4, or change the name. Canvas the industry and see the various names they call these things, then pick something different. The data and execution of the displacer is like other things called various names that so the same thing, as it has always been. 

Also, if a good goal is to bring in some more simplified form of articulation-driven mesh animation, why not define a simple root-Joints-segments-skin hierarchy setup using some plain names for the animation nodes and skin bindings. I mean, whatever. we might as well call a joint a rotary, fixed center of rotation actuator. And, of course we need joint-like things, only the most perverse (mis)understanding of the fact that bone orientation the same as joint rotation wants to do away with joints.  

I can see why a gltf vertex contains a list of actuators and weights while hanim uses the actuator to hold the list of vertices and weights of the mesh it animates. Some gltf code may show a minimum of 16 actuaor realtionships per vertex is ‘standard’ with bunches of zeros, beca the hdwr wants it that way. That is wasteful and from what I have seen x3d would not make it the prime directive that the authortime  has to look like the runtime. That has always been. There is runtime and there is authortime. HAnim, is concerned with composability and clear documentation. Sure, it may depend upon how you learned it, but for authortime, if you take care of it at the jointss, then the mesh will be ok.  

The gltf binary is for under the covers, more for rendering than for authoring. The runtimes want the data in gltf form and hanim says that is fine, take the way we do it and create your binary, maybe using a style sheet, or vice-versa. Every current x3d runtime converts the x3d utf-8 text to a target binary form at some point. That is why a set of properly structured binaries might be able to be imported rather easily because the runtimes probably use that form anyway, just input in different, more abstractly-connected containers than x3d hanim. The names and readability are different while the data, the data structures, and the necessary connections are the same under the covers. I hope all the gltf and x3d data types match. gltf is fine, as long as I can edit the source. 

➢ John - My guess is the rig + bvh + something is what we need for the skin deformations.   This is the first time I’ve tried to animate a human, but I’ve done some head shots.

HI John. maybe look at the attachment, this one with some animation. You can play with the BVH with Don, but that is not the native form and is not really required since it is easy to create animations by hand, When you start talking skin, then it is a good step up in understanding and processing and rendering so you don’t have to jump right into the skin part of it. Andreas put up a link to some hanim x3dom examples. 

Best Wishes
Joe

From: John Carlson
Sent: Thursday, November 29, 2018 9:39 AM
To: GPU Group
Cc: X3D Graphics public mailing list
Subject: Re: [x3d-public] HAnim and glTF skins

My guess is the rig + bvh + something is what we need for the skin deformations.   This is the first time I’ve tried to animate a human, but I’ve done some head shots.

On Thu, Nov 29, 2018 at 10:02 AM GPU Group <gpugroup at gmail.com> wrote:
PS I loved this little tool for making humans
http://www.makehumancommunity.org/
and used the humans in blender for static shot raytrace 
https://sites.google.com/site/commutar/
- see human shots near bottom
x but the export options don't include glTF or x3d
* 
https://github.com/makehumancommunity/makehuman
* collada .dae
* filmbox .fbx
* makehuman exchange (mhx2
* wavefront obj
* ogre 3D
* Stereolithography (stl)
Rig format
* Biovision Hierarch BVH

Q1. what do you guys use?
Q2. should makehuman export something we like, and if so what?
Thanks, Doug Sanden (freewrl)

On Thu, Nov 29, 2018 at 7:34 AM Andreas Plesch <andreasplesch at gmail.com> wrote:
My interest was in supporting animation in glTF inlines  in x3dom. With basic animation pretty much solved as TimeSensor/Interpolator/Route combos, the next step is skinned animation. Since HAnim is actually not strictly limited to humanoids (if the H is ignored), it appears more and more feasible to use it to represent glTF skins and skeletons in a fairly complete manner. At least this is what my first manual translations suggest. I am not familiar with other skinned animation systems but most can export glTF.

Another feature in glTF are morph targets which are independent of skins. They are similar to HAnim displacers but I have not looked very closely if it is possible to define a mapping between them.

One difference is that morph targets can be used outside of a joint hierarchy (skeleton) in glTF whereas displacers are confined to joints and segments within a x3d skeleton. Until displacers are released into the wild, it would mean that glTF meshes with morph targets have to be wrapped in a humanoid node. This seems possible. 

Implementing HAnim vertex wise deformation ultimately needs to be in vertex shaders. This may actually mean using a glTF skin representation since it is perhaps more shader friendly.

-Andreas

---on the phone---

Date: Wed, 28 Nov 2018 16:35:17 -0800
From: Leonard Daly <Leonard.Daly at realism.com>
To: x3d-public at web3d.org
Subject: Re: [x3d-public] HAnim and glTF skins
Message-ID: <20125083-023f-b96a-6586-ab8e6d3ea128 at realism.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"

Andreas,

I have read all of the messages with this subject. I am responding to 
the first one because I think that is best where my question arises.

Are you working on an understanding and implementation of H-Anim skinned 
animation in X3D or the more general animation of skinned surfaces that 
is done in Maya/Blender/+ in most every other application/use?

If you are doing the more general case would it make more sense to get a 
good understanding of the process and work-flow that is used in those 
applications and practices?

If the intent is to use H-Anim with X3D, you will need to resolve the 
differences between X3D V3.3/H-Anim V1.0 and X3D V4 (not yet in 
existence)/H-Anim V2.0 (perhaps); as you have noted in one of the other 
messages in this chain.

Leonard Daly

> I am collecting information on these skinning systems here:
>
> https://github.com/andreasplesch/x3dom/wiki/HAnim-and-glTF-skins
>
> [This is a editable wiki if anybody wants to add to it]
>
> It looks like the less well known
>
> jointBindingPositions/Rotations/Scales and skinBindingCoords/Normals
>
> fields used for non-humanoid skeletons may be useful but I am not
> certain how to interprete those fields. Castle may not implement those
> as a search did not find a match ?
>
> My understanding is that the transformation matrix defined by these
> fields needs to applied before joint trafos are applied. But to which
> vertices ? The ones listed in skinBindingCoords for sure (if they are
> indexed by skinIndex field in the joint) but how does
> skinBindingCoords and skinCoords field interact ? Does the
> skinBindingCoords field take precedence ? But why then have both
> fields in the first place ?
>
> Could the jointBinding fields equivalently be replaced by a sub-Joint
> node under the joint the fields map to ?
>
> Is there an example using these fields ?
>
> Thanks,
>
> -Andreas
>
>
>
>

-- 
*Leonard Daly*
3D Systems & Cloud Consultant
LA ACM SIGGRAPH Past Chair
President, Daly Realism - /Creating the Future/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20181128/f49fe788/attachment-0001.html>

------------------------------

From: Leonard Daly
Sent: Wednesday, November 28, 2018 4:35 PM
To: x3d-public at web3d.org
Subject: Re: [x3d-public] HAnim and glTF skins

Andreas,

I have read all of the messages with this subject. I am responding to the first one because I think that is best where my question arises.

Are you working on an understanding and implementation of H-Anim skinned animation in X3D or the more general animation of skinned surfaces that is done in Maya/Blender/+ in most every other application/use?

If you are doing the more general case would it make more sense to get a good understanding of the process and work-flow that is used in those applications and practices? 

If the intent is to use H-Anim with X3D, you will need to resolve the differences between X3D V3.3/H-Anim V1.0 and X3D V4 (not yet in existence)/H-Anim V2.0 (perhaps); as you have noted in one of the other messages in this chain.

Leonard Daly

I am collecting information on these skinning systems here:

https://github.com/andreasplesch/x3dom/wiki/HAnim-and-glTF-skins

[This is a editable wiki if anybody wants to add to it]

It looks like the less well known

jointBindingPositions/Rotations/Scales and skinBindingCoords/Normals

fields used for non-humanoid skeletons may be useful but I am not
certain how to interprete those fields. Castle may not implement those
as a search did not find a match ?

My understanding is that the transformation matrix defined by these
fields needs to applied before joint trafos are applied. But to which
vertices ? The ones listed in skinBindingCoords for sure (if they are
indexed by skinIndex field in the joint) but how does
skinBindingCoords and skinCoords field interact ? Does the
skinBindingCoords field take precedence ? But why then have both
fields in the first place ?

Could the jointBinding fields equivalently be replaced by a sub-Joint
node under the joint the fields map to ?

Is there an example using these fields ?

Thanks,

-Andreas

-- 
Leonard Daly
3D Systems & Cloud Consultant
LA ACM SIGGRAPH Past Chair
President, Daly Realism - Creating the Future 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20181128/d30e39f7/attachment.html>

------------------------------

Subject: Digest Footer

_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org

------------------------------

End of x3d-public Digest, Vol 116, Issue 52
*******************************************
_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org
_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20181129/9c60080b/attachment-0001.html>