[x3d-public] Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1

Wed Apr 24 08:10:06 PDT 2024

Thanks for the close look at these capabilities, Joe and Carol.  Glad it is worth review for possible benefits.

To your question: it is possible to extend LOA1 and LOA2 within any individual model, simply by adding additional joints/segments feature points, geometry and skin.  The only limitation is that behavior animations (interpolators or HAnimMotion nodes) will not be targeting those added nodes.  Often such differences are not a problem, since LOA1 and LOA2 joints are carefully chosen to be sufficient for meaningful motion.

If a suitable MPAI example is available, may I suggest close examination and then remodeling using HAnim2/X3D4.  That will allow precise comparison and correlation between the representations.  It will be interesting to see how much alignment is possible.

Proposing changes is always possible, but given the existing functionality and flexibility of LOA1 and LOA2, the threshold for accepting such changes would be to show (again through examples) that significant benefit would result.

Meanwhile, our current specification-update roadmap:

*	19775-2 SAI 4.0 working draft ready for submission to Web3D/ISO,
*	19776-1 XML 4.0 working draft essentially complete, inviting comments and finishing review of issues,
*	19776-2 ClassicVRML 4.0 working draft update has started,
*	HAnim2/X3D4 possible additions of feature points and errata fixes can be considered,
*	Submitting ISO New Work Item Proposals (NWIPs) for SC24 consideration to occur in June.
*	Am expecting Web3D will have successfully transitioned to an annual specification-update cycle at that point.

Looking forward to future steady evolution and progress.

all the best, Don

-- 

Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu

Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149

X3D graphics, virtual worlds, navy robotics https://faculty.nps.edu/brutzman

From: Carol McDonald <cemd2 at comcast.net> 
Sent: Wednesday, April 24, 2024 6:26 AM
To: John Carlson <yottzumm at gmail.com>; Joe D Williams <joedwil at earthlink.net>
Cc: Anita Havele <anita.havele at web3d.org>; Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; Doug Sanden <gpugroup at gmail.com>; Emma Scott <connect at fashionshouldempower.com>; Holger Seelig <holger.seelig at yahoo.de>; Humanoid Animation (H-Anim) Working Group <h-anim at web3d.org>; Katy Schildmeyer <katy at ksappareldesign.com>; Myeong Won Lee <myeongwonlee at gmail.com>; Richard F. Puk <puk at igraphics.com>; William Glascoe <eosocxo at comcast.net>; x3d-public at web3d.org
Subject: Re: Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1

This is awesome.  Let me look at this email and comments, and the linked document and see the adds.  I noticed that in the linked document that they reference LOA0 - LOA4.  Does this mean that we can or can not propose changes to LOA1 or LOA2?  Or that we let this group know of any updates?  

Carol  

On 04/23/2024 9:31 PM PDT John Carlson <yottzumm at gmail.com <mailto:yottzumm at gmail.com> > wrote: 

Hi Joe,  this was a bit long for me to get through, but I’ve already suggested that “on the floor between the feet” is not precise enough, in my mind, and I suggest the x location on the floor:  between ankles,  knees, hips, between any left, right pair of foot joints, or at the x location of a backbone joint  or the sacroiliac. 

“Between the feet” offers too far of a range of x locations in my mind, and even z is not well defined in my mind. 

On Tue, Apr 23, 2024 at 10:34 PM Joe D Williams <joedwil at earthlink.net <mailto:joedwil at earthlink.net> > wrote: 

https://mpai.community/wp-content/uploads/2023/10/Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1.pdf <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmpai.community%2Fwp-content%2Fuploads%2F2023%2F10%2FTechnical-Specification-Portable-Avatar-Format-MPAI-PAF-V1.pdf&data=05%7C02%7Cbrutzman%40nps.edu%7Cb6c618bca87241de74ee08dc64622ca1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638495623330616001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=SwKyVlFACHfN7jYlQHuaMlUXY2A4jMy101jjV4Ud0L0%3D&reserved=0> 

This is important HAnim stuff.  
Some basic concepts of character structure and animation best practice 
are being defined for the PAF. 

So, let's please figure a way to give some HAnim best practice guidance 
to the group. 

The HAnim standard was mentioned and followed in several instances but in important ways abandoned. To be commented on below.  

Please recall that the entire reason for x3d is HAnim Humanoid and its environment and the entire purpose of HAnim is authoring transportable Humanoids using transportable animations and interactions. 

A similar character should be able to fairly easily adapt, or even use directly, animations and interactions that work on a similar model. 
So, for skeleton-driven animations, this means a standard but customizable skeleton hierarchy and 
a standard but customizable set of surface feature interaction points.  
And most importantly, a standardized default pose before animation. 

First, HAnim proves a realistic skeleton, one that is capable of realistic simulations of realtime stimulus/response interactions with the host environment and other Humanoids. 

That is the main idea, so to have this transportable device, called a Body Model here, that can actually represent a You, we must apply some standrds. 

"7.3 Body
7.3.1 Body Model"

Is: 
"MPAI adopts the Humanoid animation (HAnim) architecture [9] that gives access to the joint and 
end-effector hierarchy of a human figure. This allows a model-independent animation of a skeleton 
and related skin vertices associated with joints and geometry/accessories/sensors of individual 
body segments and sites."  

Suggestion:  
Humanoid Animation (HAnim)  
Great! Abstract, but says a (realistic) hierarchical body composed of Joints for animation, Segments for vizualization geometry, and interaction Sites for sensors or whatever.   
And maybe a continuous deformable mesh skin. 

Is:
"The actual structure of the HAnim architecture depends on the selected element of the Level Of 
Articulations (LOA) hierarchy: LOA 1, LOA 2, LOA 3, or LOA 4. All joints of an HAnim figure 
are represented as a tree hierarchy starting with the humanoid_root joint. For an LOA 1 character, 
there are 18 joints and 18 segments in the hierarchy."

Suggestion: Fantastic!  
https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#BasicSetJointHierarchy4  

New Sentence:   
The HAnim architecture consists of nested transforms called Joint, Segment, and Site. 
The transform hierarchy is operated by applying rotation to various Joint objects in the hierarchy. 
The hierarchy is realistic so,for instance, the motion of a shoulder joint will result in the expected motion of the arm and hand child parts.   
In addition, the character is equipped with a defined set of surface feature 
landmarks used for viewer, geometry, sensor, and anything x3d.   
The Site features are certain children of Segment nodes and are animated with the Segment controlled by the parent Joint. 

Is:
"The bones of the body are described starting 
from position (x0,y0,z0) of the root (neck or 
pelvis)." 

Should be:
The Joint center values of the skeleton and the interaction landmarks 
are x y z dimensioned relative to 0 0 0 at the floor between the feet. 

This is measured using the standard default before animation pose.  
The rotation of all Joint objects in default pose is defined at 0 0 1 0 radians; x=y=z=zero degrees. 

Is:   
"The orientation of a bone attached to the root 
is defined by (α,β,γ) where α is the angle of the 
bone with the x axis, and so on. The joint of a 
bone attached to the preceding bone has a 
position (x1,y1,z1) determined by the angles 
(α1,β1,γ1) and the length of the bone."  

Suggetion:  

Bones Not important since the basis for the skeleton is Joint centers, not intersections of bones connecting them. The HAnim hierarchy is based on Joint centers and the orientation and length of a bone is not a data point in operation.   
Animation of the skeleton and body is all about Joint center rotations.  
Under the covers it is all about Joint centers so the author must be exposed to Joint centers as the base measurement landmarks and animation vehicles of the body. Although lucky that if rigged right, bone orientation and Joint rotation are same. 

Is:  
"The Body Model contains:
1. Pose composed by: 
1.1. The position of the root.
1.2. The angles of the bones with the 
(x,y,z) coordinate axes.
1.3. The orientation of the body 
defined by 3 angles."

Suggestion:   
The Humanoid Body Model contains:  
1. Pose composed of:  
1.1. The default pose prior to animation. 
This is relaxed attention, facing +z, +y up, and +x to model left.    
1.2. The 0 0 0 reference location for the model.  
At the floor, between the feet since all default pose Joint center and surface landmark dimensions 
are referenced relative to 0 0 0 for default pose.  
1.3. In the default pose before animation all Joint center transforms are default 0 0 1 0 x=y=z=zero  
1.4. The Humanoid has two levels of basic complexity.   
Level 1 uses Segment geometry.   
Level 2 continuous mesh skin plus Segment geometry.   
Level 1 Segment geometry is animated directly by the parent Joint.  
Level 2 skin is animated using weighted value sum from multiple Joint objects.   
HAnim Displacer node can additively control geometry vertices. 

Is:  
"2. The standard bone lengths.  
3. Lengths of the bones of the specific 
model."  

Suggestion:
These are not important in HAnim except figuring Joint center values for default pose. 

Is:  
4. Surface-related  
4.1. Surface  
4.2. Texture  
4.3. Material  
4.4. Cloth (an integral part of the model)"  

Suggestion:   
4. Surface features 
4.1. Surface can be represented by collections of geometry contained by Segment objects. 
The geometry can be textured and have complex surface features.  
Site objects can be decorative or functional geometery, sensors, vieweres, etc.  
4.2. clothing and accessories can be included in skin geometry, or drawn individually and 
utilized as segment geometry.  

Artwork:   
Figure 9. Some Joints of the Body Model. 

Suggestion:  
OK except that vl5 should be the base of the spine and a vc7 should be present 
to parent the skull and shoulders. Having the shoulders parented too low on the 
spine is a handicap. Please show minimum LOA2 skeleton for this. 

Is:  
"The Body Model is represented in the glTF format and transmitted in the initial Portable Avatar at 
the beginning of the Avatar-Based Videoconference.  
The Spatial Attitude of a Body is defined with respect to the global coordinate system defined by
the Visual Environment."   

Suggestion:  
The Body Model skeleton, surface geometry, accessories, and personal animations are 
represented in various forms including x3d HAnim, various gltf assets, ascii, binary, encripted formats, various image and sound formats contained by the standard Portable Avator package. 
The Body Model is scaled 1:1 in meters (human scale) aimed at a standard host environment 
1:1 scale.   

Is:  
"7.3.2 Body Descriptors
Body Descriptors are included in the data set describing the root and joints as:
1 Position and Orientation of the root.
2 Rotation angles of the joints.
The rotation of the head is treated as any other joint.

Suggested:  

Body Descriptors are from Default Pose with elements of data set in correct location in the skeleton hierarchy:   
(1) Joint Center x y z values relative to 0 0 0.   
(2) Surface Feature Landmarks x y z values relative to 0 0 0.    
Animation data consisting of Root translation and rotation, Joint rotation, and or special aspects of body pose, geometry motion, sensor interactions may be generated in real time using internal or external (from host) scripts or by internal preprogrammed interpolators.  

Is: 
"Figure 10 depicts Roll (rotation around y), Pitch (rotation around y, and Yaw (rotation around z) 
of a Body"  

Suggested:
Figure 10 depicts 
Pitch (rotation around x), 
Yaw (rotation around y), and 
Roll (rotation around z) of a Body.  

Figure 10 is not HAnim.   
Unclear whether right- or left-hand rule.
  HAnim is right-hand rule. 
Axes are mixed up from HAnim.  
  The Figure 10 Body is facing -x,
  +z to body left, 
  +y up. 

The HAnim Body Model in default before animation pose is:   
facing +z,  
up is +y. and   
+x to body left.  

Please adopt the HAnim std coordinate system, same as default web3d x3d coordinate system.   

Not so much about x and y, most every system can produce a default skeleton for x and y with +x to viewer's right and Body Model left, and +y up as standard graphing practice.    
But picking the correct direction for +z, and thus the Body Model default has been a topic for generations. To settle it, just think that relatively, the audience looking at the Body Model is facing -z with minus value increasing with distance and the Body Model is facing +z with plus value increasing with distance.   
Important that authors honor and even demand this because negative scaling is out for this round 
and that really didn't fix it anyway.   

For rotation of axes, right-hand rule.
Thumb pointed toward + of axis then curl of fingers = + positive rotation.  

When you are facing the body as a viewer, your gaze is toward -z, 
+x is to your right, and +y is up.   
The Body gaze is +z, 
+x is body left, and +y is up. 

It is not a mirror, it is a Body Model looking at you.   

So, Figure 10 needs replacement for consistency. 

Is:  
"7.3.3 Head Descriptors  
The Head Descriptors are the angles of: 
1. Roll: head has moved toward one of the shoulders.
2. Pitch: head has moved up and down.
3. Yaw: head has rotated left to right (around the vertical axis of the head).
These are useful in case when a body-less head is animated."  

Fine, this is like HAnim but not like Figure 10.  

Is:  
"7.4 Face  
7.4.1 Face Model"   

Just hoping HAnim facial animation evolving 
is pretty much the same or slightly advanced. 

Attached is latest JinScaledL1LOA4 with Sites. 
To me,this would be the best current example of an HAnim 
character with Segment Geometry and showing V2 surface 
feature landmarks. 
The user code is fairly easy to read in Notepad. 
Maybe skip the front interface stuff down to HAnimHumanoid 
to read the loa4 skeleton, then have a look at the complete set of interpolators, routes timers to interpolators, routes interpolators to skeleton joints. In fact, multiple copies, one for each of the animations.   

View with any x3d tool or even use the X_ite Playground.
Just go to   

https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/SimpleInstancing/glTF/SimpleInstancing.gltf  

then paste this .x3d file in the text area replacing what is there. 
Future versions of this will have the old, much more pleasing motions as pitch2, yaw2, roll2 animations. 

Thanks and Best,
Joe

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240424/b3edb545/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5464 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240424/b3edb545/attachment-0001.p7s>