[x3d-public] Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1

[John Carlson]

I still think z=0 as Carol says.  Trying to clarify.  I don’t think z=0
should be at the toes

On Wed, Apr 24, 2024 at 3:33 PM John Carlson <yottzumm at gmail.com> wrote:

> I agree with this, but I think that HAnim at initial pose y=0 should be
> between the feet, on the floor.  Heels are okay, but really, people wear
> shoes.  Also one needs to consider shoes with quite a bit of lift.  y=0 is
> more appropriate towards the toe, rather than the heels.
>
> Maybe Joe’s description is best.
>
> John
>
> On Wed, Apr 24, 2024 at 2:12 PM Carol McDonald <cemd2 at comcast.net> wrote:
>
>> Since I am one of those people who always has slight twist, and one foot
>> is always slightly ahead of the other and can't seem to follow the
>> directions of heels aligned with each other, I would suggest the mid point
>> in the x direction (the frontal width of the body) between the heels would
>> be good if two legs.  If one leg, then average of the hip joints or width
>> or x plane of the sagittal plane (works better for women). As for the z
>> direction, average of the two hip joints, would work.
>>
>> Carol
>>
>> On 04/24/2024 11:59 AM PDT Emma Scott <connect at fashionshouldempower.com>
>> wrote:
>>
>>
>> Hi Joe,
>>
>> My group is assessing posture of the original CAESAR data set. Following
>> are a few broad scope findings which may contribute to this discussion:
>>
>> Scan pose generally requests aligned heels. While this is not always the
>> case it is a good assumption to start from, and appropriate scanning
>> protocol. A mid-point between the heel landmarks (calcaneus_posterior)
>> could be considered a good floor ground zero.
>>
>> The various body regions can be twisted in different directions. Having a
>> set floor point provides an adequate base point of reference for noting
>> such occurrences.
>>
>> The hip joint is significant for posture assessment and automated pose
>> detection. Hence, the floor point could be relocated in ‘z’ to be the
>> average of the two hip joints. Averaging is necessary due to the above
>> mentioned possibility of lower torso twist. Eluding to the importance of
>> posture within Hanim would be prudent. This could be a way of laying
>> groundwork for future discussion.
>>
>> Best Regards,
>> Emma Scott
>>
>>
>> On Apr 24, 2024, at 10:45 AM, Joe D Williams <joedwil at earthlink.net>
>> wrote:
>> > I’ve already suggested that “on the floor between the feet” is not
>> precise enough, in my mind, and I suggest the x location on the floor:
>>  between ankles,  knees, hips, between any left, right pair of foot joints,
>> or at the x location of a backbone joint  or the sacroiliac.
>>
>>
>> On the floor between the feet is a fairly constant starting point to base
>> all other measurements of the skeleton.
>> Any other location is variable, like basing somewhere else, like anywhere
>> else means measurements are relative to some point that will vary lots from
>> skeleton to skeleton.
>> Between the feet on the floor is a constant where all features then have
>> a x 0 z (y always positive value.
>> Notice the the document now says to reference it somewhere else, like
>> another joint, but that means a variable reference and thus less transport
>> between skeletons.
>> 0 0 0 should be on the floor and between the feet is as good a location
>> as any and bound to be better relationship between measurements of other
>> skeletons than referencing it to some point that is variable between
>> models.
>> If not 0 0 0 then, for instance there will not be an obvious height
>> value, and if not approximately in the center of the model, then animation
>> gets hard fast.  Can't really compare skeleton dimensions unless the
>> reference value is at the floor and approximately centered in x y z. Hard
>> to compare skeletons and share animations if measurement reference is at a
>> certain (variable) landmark (like a joint) (This was done when Root was not
>> used and some made 0 0 0 at some joint.)
>>
>>
>> Thanks and Best,
>> Joe
>>
>>
>>
>>
>>
>>
>> Thanks,
>> Joe
>>
>>
>>
>>
>>
>>
>>
>> -----Original Message-----
>> From: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> Sent: Apr 24, 2024 8:10 AM
>> To: Carol McDonald <cemd2 at comcast.net>, Joe D Williams <
>> joedwil at earthlink.net>
>> Cc: Anita Havele <anita.havele at web3d.org>, Doug Sanden <
>> gpugroup at gmail.com>, Emma Scott <connect at fashionshouldempower.com>,
>> Holger Seelig <holger.seelig at yahoo.de>, John Carlson <yottzumm at gmail.com>,
>> Humanoid Animation (H-Anim) Working Group <h-anim at web3d.org>, Katy
>> Schildmeyer <katy at ksappareldesign.com>, Myeong Won Lee <
>> myeongwonlee at gmail.com>, Richard F. Puk <puk at igraphics.com>, William
>> Glascoe <eosocxo at comcast.net>, x3d-public at web3d.org <x3d-public at web3d.org>,
>> Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> Subject: RE: Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1
>>
>>
>>
>> Thanks for the close look at these capabilities, Joe and Carol.  Glad it
>> is worth review for possible benefits.
>>
>>
>> To your question: it is possible to extend LOA1 and LOA2 within any
>> individual model, simply by adding additional joints/segments feature
>> points, geometry and skin.  The only limitation is that behavior animations
>> (interpolators or HAnimMotion nodes) will not be targeting those added
>> nodes.  Often such differences are not a problem, since LOA1 and LOA2
>> joints are carefully chosen to be sufficient for meaningful motion.
>>
>>
>> If a suitable MPAI example is available, may I suggest close examination
>> and then remodeling using HAnim2/X3D4.  That will allow precise comparison
>> and correlation between the representations.  It will be interesting to see
>> how much alignment is possible.
>>
>>
>> Proposing changes is always possible, but given the existing
>> functionality and flexibility of LOA1 and LOA2, the threshold for accepting
>> such changes would be to show (again through examples) that significant
>> benefit would result.
>>
>>
>> Meanwhile, our current specification-update roadmap:
>>
>>    - 19775-2 SAI 4.0 working draft ready for submission to Web3D/ISO,
>>    - 19776-1 XML 4.0 working draft essentially complete, inviting
>>    comments and finishing review of issues,
>>    - 19776-2 ClassicVRML 4.0 working draft update has started,
>>    - HAnim2/X3D4 possible additions of feature points and errata fixes
>>    can be considered,
>>    - Submitting ISO New Work Item Proposals (NWIPs) for SC24
>>    consideration to occur in June.
>>    - Am expecting Web3D will have successfully transitioned to an annual
>>    specification-update cycle at that point.
>>
>>
>>
>> Looking forward to future steady evolution and progress.
>>
>>
>> all the best, Don
>>
>> --
>>
>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>>
>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>>
>> X3D graphics, virtual worlds, navy robotics
>> https://faculty.nps.edu/brutzman
>>
>>
>> *From:* Carol McDonald <cemd2 at comcast.net>
>> *Sent:* Wednesday, April 24, 2024 6:26 AM
>> *To:* John Carlson <yottzumm at gmail.com>; Joe D Williams <
>> joedwil at earthlink.net>
>> *Cc:* Anita Havele <anita.havele at web3d.org>; Brutzman, Donald (Don)
>> (CIV) <brutzman at nps.edu>; Doug Sanden <gpugroup at gmail.com>; Emma Scott <
>> connect at fashionshouldempower.com>; Holger Seelig <holger.seelig at yahoo.de>;
>> Humanoid Animation (H-Anim) Working Group <h-anim at web3d.org>; Katy
>> Schildmeyer <katy at ksappareldesign.com>; Myeong Won Lee <
>> myeongwonlee at gmail.com>; Richard F. Puk <puk at igraphics.com>; William
>> Glascoe <eosocxo at comcast.net>; x3d-public at web3d.org
>> *Subject:* Re: Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1
>>
>>
>> This is awesome.  Let me look at this email and comments, and the linked
>> document and see the adds.  I noticed that in the linked document that they
>> reference LOA0 - LOA4.  Does this mean that we can or can not propose
>> changes to LOA1 or LOA2?  Or that we let this group know of any updates?
>>
>>
>> Carol
>>
>> On 04/23/2024 9:31 PM PDT John Carlson <yottzumm at gmail.com> wrote:
>>
>>
>>
>> Hi Joe,  this was a bit long for me to get through, but I’ve already
>> suggested that “on the floor between the feet” is not precise enough, in my
>> mind, and I suggest the x location on the floor:  between ankles,  knees,
>> hips, between any left, right pair of foot joints, or at the x location of
>> a backbone joint  or the sacroiliac.
>>
>>
>> “Between the feet” offers too far of a range of x locations in my mind,
>> and even z is not well defined in my mind.
>>
>>
>> On Tue, Apr 23, 2024 at 10:34 PM Joe D Williams <joedwil at earthlink.net>
>> wrote:
>>
>>
>>
>> https://mpai.community/wp-content/uploads/2023/10/Technical-Specification-Portable-Avatar-Format-MPAI-PAF-V1.pdf
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmpai.community%2Fwp-content%2Fuploads%2F2023%2F10%2FTechnical-Specification-Portable-Avatar-Format-MPAI-PAF-V1.pdf&data=05%7C02%7Cbrutzman%40nps.edu%7Cb6c618bca87241de74ee08dc64622ca1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638495623330616001%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=SwKyVlFACHfN7jYlQHuaMlUXY2A4jMy101jjV4Ud0L0%3D&reserved=0>
>>
>>
>> This is important HAnim stuff.
>> Some basic concepts of character structure and animation best practice
>> are being defined for the PAF.
>>
>>
>> So, let's please figure a way to give some HAnim best practice guidance
>> to the group.
>>
>>
>> The HAnim standard was mentioned and followed in several instances but in
>> important ways abandoned. To be commented on below.
>>
>>
>> Please recall that the entire reason for x3d is HAnim Humanoid and its
>> environment and the entire purpose of HAnim is authoring transportable
>> Humanoids using transportable animations and interactions.
>>
>> A similar character should be able to fairly easily adapt, or even use
>> directly, animations and interactions that work on a similar model.
>> So, for skeleton-driven animations, this means a standard but
>> customizable skeleton hierarchy and
>> a standard but customizable set of surface feature interaction points.
>> And most importantly, a standardized default pose before animation.
>>
>> First, HAnim proves a realistic skeleton, one that is capable of
>> realistic simulations of realtime stimulus/response interactions with the
>> host environment and other Humanoids.
>> That is the main idea, so to have this transportable device, called a
>> Body Model here, that can actually represent a You, we must apply some
>> standrds.
>>
>> "7.3 Body
>> 7.3.1 Body Model"
>>
>>
>> Is:
>> "MPAI adopts the Humanoid animation (HAnim) architecture [9] that gives
>> access to the joint and
>> end-effector hierarchy of a human figure. This allows a model-independent
>> animation of a skeleton
>> and related skin vertices associated with joints and
>> geometry/accessories/sensors of individual
>> body segments and sites."
>>
>>
>> Suggestion:
>> Humanoid Animation (HAnim)
>> Great! Abstract, but says a (realistic) hierarchical body composed of
>> Joints for animation, Segments for vizualization geometry, and interaction
>> Sites for sensors or whatever.
>> And maybe a continuous deformable mesh skin.
>>
>> Is:
>> "The actual structure of the HAnim architecture depends on the selected
>> element of the Level Of
>> Articulations (LOA) hierarchy: LOA 1, LOA 2, LOA 3, or LOA 4. All joints
>> of an HAnim figure
>> are represented as a tree hierarchy starting with the humanoid_root
>> joint. For an LOA 1 character,
>> there are 18 joints and 18 segments in the hierarchy."
>>
>>
>> Suggestion: Fantastic!
>>
>> https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#BasicSetJointHierarchy4
>> <https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#BasicSetJointHierarchy4>
>>
>>
>> New Sentence:
>> The HAnim architecture consists of nested transforms called Joint,
>> Segment, and Site.
>> The transform hierarchy is operated by applying rotation to various Joint
>> objects in the hierarchy.
>> The hierarchy is realistic so,for instance, the motion of a shoulder
>> joint will result in the expected motion of the arm and hand child parts.
>> In addition, the character is equipped with a defined set of surface
>> feature
>> landmarks used for viewer, geometry, sensor, and anything x3d.
>> The Site features are certain children of Segment nodes and are animated
>> with the Segment controlled by the parent Joint.
>>
>>
>> Is:
>> "The bones of the body are described starting
>> from position (x0,y0,z0) of the root (neck or
>> pelvis)."
>>
>>
>> Should be:
>> The Joint center values of the skeleton and the interaction landmarks
>> are x y z dimensioned relative to 0 0 0 at the floor between the feet.
>>
>> This is measured using the standard default before animation pose.
>> The rotation of all Joint objects in default pose is defined at 0 0 1 0
>> radians; x=y=z=zero degrees.
>>
>>
>> Is:
>> "The orientation of a bone attached to the root
>> is defined by (α,β,γ) where α is the angle of the
>> bone with the x axis, and so on. The joint of a
>> bone attached to the preceding bone has a
>> position (x1,y1,z1) determined by the angles
>> (α1,β1,γ1) and the length of the bone."
>>
>>
>> Suggetion:
>> Bones Not important since the basis for the skeleton is Joint centers,
>> not intersections of bones connecting them. The HAnim hierarchy is based on
>> Joint centers and the orientation and length of a bone is not a data point
>> in operation.
>> Animation of the skeleton and body is all about Joint center rotations.
>> Under the covers it is all about Joint centers so the author must be
>> exposed to Joint centers as the base measurement landmarks and animation
>> vehicles of the body. Although lucky that if rigged right, bone orientation
>> and Joint rotation are same.
>>
>>
>> Is:
>> "The Body Model contains:
>> 1. Pose composed by:
>> 1.1. The position of the root.
>> 1.2. The angles of the bones with the
>> (x,y,z) coordinate axes.
>> 1.3. The orientation of the body
>> defined by 3 angles."
>>
>>
>> Suggestion:
>> The Humanoid Body Model contains:
>> 1. Pose composed of:
>> 1.1. The default pose prior to animation.
>> This is relaxed attention, facing +z, +y up, and +x to model left.
>> 1.2. The 0 0 0 reference location for the model.
>> At the floor, between the feet since all default pose Joint center and
>> surface landmark dimensions
>> are referenced relative to 0 0 0 for default pose.
>> 1.3. In the default pose before animation all Joint center transforms are
>> default 0 0 1 0 x=y=z=zero
>> 1.4. The Humanoid has two levels of basic complexity.
>> Level 1 uses Segment geometry.
>> Level 2 continuous mesh skin plus Segment geometry.
>> Level 1 Segment geometry is animated directly by the parent Joint.
>> Level 2 skin is animated using weighted value sum from multiple Joint
>> objects.
>> HAnim Displacer node can additively control geometry vertices.
>>
>>
>> Is:
>> "2. The standard bone lengths.
>> 3. Lengths of the bones of the specific
>> model."
>>
>>
>> Suggestion:
>> These are not important in HAnim except figuring Joint center values for
>> default pose.
>>
>>
>> Is:
>> 4. Surface-related
>> 4.1. Surface
>> 4.2. Texture
>> 4.3. Material
>> 4.4. Cloth (an integral part of the model)"
>>
>>
>> Suggestion:
>> 4. Surface features
>> 4.1. Surface can be represented by collections of geometry contained by
>> Segment objects.
>> The geometry can be textured and have complex surface features.
>> Site objects can be decorative or functional geometery, sensors,
>> vieweres, etc.
>> 4.2. clothing and accessories can be included in skin geometry, or drawn
>> individually and
>> utilized as segment geometry.
>>
>>
>> Artwork:
>> Figure 9. Some Joints of the Body Model.
>>
>>
>> Suggestion:
>> OK except that vl5 should be the base of the spine and a vc7 should be
>> present
>> to parent the skull and shoulders. Having the shoulders parented too low
>> on the
>> spine is a handicap. Please show minimum LOA2 skeleton for this.
>>
>>
>> Is:
>> "The Body Model is represented in the glTF format and transmitted in the
>> initial Portable Avatar at
>> the beginning of the Avatar-Based Videoconference.
>> The Spatial Attitude of a Body is defined with respect to the global
>> coordinate system defined by
>> the Visual Environment."
>>
>>
>> Suggestion:
>> The Body Model skeleton, surface geometry, accessories, and personal
>> animations are
>> represented in various forms including x3d HAnim, various gltf assets,
>> ascii, binary, encripted formats, various image and sound formats contained
>> by the standard Portable Avator package.
>> The Body Model is scaled 1:1 in meters (human scale) aimed at a standard
>> host environment
>> 1:1 scale.
>>
>>
>> Is:
>> "7.3.2 Body Descriptors
>> Body Descriptors are included in the data set describing the root and
>> joints as:
>> 1 Position and Orientation of the root.
>> 2 Rotation angles of the joints.
>> The rotation of the head is treated as any other joint.
>>
>> Suggested:
>> Body Descriptors are from Default Pose with elements of data set in
>> correct location in the skeleton hierarchy:
>> (1) Joint Center x y z values relative to 0 0 0.
>> (2) Surface Feature Landmarks x y z values relative to 0 0 0.
>> Animation data consisting of Root translation and rotation, Joint
>> rotation, and or special aspects of body pose, geometry motion, sensor
>> interactions may be generated in real time using internal or external (from
>> host) scripts or by internal preprogrammed interpolators.
>>
>> Is:
>> "Figure 10 depicts Roll (rotation around y), Pitch (rotation around y,
>> and Yaw (rotation around z)
>> of a Body"
>>
>>
>> Suggested:
>> Figure 10 depicts
>> Pitch (rotation around x),
>> Yaw (rotation around y), and
>> Roll (rotation around z) of a Body.
>>
>>
>> Figure 10 is not HAnim.
>> Unclear whether right- or left-hand rule.
>>   HAnim is right-hand rule.
>> Axes are mixed up from HAnim.
>>   The Figure 10 Body is facing -x,
>>   +z to body left,
>>   +y up.
>> The HAnim Body Model in default before animation pose is:
>> facing +z,
>> up is +y. and
>> +x to body left.
>> Please adopt the HAnim std coordinate system, same as default web3d x3d
>> coordinate system.
>>
>> Not so much about x and y, most every system can produce a default
>> skeleton for x and y with +x to viewer's right and Body Model left, and +y
>> up as standard graphing practice.
>> But picking the correct direction for +z, and thus the Body Model default
>> has been a topic for generations. To settle it, just think that relatively,
>> the audience looking at the Body Model is facing -z with minus value
>> increasing with distance and the Body Model is facing +z with plus value
>> increasing with distance.
>> Important that authors honor and even demand this because negative
>> scaling is out for this round
>> and that really didn't fix it anyway.
>>
>>
>> For rotation of axes, right-hand rule.
>> Thumb pointed toward + of axis then curl of fingers = + positive
>> rotation.
>> When you are facing the body as a viewer, your gaze is toward -z,
>> +x is to your right, and +y is up.
>> The Body gaze is +z,
>> +x is body left, and +y is up.
>>
>>
>> It is not a mirror, it is a Body Model looking at you.
>> So, Figure 10 needs replacement for consistency.
>>
>>
>> Is:
>> "7.3.3 Head Descriptors
>> The Head Descriptors are the angles of:
>> 1. Roll: head has moved toward one of the shoulders.
>> 2. Pitch: head has moved up and down.
>> 3. Yaw: head has rotated left to right (around the vertical axis of the
>> head).
>> These are useful in case when a body-less head is animated."
>>
>>
>> Fine, this is like HAnim but not like Figure 10.
>>
>>
>> Is:
>> "7.4 Face
>> 7.4.1 Face Model"
>>
>>
>> Just hoping HAnim facial animation evolving
>> is pretty much the same or slightly advanced.
>>
>> Attached is latest JinScaledL1LOA4 with Sites.
>> To me,this would be the best current example of an HAnim
>> character with Segment Geometry and showing V2 surface
>> feature landmarks.
>> The user code is fairly easy to read in Notepad.
>> Maybe skip the front interface stuff down to HAnimHumanoid
>> to read the loa4 skeleton, then have a look at the complete set of
>> interpolators, routes timers to interpolators, routes interpolators to
>> skeleton joints. In fact, multiple copies, one for each of the animations.
>>
>>
>> View with any x3d tool or even use the X_ite Playground.
>> Just go to
>>
>>
>> https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/SimpleInstancing/glTF/SimpleInstancing.gltf
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcreate3000.github.io%2Fx_ite%2Fplayground%2F%3Furl%3Dhttps%3A%2F%2Fraw.githubusercontent.com%2FKhronosGroup%2FglTF-Sample-Models%2Fmaster%2F2.0%2FSimpleInstancing%2FglTF%2FSimpleInstancing.gltf&data=05%7C02%7Cbrutzman%40nps.edu%7Cb6c618bca87241de74ee08dc64622ca1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638495623330628463%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=sLnru8dmmX04wgvlUO3ybd7UorhKWEexYFz0FMdpSis%3D&reserved=0>
>>
>> then paste this .x3d file in the text area replacing what is there.
>> Future versions of this will have the old, much more pleasing motions as
>> pitch2, yaw2, roll2 animations.
>>
>> Thanks and Best,
>> Joe
>>
>>
>>
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240424/481cf455/attachment-0001.html>