[x3d-public] X3D4 HAnim2 support for skeleton, skin, sites and (potentially) apparel

Mon Jan 2 12:40:35 PST 2023

Hoping everyone is enjoying a Happy New Year.

Joe Williams and I have been "woodshedding" (not bikeshedding) for the past
few weeks, and finished our 2022 activity with a flourish.  We have been
able to successfully demonstrate X3D4 HAnim2 support for skeleton, skin,
sites, and (potentially) apparel.  Details follow.

Summary:  nearly everything we modeled now works, repeatedly and repeatably.
This opens several doors to further opportunities for Web3D Consortium
working groups on Humanoid Animation (HAnim) and Medical applications.
Though background details are numerous, progress is always demonstrated and
proven through individual X3D modes.

Our block of work was motivated by the stellar workshop at the Web3D 2022
Conference in Paris France.

*	Web3D 27th International Conference on 3D Web Technology
*	https://web3d.siggraph.org/complete-program

*	"Made-to-Measure Unit-1 on-Demand Manufacturing of Physical and
Digital Apparel" Workshop
*	Monika Januszkiewicz, Carol McDonald and Sydney Otten (in person,
video, slides)
*	The aim of the workshop is to develop a conversation between
providers of body and material data and the manufacturing consumers of data.
The presentation offers a framework for classifying manufacturing stages as
independent components connected to the processes and technologies based on
the required input and output. To reach this goal, pipeline is designed for
the customization of two unisex products: a shirt and a pair of jeans.
*
https://web3d.siggraph.org/wp-content/uploads/2022/11/phi_phiber_workshop_pr
esentation_1080p_v7_compressed.pdf
*
https://www.youtube.com/watch?v=E4tFqHchDzc&list=PLibbKqU7MceM4IfepyHkluX4GJ
HBvGXA4&index=2&t=0s

We have articulated a strategy for X3D4 HAnim2 technical capabilities for
use in Apparel and Medical applications.  We had expected this to be a plan
for future work, but were happily surprised to find that current X3D browser
capabilities are satisfactory to pursue such work now.  The following
strategic points are maintained up-to-date online.

         Design patterns for HAnimHumanoid presenting skin and apparel

                         using HAnim2 standard in X3D4

   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-

1. *Simple Skeleton*. skeleton field, simple skeleton structure, invisible
or

    visualized by rendered ball-and-stick for joints, segments, sites.

2. *Simple Skeleton Mesh*. skeleton field, Shape geometry attached to
segments,

   moving with skeleton but rendering is crude since no deformation occurs

   during animation if overlapping.

3. *Skin, Indexed Geometry only*. skin field, indexed Coordinate mesh with

   skinCoordIndex and weight connections at each of joint nodes for browser

   performance of smooth deformation when moving.

4. *Skin, Shape holding indexed geometry*. Allow skin field to hold Shape

   containing Appearance with Material or texture, similarly holding indexed

   mesh with skinCoordIndex connections at joints for smooth deformation 

   when moving. (Specification revision pending, Mantis 1408).

5. *Group, Transform, Switch, or LOD*. Similarly allow skin field to hold 

   Group or Transform-scaled collection of geometry, Switch, or LOD, then 

   one or more Shape nodes holding the combined indexed mesh.  Once again, 

   HAnim specification requirements holding identical indexed mesh with 

   skinCoord, skinCoordIndex connections for joints allows smooth
deformation.

   Thus geometric mesh morphology is unchanged, only Color Appearance
Material

   and Texture can vary among the different skin shapes.

6. *Synthesis*. Extra-large Coordinate mesh using duplicate points for 

   inner-volume (flesh mesh) and outer-volume (apparel mesh) geometry.  

   Likely possible with existing X3D4 specification but requires complex

   preprocessing of humanoid geometry.  Applying cloth physics likely not

   practical or possible.

7. *Apparel*. Future mainstreaming: add apparel field to future
specification

   versions (namely HAnim2.1 and X3D4.1), allowing separate definition of 

   clothing that is independent of skin mesh.  Likely want an MFNode array 

   (rather than Group or Switch) to directly define multiple clothing
layers.

   Likely need to consider special cases for cloth physics.

8. *Anatomy*.  Similar to apparel field: multiple epithelial tissue layers

   for surfaces of human organs are common, often with similar construction

   about concentric center-line spine through deformable organ.  This
suggests

   that an adapted Extrusion node may be more relevant than meshes.  Likely 

   will also need to consider special cases for organ physics.  Note that
this

   is similar (possibly identical) rationale to preceding analysis of
apparel.

   - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-

Document online at

*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/DesignPatt
ernsApparelVariations.txt

Design-pattern template X3D model illustrating these patterns at

*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/DesignPatt
ernsApparelMedicalSkinLayersIndex.html

X3D4 Draft International Specification (DIS) refinements maintained at

* Mantis 1410: HAnimHumanoid skin definitions need to assign textures via
Shape, Group, Switch etc.

* https://www.web3d.org/member-only/mantis/view.php?id=1410

Repeatability.  Our primary humanoid models and test tools for conducting
this work:

*	HumanoidAnimation (HAnim) X3D Examples Archive
*	https://www.web3d.org/x3d/content/examples/HumanoidAnimation

*	X3D Example Archives: Humanoid Animation, Skin, Joe Skeleton Skin
Site Salute Walk, author Joe Williams
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalkIndex.html
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk.html
*	Notable usefulness: the following "document tree" clearly
illustrating actual model structure.
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk.html#HumanHAnimHumanoidReport

*	Castle Game Engine, view3dscene by Michalis Kamburelis
*	https://castle-engine.io/view3dscene.php
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_view3dscene.png
*	Castle Game Engine, Convert Everything to X3D 
*	(glTF, X3D, VRML, Wavefront OBJ, STL, Collada, 3DS, MD3, Spine JSON
and others)
*	https://castle-engine.io/convert.php

*	X3D-Edit 4.0 Authoring Tool for Extensible 3D (X3D) Graphics,
especially for X3D Quality Assurance (QA)
*	https://savage.nps.edu/X3D-Edit
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_X3D-Edit.png
*
https://www.web3d.org/x3d/content/examples/X3dResources.html#QualityAssuranc
e

Snapshot results.  Links for X3D browser demonstration images:

*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_composite.2023JAN2.png
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_X_ITE.png (works)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_X3DOM.png (works)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_H3DViewer.png (works)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_freeWrl.png (works)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_Octaga.png (partial)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_vivaty.png (works)
*
https://www.web3d.org/x3d/content/examples/HumanoidAnimation/Skin/JoeSkeleto
nSkinSiteSaluteWalk_view3dscene.png (works)

Additional X3D browser screenshot images will be welcome.  Extra surprise:
to our knowledge, vivaty player has been defunct for over a decade and yet
it worked anyway (no doubt thanks to original HAnim contributor Keith
Victor)!

These results certainly convey a lot about the power of steady forward
evolution using open standards.  Individuals and businesses can implement
models and innovate applications with confidence that their investments of
time and labor are both stable and extendible.  Protection from "industry
churn" that is common when dependent on a single software company is
important.

Since human bodies contain immense amounts of informational detail, we'd
like to point out that the necessary X3D HAnim model development would
simply not be possible without the suite of QA and validation tools that
continue to get refined during this work.  Each has different expressive
power for diagnostics and correction, the combination of each is powerful.
The majority of tools are integrated in a servlet and available online.

*	X3D Validator 
*	X3D Validator performs comprehensive Quality Assurance (QA) tests to
ensure the validity of X3D3, X3D4, and HAnim graphics models.
*	https://savage.nps.edu/X3dValidator/validate

Currently quite active, now with over 2,00 member organizations: the
Metaverse Standards Forum (MSF). We can further expect to see consolidated
use-case requirements emerge from numerous companies and individuals working
in MSF working groups.  Their proposed outputs will likely save a lot of
time for our Web3D Working Groups focused on for Humanoid Animation and
Medical Applications working groups.    

*	Metaverse Standards Forum (MSF)
*	https://metaverse-standards.org
*	"Where Leading Standards Organizations and Companies Cooperate to
Foster Interoperability Standards for an Open Metaverse."
*	"The Forum will not create standards itself but will coordinate
requirements and resources to foster the creation and evolution of standards
within standards organizations working in relevant domains."

*	Metaverse Standards Forum Establishes Pipeline of Exploratory and
Working Groups
*
https://metaverse-standards.org/news/blog/metaverse-standards-forum-announce
s-first-approved-domain-groups
*	Digital Fashion Wearables for Avatars  (approved Exploratory Group).
Generate insights and interoperability on Digital Fashion including clothing
(with a layering system), shoes, hats, and accessories. The group is
surveying the current landscape of existing file formats for storing avatars
and related assets, generating use cases and requirements for avatar
customisation, and defining logic for a wearables system including dress up,
dress down, and replace options. The group is also considering developing an
open source avatar validation tool for reliable import and export from
popular authoring tools such as Blender.
*	Interoperable Characters/Avatars (proposal being drafted).  Identify
and prioritize use cases for character/avatar interoperability across
multiple industries such as film, gaming, fashion and social, and platforms
such as web, mobile, console, and XR. Will generate recommendations for a
standardized character/avatar file format that can be dynamically loaded in
multiple run times while maintaining consistent appearance, behaviors and
animations.

Web3D Consortium holds a Class A Liaison with International Standards
Organization (ISO) and is a charter Standards Development Organization (SDO)
in MSF as well.  Web3D Consortium membership is valuable, for organizations
and individuals.

*	Web3D Liaisons and Partnerships
*	https://www.web3d.org/about/liaisons
*	Join the Web3D Consortium
*	https://www.web3d.org/join

Next Steps.  So: many avenues of work are possible in 2023.  Most steps can
be based on practice, not just theory, utilizing an effective ecosystem of
X3D4 tools and models.

a.	Motion.  Continued testing of initial successes with BVH mocap model
conversion for X3D animation using HAnimMotion node
b.	Apparel.  Cooperative development with IEEE Industry Connections and
Standards Group for 3D Body Processing (3DBP).  Our current Web3D Consortium
liaison is William Glascoe, who is also HAnim co-chair with Dr. Myeon Won
Lee.  https://standards.ieee.org/industry-connections/3d/bodyprocessing
c.	Medical.  Review and consider formalizing earlier model designs for
human internal organs by Web3D Korea Chapter principals, hopefully as part
of a concerted strategy to support human medical records.
d.	Metadata.  Keeping track of details is important!  Essential work by
Web3D Design Printing and Scanning (CAD) working group already provides the
technical basis for broader approaches with X3D4 HAnim2.

Joe and I gratefully acknowledge and thank the numerous people whose efforts
allowed us to reach this successful milestone.

Have fun with X3D4 and HAnim2!   8)

all the best, Don

-- 

Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu

Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149

X3D graphics, virtual worlds, Navy robotics https://
faculty.nps.edu/brutzman

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230102/c10d4775/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: JoeSkeletonSkinSiteSaluteWalk_composite.2023JAN2.png
Type: image/png
Size: 428941 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230102/c10d4775/attachment-0001.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5353 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230102/c10d4775/attachment-0001.p7s>