[x3d-public] X3D Facial Animation examples and Facial Action Coding System (FACS)

Wed Oct 30 17:35:36 PDT 2024

FreeWRL:  try the DUK JavaScript engine.

I’ll look into BS Contact.  I’m using fields.

I know CGE model viewer doesn’t work.

Sunrize/X_ITE may be your best besides FreeWRL.

I’ll test the current version of Yehudi in a bit.

It looks like you *might* have an old version based in getNamedNode().
There’s 30 Scripts in the new one

On Wed, Oct 30, 2024 at 7:14 PM Joe D Williams <joedwil at earthlink.net>
wrote:

> HI John,
>
> Use of uninitialized local var node in function touchPoint() in Script
> MenuScript ()
> undefined Couldn't enable JinNoseWrinkler_Clock
> Selected 23 JinNoseWrinkler
> Hit 18.9667 -33.0274 -0.1 23
> Node Player has no eventOut currentScene in function touchPoint() in
> Script MenuScript ()
> script error: Can't get object for method getNamedNode in function
> touchPoint() in Script MenuScript ()
> Use of uninitialized local var node in function touchPoint() in Script
> MenuScript ()
> Use of uninitialized local var node in function touchPoint() in Script
> MenuScript ()
> undefined Couldn't enable JinNoseWrinkler_Clock
> Selected 23 JinNoseWrinkler
>
>
>
> Above is what BS Contact said, Freewrl not run but list of messages with
> isOnver, castle no like script.
>
> Thanks,
>
> Joe
>
>
>
> -----Original Message-----
> From: John Carlson <yottzumm at gmail.com>
> Sent: Oct 30, 2024 4:50 AM
> To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> Cc: Myeong Won Lee <myeongwonlee at gmail.com>, puk at igraphics.com <
> puk at igraphics.com>, Joseph D Williams <joedwil at earthlink.net>, William
> Glascoe <eosocxo at comcast.net>, Carol McDonald <cemd2 at comcast.net>, Katy
> Schildmeyer KS APPAREL DESIGN <katy at ksappareldesign.com>, Anita Havele <
> anita.havele at web3d.org>, Nicholas Polys <npolys at vt.edu>, hanim at web3d.org <
> hanim at web3d.org>, X3D Public Mailing List (x3d-public at web3d.org) <
> x3d-public at web3d.org>
> Subject: Re: X3D Facial Animation examples and Facial Action Coding System
> (FACS)
>
>
> Thanks for the new examples, Don and Joe, they make replacing
> CoordinateInterpolators with HAnimDisplacers look great!  It was very
> frustrating to have examples where the animation didn’t seem to move and
> the cycleInterval was too long.
>
> Here’s Yehudi (Menu) Jin as a single file (provide image from the archive,
> please) Note that I’m still using a Switch on multiple humanoids, but no
> Inlines.
>
>
> https://raw.githubusercontent.com/coderextreme/ci2had/refs/heads/main/resources/YehudiMenuJin.x3d
>
> Joe will probably frown on the use of Scripts.  I’ll get to TouchSensor
> touchTime firing a TimeSensor startTime and stopping other  TimeSensors
> eventually.  It’s just great to see lots of animation.  This will help with
> taking out the Switch.
> Discussions about how to merge humanoids, welcome, like how do I manage to
> turn on and off multiple displacers inside the same segment.  I currently
> have TimeSensor for a Facial Action  sending an event to a
> ScalarInterpolator which changes weights in many HAnimDisplacers.  Perhaps
> the key is to combine a segment's displacers into one humanoid.  If that’s
> all I have to do, great.  I  did try, that, with the geometry found in the
> 0th file (JinBlink I think).  It's a mess but maybe someone can repair it.
> I think I need to disable all TimeSensors at the start.
>
> Note that there are segments in my code for teeth ears, tongue, glabella,
> vermillion (?), etc.  I guess these are really sites?  Joe?
>
> I was also able to convert the Text MFString Menu to several Text nodes,
> and it appears to be portable.  It would be nice to get a HUD in there too,
> eventually.
>
> Maybe it works in BS Contact now?  X_ITE, Sunrize and FreeWRL seem to work.
>
> John
>
> On Tue, Oct 29, 2024 at 5:17 AM Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
>> At long last we have published and refined an important new set of
>> examples, using 30 excellent original X3D models created by Dr. Myeong Won
>> Lee and her team.
>>
>>    - X3D Example Archives: Humanoid Animation, Facial Animation
>>    -
>>    Https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation
>>
>>    - *Background.* Computer facial animation
>>    <https://en.wikipedia.org/wiki/Computer_facial_animation> is an area
>>    of current work for HAnim. Example models in this directory use the Facial
>>    Action Coding System (FACS)
>>    <https://en.wikipedia.org/wiki/Facial_Action_Coding_System> to
>>    provide Facial Animation examples. A variety of Action Unit (AU)
>>    <https://web.cs.wpi.edu/~matt/courses/cs563/talks/face_anim/ekman.html> enumeration
>>    values are defined for each facial expression in FACS. The FACS
>>    Visual Guidebook
>>    <https://imotions.com/blog/learning/research-fundamentals/facial-action-coding-system> showcases
>>    the expressive power of this approach.
>>    - *Initial examples.* The original example models
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals> show
>>    basic patterns for modifying coordinates of a custom mesh. The Suwon
>>    VRLAB X3D Converter
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/SuwonHAnimFacialMotionConverter.mp4> was
>>    used to compute CoordinateInterpolator animation values.
>>    The README: Original FACS Models
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html> page
>>    describes the original contributions and subsequent X3D model refinements.
>>    The HAnim Facial Animation Design Patterns
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/HAnimFacialAnimationDesignPatterns.png> diagram
>>    illustrates these modifications.
>>    - *Next steps.* The Future Work: FACS for X3D HAnim
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/FutureWork.html> page
>>    describes plans for continuing progress. Continuing refinement of animation
>>    models may lead to formally defining additional features and functionality
>>    in the Humanoid Animation (HAnim) International Standard
>>    <https://www.web3d.org/documents/specifications/19774/V2.0/index.html>
>>    .
>>
>> Details regarding model changes follow, along with plans for future work.
>>
>> Special thanks to Joe Williams for insights and assistance working
>> through these issues.  It feels like a solid basis for X3D HAnim Facial
>> Animation is emerging.
>>
>> These deserve review by the HAnim working group, perhaps in a special
>> meeting.  Thanks in advance for all insights, improvements, and feedback.
>>
>> =======================================
>>
>>    - *README: Original FACS Models*
>>    -
>>    https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html
>>
>> *Facial Action Coding System (FACS) original X3D models*
>> Dr. Myeong Won Lee and team created these original models. She provided
>> the following documentation 8 July 2024.
>>
>> The HAnim FACS animations are generated as follows:
>>
>>    1. A simple motion of facial expression is represented by a single
>>    action unit of FACS, such as inner brow raise or wink. More complex facial
>>    animation that is made up of a combination of motions, such as a smile made
>>    up of both eye and mouth motions, is defined by combining several action
>>    units.
>>    2. To represent an action unit in FACS, a minimum of three sets of an
>>    LOE1 HAnim facial model (three X3D files) can be used. An LOE1 HAnim facial
>>    model is defined by 43 facial regions. Currently, up to five sets of HAnim
>>    faces can be used to generate a FACS action unit for a facial expression.
>>    3. The sets of HAnim facial models for an action unit of FACS are
>>    animated using X3D keyframe interpolators. This animation can be generated
>>    by the HAnim facial converter.
>>
>> Example X3D animation files for FACS action units for facial animation
>> are located at the first shared drive
>> <https://drive.google.com/drive/folders/1Ja95KSOmoY54iqDdO7fZ3KR2Jo80WbFT?usp=sharing>
>>  link.
>> The HAnim Facial Motion Converter used for the HAnim FACS animation can
>> be downloaded at the second shared drive
>> <https://drive.google.com/drive/folders/1v5CPn_EFig9jxLpfjAW34WKDQU6KdmCk?usp=drive_link>
>>  link.
>> All the files submitted or notified to Web3D can be used freely without
>> any restriction. It would be good if the facial examples can be uploaded to
>> the Web3D resource pages. In addition, it is ok if all the submitted files
>> that we developed are assigned Web3D software licenses.
>>
>> *X3D Model Changes*
>> Continuing changes have been applied to content in this originals
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/> subdirectory
>> to produced modified models in the parent FacialAnimation
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/> directory.
>> List of changes follows.
>>
>>    1. File names adjusted to meet X3D Scene Authoring Hints: Naming
>>    Conventions
>>    <https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#NamingConventions>
>>    2. Added DOCTYPE DTD statements for improved XML validation
>>    3. X3D canonicalization (c14n) for consistent whitespace and
>>    attribute ordering
>>    4. Model metadata of various kinds, following best practices
>>    described in X3D Scene Authoring Hints: meta Statements
>>    <https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#metaStatements>
>>    5. Added Web3D Consortium open-source license
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/license.html>
>>    6. Fixed DEF/USE of ImageTexture files to avoid repetitive,
>>    time-consuming reloading of the renamed file from Jin.jpg
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/Jin.jpg> to
>>    JinImageAtlas.jpg
>>    <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/JinImageAtlas.jpg>
>>    7. Fixed DEF of Coordinate nodes to end in _COORD vice -COORD for
>>    best portability and to match naming conventions
>>    8. Fixed DEF/USE of Material files to avoid repetitive reloading and
>>    then Appearance files
>>    9. Renamed all files to put "Jin" first, which will help
>>    alphabetization when we add more FACS examples in the future
>>    10. Applied correct name to meta title, meta identifier, and
>>    WorldInfo title attributes
>>    11. Added meta reference links to original version of each file
>>    12. Combined duplicative clocks and ScalarInterpolator adapters, omit
>>    unnecessary Adapter ROUTEs
>>    13. Added directory package descriptions
>>    14. Under evaluation: changing animation loop period from
>>    cycleInteval="4" to cycleInteval="0.99" in order to make model-animation
>>    visualizations more obvious.
>>    15. TODO adjust select models to restore prior more-complex
>>    ScalarInterpolator clock reversals
>>    16. TODO rescale units to meters
>>
>> Many model refinements have been applied. For example, see the subversion
>> diff log at revision 36730 <https://sourceforge.net/p/x3d/code/36730>.
>> These changes all preserve and enhance the orginal features of this model
>> set.
>> =======================================
>>
>>    - *Future Work: Facial Action Coding System (FACS) for X3D HAnim
>>    models*
>>    -
>>    https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/FutureWork.html
>>
>> *Future Work: Facial Action Coding System (FACS) for X3D HAnim models*
>> These plans are based on work adapting models by Dr. Myeong Won Lee, as
>> described in the README: Original FACS Models
>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html>
>> .
>> *Lessons Learned, So Far*
>>
>>    1. Every head will likely have different geometry, textures, colors,
>>    etc. There is no need for every head to be the same.
>>    2. A single head model file needs to align with HAnim joints and
>>    segments, if it is going to be interoperable with other HAnim skeleton/skin
>>    models.
>>    3. Each head model will connect to the skeleton as shown in
>>       Part 1:  Humanoid animation (HAnim) architecture, Section 4
>>    Concepts, 4.9.6.3 LOA-3 hierarchy
>>    <https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#Hierarchy3>
>>     humanoid_root : sacrum        [...]            vc7 : c7            | vc6 :
>>    c6            |   vc5 : c5            |     vc4 : c4            |       vc3
>>    : c3            |         vc2 : c2            |           vc1 : c1
>>       |             *skullbase : skull*            |
>>    l_eyelid_joint : l_eyelid            |               r_eyelid_joint :
>>    r_eyelid            |               l_eyeball_joint : l_eyeball
>>     |               r_eyeball_joint : r_eyeball            |
>>    l_eyebrow_joint : l_eyebrow            |               r_eyebrow_joint :
>>    r_eyebrow            |               temporomandibular : jaw
>>    4. Typically a new head will connect at *HAnimJoint skullbase* and *HAnimSegment
>>    skull*, possibly including neck joints as well.
>>    5. Typically an interoperable HAnimHumanoid head will include
>>    HAnimJoint and HAnimSegment definitions for eyelid, eyeball, eyebrow, and
>>    jaw.
>>    6. The HAnim specification includes guidance on connecting partial
>>    body models together in 4.8.6 Partial HAnim figures
>>    <https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#PartialHAnimFigures>
>>    .
>>    7. No HAnimSite feature points are required in the HAnim
>>    specification.
>>    8. We can add HAnimSite feature points to the above HAnimSegment
>>    notes, without interfering with any of the native HAnim motion animations.
>>    However adding HAnimSite locations to the other facial geometry many need
>>    further attention. Adding a useful set of HAnimSite feature points for
>>    facial animation might be very useful.
>>
>> *Next Steps*
>>
>>    1. We want head models that include the ability to use an FACS Action
>>    Unit (AU) enumeration that shows a new expression.
>>    2. We currently have single expressions demonstrated in separate
>>    model files. Showing one expression at a time is the initial goal for
>>    facial expression animations.
>>    3. Integrating the animations together efficiently, within a single
>>    X3D model file, is the path towards having interoperably animatable heads
>>    that can move eyes/jaw and also show expressions.
>>    4. We first want single-AU responses working, and then multiple-AU
>>    responses working, in order to achieve the full range of FACS expressions
>>    that are possible.
>>    5. For full animation compatibility, head models need to include the
>>    standard HAnim joints and segments for eyelids, eyeballs, eyebrows, and
>>    jaw. This requirement deserves careful consideration when designing mesh
>>    geometry.
>>    6. See the FACS Visual Guidebook
>>    <https://imotions.com/blog/learning/research-fundamentals/facial-action-coding-system/> for
>>    video examples with corresponding descriptions that illustrate the
>>    expressions conveyed by each Action Unit (AU), either individually or in
>>    combination.
>>
>> Building example models is the path towards showing repeatable
>> capabilities. Conceivably these capabilities might become a new HAnim node
>> for HAnimHumanoid. Successful, repeatable design patterns can then be
>> formalized in the future HAnim Specification.
>> =======================================
>>
>> Summary: interoperable facial animation behaviors are beginning to look
>> feasible and repeatable.   More demonstration work awaits.
>>
>> Have fun with X3D Humanoid Animation (HAnim)!   🙂
>>
>>
>> all the best, Don
>>
>> --
>>
>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>>
>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>>
>> X3D graphics, virtual worlds, navy robotics
>> https://faculty.nps.edu/brutzman
>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20241030/b3445a8e/attachment-0001.html>