[x3d-public] X3D Facial Animation examples and Facial Action Coding System (FACS)
John Carlson
yottzumm at gmail.com
Wed Oct 30 20:12:41 PDT 2024
I successfully reduced the number of ROUTEs with no apparent change to the
SingleMenuJin.x3d appearance and mechanics. The geometry is still iffy.
We need to discuss something beyond choosing the geometry in the first
file. Maybe the second? Lol. I’m not sure what to do except a Switch on
geometry or transform in each segment, and then add back the Scripts.
https://github.com/coderextreme/ci2had/blob/main/resources/SingleMenuJin.x3d
I can remove the ProximitySensor if you think it will help.
BS Contact doesn’t even seem to animate. I has the WorldInfo of the first
file in the log.
Meanwhile, don’t forget to test:
https://github.com/coderextreme/ci2had/blob/main/resources/YehudiMenuJin.x3d
Perhaps a next task is to add stopTime events there.
So perhaps starting fresh with a common mesh for each Segment (Site?)
would be appropriate? Or would initially disabling TimeSensors do the
trick?
Happy Trick-or-Treating!
John
On Wed, Oct 30, 2024 at 9:09 PM John Carlson <yottzumm at gmail.com> wrote:
> Joe, I ran all foure menus with FreeWRL without consequence, with and
> without the DUK JS Engine. Perhaps you could download FreeWRL 6.7? Note
> that the download might need installation of Microsoft
> .dll redistributeables. Michalis gives a good summary here: FreeWRL
> VRML/X3D browser / Tickets / #3 Missing DLLs (msvcp140.dll, likely also
> vcruntime140.dll) upon installing FreeWRL on fresh Windows 11 installation
> <https://sourceforge.net/p/freewrl/tickets/3/> Probably you won't have
> an issue on older versions of Windows? IDK!
>
> I did note that BS Contact reported identical ROUTE issues in
> SingleMenuJin.x3d, for quite some time. I need to look into it
>
> There are more duplicate lines in the file too, like:
>
> 29 < <FontStyle size="2.4" spacing="1.2"
> justify='"MIDDLE" "MIDDLE"'/>
> 29 < <Material diffuseColor="0 0 1"/>
> 29 < <Material diffuseColor="1 1 1"/>
> 29 < <Coordinate point="20 1.3833333333333333 -0.1, -20
> 1.3833333333333333 -0.1, -20 -1.616666666666667 -0.1, 20 -1.616666666666667
> -0.1"/>
> 29 < <IndexedFaceSet coordIndex='0 1 2 3 -1'>
>
> I'm not too worried about duplicate stuff in a menu. I will try to reduce
> the number of ROUTEs.
>
> John
>
> On Wed, Oct 30, 2024 at 7:14 PM Joe D Williams <joedwil at earthlink.net>
> wrote:
>
>> HI John,
>>
>> Use of uninitialized local var node in function touchPoint() in Script
>> MenuScript ()
>> undefined Couldn't enable JinNoseWrinkler_Clock
>> Selected 23 JinNoseWrinkler
>> Hit 18.9667 -33.0274 -0.1 23
>> Node Player has no eventOut currentScene in function touchPoint() in
>> Script MenuScript ()
>> script error: Can't get object for method getNamedNode in function
>> touchPoint() in Script MenuScript ()
>> Use of uninitialized local var node in function touchPoint() in Script
>> MenuScript ()
>> Use of uninitialized local var node in function touchPoint() in Script
>> MenuScript ()
>> undefined Couldn't enable JinNoseWrinkler_Clock
>> Selected 23 JinNoseWrinkler
>>
>>
>>
>> Above is what BS Contact said, Freewrl not run but list of messages with
>> isOnver, castle no like script.
>>
>> Thanks,
>>
>> Joe
>>
>>
>>
>> -----Original Message-----
>> From: John Carlson <yottzumm at gmail.com>
>> Sent: Oct 30, 2024 4:50 AM
>> To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> Cc: Myeong Won Lee <myeongwonlee at gmail.com>, puk at igraphics.com <
>> puk at igraphics.com>, Joseph D Williams <joedwil at earthlink.net>, William
>> Glascoe <eosocxo at comcast.net>, Carol McDonald <cemd2 at comcast.net>, Katy
>> Schildmeyer KS APPAREL DESIGN <katy at ksappareldesign.com>, Anita Havele <
>> anita.havele at web3d.org>, Nicholas Polys <npolys at vt.edu>, hanim at web3d.org
>> <hanim at web3d.org>, X3D Public Mailing List (x3d-public at web3d.org) <
>> x3d-public at web3d.org>
>> Subject: Re: X3D Facial Animation examples and Facial Action Coding
>> System (FACS)
>>
>>
>> Thanks for the new examples, Don and Joe, they make replacing
>> CoordinateInterpolators with HAnimDisplacers look great! It was very
>> frustrating to have examples where the animation didn’t seem to move and
>> the cycleInterval was too long.
>>
>> Here’s Yehudi (Menu) Jin as a single file (provide image from the
>> archive, please) Note that I’m still using a Switch on multiple humanoids,
>> but no Inlines.
>>
>>
>> https://raw.githubusercontent.com/coderextreme/ci2had/refs/heads/main/resources/YehudiMenuJin.x3d
>>
>> Joe will probably frown on the use of Scripts. I’ll get to TouchSensor
>> touchTime firing a TimeSensor startTime and stopping other TimeSensors
>> eventually. It’s just great to see lots of animation. This will help with
>> taking out the Switch.
>> Discussions about how to merge humanoids, welcome, like how do I manage
>> to turn on and off multiple displacers inside the same segment. I
>> currently have TimeSensor for a Facial Action sending an event to a
>> ScalarInterpolator which changes weights in many HAnimDisplacers. Perhaps
>> the key is to combine a segment's displacers into one humanoid. If that’s
>> all I have to do, great. I did try, that, with the geometry found in the
>> 0th file (JinBlink I think). It's a mess but maybe someone can repair it.
>> I think I need to disable all TimeSensors at the start.
>>
>> Note that there are segments in my code for teeth ears, tongue, glabella,
>> vermillion (?), etc. I guess these are really sites? Joe?
>>
>> I was also able to convert the Text MFString Menu to several Text nodes,
>> and it appears to be portable. It would be nice to get a HUD in there too,
>> eventually.
>>
>> Maybe it works in BS Contact now? X_ITE, Sunrize and FreeWRL seem to
>> work.
>>
>> John
>>
>> On Tue, Oct 29, 2024 at 5:17 AM Brutzman, Donald (Don) (CIV) <
>> brutzman at nps.edu> wrote:
>>
>>> At long last we have published and refined an important new set of
>>> examples, using 30 excellent original X3D models created by Dr. Myeong Won
>>> Lee and her team.
>>>
>>> - X3D Example Archives: Humanoid Animation, Facial Animation
>>> -
>>> Https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation
>>>
>>> - *Background.* Computer facial animation
>>> <https://en.wikipedia.org/wiki/Computer_facial_animation> is an area
>>> of current work for HAnim. Example models in this directory use the Facial
>>> Action Coding System (FACS)
>>> <https://en.wikipedia.org/wiki/Facial_Action_Coding_System> to
>>> provide Facial Animation examples. A variety of Action Unit (AU)
>>> <https://web.cs.wpi.edu/~matt/courses/cs563/talks/face_anim/ekman.html> enumeration
>>> values are defined for each facial expression in FACS. The FACS
>>> Visual Guidebook
>>> <https://imotions.com/blog/learning/research-fundamentals/facial-action-coding-system> showcases
>>> the expressive power of this approach.
>>> - *Initial examples.* The original example models
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals> show
>>> basic patterns for modifying coordinates of a custom mesh. The Suwon
>>> VRLAB X3D Converter
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/SuwonHAnimFacialMotionConverter.mp4> was
>>> used to compute CoordinateInterpolator animation values.
>>> The README: Original FACS Models
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html> page
>>> describes the original contributions and subsequent X3D model refinements.
>>> The HAnim Facial Animation Design Patterns
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/HAnimFacialAnimationDesignPatterns.png> diagram
>>> illustrates these modifications.
>>> - *Next steps.* The Future Work: FACS for X3D HAnim
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/FutureWork.html> page
>>> describes plans for continuing progress. Continuing refinement of animation
>>> models may lead to formally defining additional features and functionality
>>> in the Humanoid Animation (HAnim) International Standard
>>> <https://www.web3d.org/documents/specifications/19774/V2.0/index.html>
>>> .
>>>
>>> Details regarding model changes follow, along with plans for future work.
>>>
>>> Special thanks to Joe Williams for insights and assistance working
>>> through these issues. It feels like a solid basis for X3D HAnim Facial
>>> Animation is emerging.
>>>
>>> These deserve review by the HAnim working group, perhaps in a special
>>> meeting. Thanks in advance for all insights, improvements, and feedback.
>>>
>>> =======================================
>>>
>>> - *README: Original FACS Models*
>>> -
>>> https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html
>>>
>>> *Facial Action Coding System (FACS) original X3D models*
>>> Dr. Myeong Won Lee and team created these original models. She provided
>>> the following documentation 8 July 2024.
>>>
>>> The HAnim FACS animations are generated as follows:
>>>
>>> 1. A simple motion of facial expression is represented by a single
>>> action unit of FACS, such as inner brow raise or wink. More complex facial
>>> animation that is made up of a combination of motions, such as a smile made
>>> up of both eye and mouth motions, is defined by combining several action
>>> units.
>>> 2. To represent an action unit in FACS, a minimum of three sets of
>>> an LOE1 HAnim facial model (three X3D files) can be used. An LOE1 HAnim
>>> facial model is defined by 43 facial regions. Currently, up to five sets of
>>> HAnim faces can be used to generate a FACS action unit for a facial
>>> expression.
>>> 3. The sets of HAnim facial models for an action unit of FACS are
>>> animated using X3D keyframe interpolators. This animation can be generated
>>> by the HAnim facial converter.
>>>
>>> Example X3D animation files for FACS action units for facial animation
>>> are located at the first shared drive
>>> <https://drive.google.com/drive/folders/1Ja95KSOmoY54iqDdO7fZ3KR2Jo80WbFT?usp=sharing>
>>> link.
>>> The HAnim Facial Motion Converter used for the HAnim FACS animation can
>>> be downloaded at the second shared drive
>>> <https://drive.google.com/drive/folders/1v5CPn_EFig9jxLpfjAW34WKDQU6KdmCk?usp=drive_link>
>>> link.
>>> All the files submitted or notified to Web3D can be used freely without
>>> any restriction. It would be good if the facial examples can be uploaded to
>>> the Web3D resource pages. In addition, it is ok if all the submitted files
>>> that we developed are assigned Web3D software licenses.
>>>
>>> *X3D Model Changes*
>>> Continuing changes have been applied to content in this originals
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/> subdirectory
>>> to produced modified models in the parent FacialAnimation
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/> directory.
>>> List of changes follows.
>>>
>>> 1. File names adjusted to meet X3D Scene Authoring Hints: Naming
>>> Conventions
>>> <https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#NamingConventions>
>>> 2. Added DOCTYPE DTD statements for improved XML validation
>>> 3. X3D canonicalization (c14n) for consistent whitespace and
>>> attribute ordering
>>> 4. Model metadata of various kinds, following best practices
>>> described in X3D Scene Authoring Hints: meta Statements
>>> <https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#metaStatements>
>>> 5. Added Web3D Consortium open-source license
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/license.html>
>>> 6. Fixed DEF/USE of ImageTexture files to avoid repetitive,
>>> time-consuming reloading of the renamed file from Jin.jpg
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/Jin.jpg> to
>>> JinImageAtlas.jpg
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/JinImageAtlas.jpg>
>>> 7. Fixed DEF of Coordinate nodes to end in _COORD vice -COORD for
>>> best portability and to match naming conventions
>>> 8. Fixed DEF/USE of Material files to avoid repetitive reloading and
>>> then Appearance files
>>> 9. Renamed all files to put "Jin" first, which will help
>>> alphabetization when we add more FACS examples in the future
>>> 10. Applied correct name to meta title, meta identifier, and
>>> WorldInfo title attributes
>>> 11. Added meta reference links to original version of each file
>>> 12. Combined duplicative clocks and ScalarInterpolator adapters,
>>> omit unnecessary Adapter ROUTEs
>>> 13. Added directory package descriptions
>>> 14. Under evaluation: changing animation loop period from
>>> cycleInteval="4" to cycleInteval="0.99" in order to make model-animation
>>> visualizations more obvious.
>>> 15. TODO adjust select models to restore prior more-complex
>>> ScalarInterpolator clock reversals
>>> 16. TODO rescale units to meters
>>>
>>> Many model refinements have been applied. For example, see the
>>> subversion diff log at revision 36730
>>> <https://sourceforge.net/p/x3d/code/36730>. These changes all preserve
>>> and enhance the orginal features of this model set.
>>> =======================================
>>>
>>> - *Future Work: Facial Action Coding System (FACS) for X3D HAnim
>>> models*
>>> -
>>> https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/FutureWork.html
>>>
>>> *Future Work: Facial Action Coding System (FACS) for X3D HAnim models*
>>> These plans are based on work adapting models by Dr. Myeong Won Lee, as
>>> described in the README: Original FACS Models
>>> <https://www.web3d.org/x3d/content/examples/HumanoidAnimation/FacialAnimation/originals/README.html>
>>> .
>>> *Lessons Learned, So Far*
>>>
>>> 1. Every head will likely have different geometry, textures, colors,
>>> etc. There is no need for every head to be the same.
>>> 2. A single head model file needs to align with HAnim joints and
>>> segments, if it is going to be interoperable with other HAnim skeleton/skin
>>> models.
>>> 3. Each head model will connect to the skeleton as shown in
>>> Part 1: Humanoid animation (HAnim) architecture, Section 4
>>> Concepts, 4.9.6.3 LOA-3 hierarchy
>>> <https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#Hierarchy3>
>>> humanoid_root : sacrum [...] vc7 : c7 | vc6 :
>>> c6 | vc5 : c5 | vc4 : c4 | vc3
>>> : c3 | vc2 : c2 | vc1 : c1
>>> | *skullbase : skull* |
>>> l_eyelid_joint : l_eyelid | r_eyelid_joint :
>>> r_eyelid | l_eyeball_joint : l_eyeball
>>> | r_eyeball_joint : r_eyeball |
>>> l_eyebrow_joint : l_eyebrow | r_eyebrow_joint :
>>> r_eyebrow | temporomandibular : jaw
>>> 4. Typically a new head will connect at *HAnimJoint skullbase* and *HAnimSegment
>>> skull*, possibly including neck joints as well.
>>> 5. Typically an interoperable HAnimHumanoid head will include
>>> HAnimJoint and HAnimSegment definitions for eyelid, eyeball, eyebrow, and
>>> jaw.
>>> 6. The HAnim specification includes guidance on connecting partial
>>> body models together in 4.8.6 Partial HAnim figures
>>> <https://www.web3d.org/documents/specifications/19774/V2.0/Architecture/concepts.html#PartialHAnimFigures>
>>> .
>>> 7. No HAnimSite feature points are required in the HAnim
>>> specification.
>>> 8. We can add HAnimSite feature points to the above HAnimSegment
>>> notes, without interfering with any of the native HAnim motion animations.
>>> However adding HAnimSite locations to the other facial geometry many need
>>> further attention. Adding a useful set of HAnimSite feature points for
>>> facial animation might be very useful.
>>>
>>> *Next Steps*
>>>
>>> 1. We want head models that include the ability to use an FACS
>>> Action Unit (AU) enumeration that shows a new expression.
>>> 2. We currently have single expressions demonstrated in separate
>>> model files. Showing one expression at a time is the initial goal for
>>> facial expression animations.
>>> 3. Integrating the animations together efficiently, within a single
>>> X3D model file, is the path towards having interoperably animatable heads
>>> that can move eyes/jaw and also show expressions.
>>> 4. We first want single-AU responses working, and then multiple-AU
>>> responses working, in order to achieve the full range of FACS expressions
>>> that are possible.
>>> 5. For full animation compatibility, head models need to include the
>>> standard HAnim joints and segments for eyelids, eyeballs, eyebrows, and
>>> jaw. This requirement deserves careful consideration when designing mesh
>>> geometry.
>>> 6. See the FACS Visual Guidebook
>>> <https://imotions.com/blog/learning/research-fundamentals/facial-action-coding-system/> for
>>> video examples with corresponding descriptions that illustrate the
>>> expressions conveyed by each Action Unit (AU), either individually or in
>>> combination.
>>>
>>> Building example models is the path towards showing repeatable
>>> capabilities. Conceivably these capabilities might become a new HAnim node
>>> for HAnimHumanoid. Successful, repeatable design patterns can then be
>>> formalized in the future HAnim Specification.
>>> =======================================
>>>
>>> Summary: interoperable facial animation behaviors are beginning to look
>>> feasible and repeatable. More demonstration work awaits.
>>>
>>> Have fun with X3D Humanoid Animation (HAnim)! 🙂
>>>
>>>
>>> all the best, Don
>>>
>>> --
>>>
>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>>
>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>>
>>> X3D graphics, virtual worlds, navy robotics
>>> https://faculty.nps.edu/brutzman
>>>
>>>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20241030/444ba024/attachment-0001.html>
More information about the x3d-public
mailing list