<div dir="auto">Len,</div><div dir="auto"><br></div><div dir="auto">This seems like an important capability that could point to more generic handling of displacers, especially outside of HAnim.</div><div dir="auto"><br></div><div dir="auto">My requirements were “no scripting/SAI,” so of course, I devised a different solution for facial animations. A no-scripting solution is highly desirable due to different SAI solutions.</div><div dir="auto"><br></div><div dir="auto">This was quite the technical feat, Len, and is appreciated. Perhaps a presentation is in order. Please let me know if you do one.</div><div dir="auto"><br></div><div dir="auto">One might consider a Proto with a script instead of coming up with new XML schema.</div><div dir="auto"><br></div><div dir="auto">In the coming days/weeks/months, I will attempt to combine displacer animation and joint animation in the same scene, and I would appreciate some of your test files, so I can build a minimal example without scripting.</div><div dir="auto"><br></div><div dir="auto">I appreciate your efforts, but we really would like to get displacers and joint rotations working together, especially for exports from Maya and Blender. This may mean we modify X_ITE.</div><div dir="auto"><br></div><div dir="auto">John </div><div><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Len Bullard via x3d-public</strong> <span dir="auto"><<a href="mailto:x3d-public@web3d.org">x3d-public@web3d.org</a>></span><br>Date: Sat, May 30, 2026 at 12:40 PM<br>Subject: [x3d-public] MCCF Facial Expressions for H-Anim Editor<br>To: <<a href="mailto:x3d-public@web3d.org">x3d-public@web3d.org</a>><br>CC: <<a href="mailto:cbullard@hiwaay.net">cbullard@hiwaay.net</a>><br></div><br><br>To the W3D Consortium Technical Community,<br>
<br>
We wanted to share a brief technical note on the face deformation system <br>
we<br>
have implemented in the MCCF (Multi-Channel Coherence Field) character<br>
pipeline, specifically regarding our approach to HAnim facial action <br>
unit<br>
animation in X_ITE.<br>
<br>
<br>
BACKGROUND<br>
<br>
MCCF uses X_ITE (version 10.5.2) as its real-time X3D renderer. Our <br>
character<br>
Cindy is a full LOA-4 HAnim figure with Walk, Run, Jump, Kick, Pitch, <br>
Yaw, and<br>
Roll animation clips, driven by TimeSensor and OrientationInterpolator <br>
ROUTEs.<br>
The face system needed to drive 30 FACS action units (AUs) in real time <br>
from an<br>
affective engine that maps emotional state to AU weight vectors.<br>
<br>
<br>
HANIMDISPLACER: WHAT WE FOUND<br>
<br>
Our initial architecture used HAnimDisplacer nodes in the standard way <br>
-- one<br>
displacer per AU per mesh segment, with weight driven via <br>
AnimationAdapter and<br>
TimeSensor ROUTEs. We spent considerable time validating this approach <br>
using a<br>
minimal reproducible X3D file and a SAI test harness, and identified the<br>
following:<br>
<br>
- profile='Full' causes X_ITE parse failure. profile='Interchange' is<br>
required.<br>
<br>
- loa='1' renders geometry invisible. loa='4' is required.<br>
<br>
- The Interpolation component must be explicitly declared or <br>
TimeSensor<br>
ROUTEs do not fire.<br>
<br>
- HAnimDisplacer nodes parse correctly and accept SAI weight writes.<br>
Readback confirms the value is stored.<br>
<br>
- However, mesh deformation does not occur. The weight is accepted but<br>
does not propagate to geometry.<br>
<br>
This behaviour is consistent across all test configurations in the X_ITE <br>
10.5.2<br>
CDN build. Joint rotation via OrientationInterpolator ROUTEs works <br>
correctly.<br>
HAnimDisplacer deformation does not.<br>
<br>
SAI MORPH DRIVER: OUR SOLUTION<br>
<br>
We implemented a JavaScript SAI<br>
morph driver that operates directly on Coordinate.point via the X_ITE <br>
Scene<br>
Authoring Interface.<br>
<br>
The approach:<br>
<br>
1. At scene load, the driver reads each face Coordinate node by DEF <br>
name and<br>
caches the rest-pose vertex positions as Float32Arrays.<br>
<br>
2. AU displacement data is loaded from an external XML file<br>
(cindy_expressions.xml) in our MCCFExpressions format.<br>
<br>
3. When an AU weight changes, the driver rebuilds the full blended <br>
pose:<br>
starting from rest pose, it additively accumulates all active AU<br>
displacements weighted by their current values, then writes the <br>
result<br>
back to Coordinate.point as a new MFVec3f.<br>
<br>
We confirmed through testing that:<br>
<br>
- _scene.getNamedNode('DEF') resolves Coordinate nodes correctly.<br>
- Direct property assignment (node.point = new X3D.MFVec3f(...)) <br>
updates<br>
the rendered mesh.<br>
- Multiple AUs blend additively without interference.<br>
- The system runs at interactive frame rates for Cindy's mesh: 356 <br>
skull<br>
vertices plus eyebrow, eyelid, eyeball, and jaw segments, covering <br>
all<br>
30 FACS AUs simultaneously.<br>
<br>
<br>
THE MCCF EXPRESSIONS XML FORMAT<br>
<br>
Displacement data lives in a single XML file per character. We chose XML <br>
over<br>
JSON deliberately: our entire pipeline is XML-based (cultivar <br>
definitions,<br>
scene configurations, behavior clips), enabling XSLT transforms and <br>
standard<br>
XML tooling across the system.<br>
<br>
The format has two sections. AU geometry blocks define per-vertex <br>
displacements<br>
tied to Coordinate node DEF names. Expression blocks define named <br>
emotional<br>
states as AU weight vectors, which the runtime lerps between smoothly.<br>
<br>
The following is a representative fragment:<br>
<br>
------------------------------------------------------------------------<br>
<?xml version="1.0" encoding="UTF-8"?><br>
<MCCFExpressions version="1.0"><br>
<br>
<!-- AU displacement geometry -- defined once per mesh --><br>
<AU name="JinJawDrop"><br>
<Displacement coord="CindyCoord_jaw"<br>
coordIndex="0 1 2 3 4 5 6 7 8 9 10 ..."<br>
vectors="0 -0.025 0 0 -0.025 0 0 -0.025 0 ..."/><br>
</AU><br>
<br>
<AU name="JinLipCornerPuller"><br>
<Displacement coord="CindyCoord_skull"<br>
coordIndex="37 38 39 40 41 42 43 44 ..."<br>
vectors="0.016 0.016 0 0.016 0.016 0<br>
-0.016 0.016 0 -0.016 0.016 0 ..."/><br>
</AU><br>
<br>
<AU name="JinBrowLowerer"><br>
<Displacement coord="CindyCoord_l_eyebrow"<br>
coordIndex="0 1 2 3 4 5 6 7 8 ..."<br>
vectors="0 -0.012 0 0 -0.012 0 ..."/><br>
<Displacement coord="CindyCoord_r_eyebrow"<br>
coordIndex="0 1 2 3 4 5 6 7 8 ..."<br>
vectors="0 -0.012 0 0 -0.012 0 ..."/><br>
</AU><br>
<br>
<!-- Named expressions -- AU weight vectors for runtime lerp --><br>
<Expression name="content"><br>
<Weight au="JinLipCornerPuller" value="0.6"/><br>
<Weight au="JinCheekRaiser" value="0.3"/><br>
</Expression><br>
<br>
<Expression name="curious"><br>
<Weight au="JinOuterBrowRaiser" value="0.5"/><br>
<Weight au="JinInnerBrowRaiser" value="0.3"/><br>
<Weight au="JinLipsPart" value="0.2"/><br>
</Expression><br>
<br>
<ExpressionState current="content" target="curious" lerp="0.3"/><br>
<br>
</MCCFExpressions><br>
------------------------------------------------------------------------<br>
<br>
At runtime, the MCCF affective engine maps emotional state (phi/epsilon) <br>
to an<br>
expression name, looks up the AU weight vector, and lerps from the <br>
current<br>
weights to the target weights over time for smooth transitions.<br>
<br>
<br>
THE HANIM EDITOR<br>
<br>
AU displacement data and named expressions are authored interactively <br>
through<br>
the MCCF HAnim Editor -- a panel in the character creator that opens an <br>
X_ITE<br>
live preview alongside face tab sliders, one per AU.<br>
<br>
The authoring workflow:<br>
<br>
1. Load a character. The morph driver caches rest pose on scene load.<br>
2. Move a slider. The displacement applies in real time to the live <br>
mesh.<br>
3. Dial in the look for a named expression (Happy, Curious, <br>
Frustrated, etc.).<br>
4. Save the expression. AU weights are written as an <Expression> <br>
block<br>
in the XML.<br>
5. At runtime, the affective engine references expressions by name and<br>
lerps between them.<br>
<br>
Ekman presets (Happy, Sad, Angry, Fearful, Disgusted, Surprised) are <br>
provided<br>
as starting points. All 30 FACS AUs are individually adjustable. A reset <br>
button<br>
returns the mesh and all sliders to neutral.<br>
<br>
We are happy to share test files, the minimal reproducer, or discuss <br>
further.<br>
<br>
We are continuing to test the editor and have pushed the current files <br>
to the public MCCF GitHub.<br>
<br>
With respect,<br>
Len Bullard<br>
MCCF Project<br>
<br>
_______________________________________________<br>
x3d-public mailing list<br>
<a href="mailto:x3d-public@web3d.org" target="_blank">x3d-public@web3d.org</a><br>
<a href="http://web3d.org/mailman/listinfo/x3d-public_web3d.org" rel="noreferrer" target="_blank">http://web3d.org/mailman/listinfo/x3d-public_web3d.org</a><br>
</div></div>