Difference between revisions of "Techniques"

From Web3D.org
Jump to: navigation, search
m (pasted BVH to X3D content here)
Line 1: Line 1:
 +
== BVH mocap conversion to X3D interpolators ==
 +
 +
Myeong Won Lee example: http://web3d.org/wiki/images/4/4b/197_a.bvh.txt
 +
 +
Proposed investigation procedure:  divide and conquer!  Let's take each skeleton apart and put them back together again.
 +
 +
----
 +
===1. Is there an X3D H-Anim skeleton that corresponds to the BVH ROOT/JOINT HIERARCHY? ===
 +
 +
The structures used by BVH and H-Anim are different.  There is not a direct correspondence between them.  Differences include:
 +
* The hierarchical nesting of parent-child relationships is different
 +
* Values are therefore in different coordinate frames (one might be relative to parent joint, the other absolute coordinates for the human)
 +
* Formats are quite different
 +
* (others will be listed here)
 +
 +
----
 +
=== 2. However many similarities also exist. Maybe they can be exploited? ===
 +
 +
Perhaps we can build a mapping between the two.  If we perform rotation transformations, a conversion algorithm can be computed for each hierarchy and corresponding units.
 +
 +
Here is a possible set of correspondences between a BVH model and an X3D H-Anim LOA-1 model.
 +
 +
* '''BVH :: X3D H-Anim'''
 +
** ROOT :: HAnimHumanoid
 +
** JOINT :: HAnimJoint
 +
*** JOINT name:: HAnimJoint name
 +
*** OFFSET :: HAnimJoint center
 +
** CHANNEL definitions :: HAnimJoint rotation, translation
 +
*** MOTION array of Euler-degree rotations :: OrientationInterpolator MFRotation array with ROUTE connections
 +
*** CHANNEL array of position translations :: PositionInterpolator MFVec3f array with ROUTE connections
 +
** End Site :: HAnimSite
 +
** Anything else?
 +
 +
TODO
 +
* map the BVH names to the approved H-Anim X3D names.
 +
 +
----
 +
=== 3. Once we have the skeleton mapping, we can work on the animation values. ===
 +
 +
Let's look at the BVH file next.  Hypothesis:
 +
* BVH MOTION arrays providing Zrotation Xrotation Yrotation values
 +
to
 +
* corresponding MFRotation arrays with 4-tuple SFRotation values
 +
 +
Example reconstruction follows.  First is an extract from the BVH file, then a counting exercise of how many floats appear.
 +
 +
  * ROOT Hips
 +
    CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation
 +
    * JOINT Chest
 +
      CHANNELS 3 Zrotation Xrotation Yrotation
 +
      * JOINT LeftCollar
 +
        CHANNELS 3 Zrotation Xrotation Yrotation
 +
        * JOINT LeftShoulder
 +
          CHANNELS 3 Zrotation Xrotation Yrotation
 +
          * JOINT LeftElbow
 +
            CHANNELS 3 Zrotation Xrotation Yrotation
 +
            * JOINT LeftWrist
 +
              CHANNELS 3 Zrotation Xrotation Yrotation
 +
      * JOINT RightCollar
 +
        CHANNELS 3 Zrotation Xrotation Yrotation
 +
        * JOINT RightShoulder
 +
          CHANNELS 3 Zrotation Xrotation Yrotation
 +
          * JOINT RightElbow
 +
            CHANNELS 3 Zrotation Xrotation Yrotation
 +
          * JOINT RightWrist
 +
            CHANNELS 3 Zrotation Xrotation
 +
      * JOINT Neck
 +
        CHANNELS 3 Zrotation Xrotation Yrotation
 +
        * JOINT Head
 +
          CHANNELS 3 Zrotation Xrotation Yrotation
 +
    * JOINT LeftHip
 +
      CHANNELS 3 Zrotation Xrotation Yrotation
 +
      * JOINT LeftKnee
 +
      CHANNELS 3 Zrotation Xrotation Yrotation
 +
      * JOINT LeftAnkle
 +
        CHANNELS 3 Zrotation Xrotation Yrotation
 +
    * JOINT RightHip
 +
      CHANNELS 3 Zrotation Xrotation Yrotation
 +
      * JOINT RightKnee
 +
        CHANNELS 3 Zrotation Xrotation Yrotation
 +
        * JOINT RightAnkle
 +
        * CHANNELS 3 Zrotation Xrotation Yrotation
 +
 +
Defines a MOTION array consisting of numeric values:
 +
 +
  3 floats [Hip location]
 +
  3 floats [Hip rotation]
 +
  3 floats [Chest rotation]
 +
  3 floats [LeftCollar rotation]
 +
  3 floats [LeftShoulder rotation]
 +
  3 floats [LeftElbow rotation]
 +
  3 floats [LeftWrist rotation]
 +
  3 floats [RightCollar rotation]
 +
  3 floats [RightShoulder rotation]
 +
  3 floats [RightElbow rotation]
 +
  3 floats [RightWrist rotation]
 +
  3 floats [Neck rotation]
 +
  3 floats [Head rotation]
 +
  3 floats [LeftHip rotation]
 +
  3 floats [LeftKnee rotation]
 +
  3 floats [LeftAnkle rotation]
 +
  3 floats [RightHip rotation]
 +
  3 floats [RightKnee rotation]
 +
  3 floats [RightAnkle rotation]
 +
  = 19*3 = 54 floats
 +
 +
Each line of the MOTION data has 19 triples.  Therefore we do have a good match.
 +
 +
----
 +
 +
=== 4. Units ===
 +
 +
Length
 +
* BVH position units: author-defined, arbitrary - no fixed units, apparently (TODO confirm)
 +
* X3D position units: 3-tuple x-y-z meters relative to point between feet on floor
 +
 +
Some scaling may be needed to get from BVH to X3D, either by a Transform scale or by a UNIT statement.
 +
 +
Angle
 +
* BVH orientation units: (Zrotation Xrotation Yrotation) Euler angles in degrees. Note that order may vary. (TODO confirm)
 +
* X3D rotation units: SFRotation 4-tuple (normalized x-y-z axis vector, angle in radians)
 +
 +
Given these definitions, we will need to convert the BVH orientation to X3D rotation.
 +
Ordinarily in 3D graphics this is performed using a 3x3 or 4x4 transformation matrix.
 +
 
=== 5. Converting BVH Motion Arrays to X3D Interpolators ===
 
=== 5. Converting BVH Motion Arrays to X3D Interpolators ===
  

Revision as of 17:53, 23 April 2014

BVH mocap conversion to X3D interpolators

Myeong Won Lee example: http://web3d.org/wiki/images/4/4b/197_a.bvh.txt

Proposed investigation procedure: divide and conquer! Let's take each skeleton apart and put them back together again.

1. Is there an X3D H-Anim skeleton that corresponds to the BVH ROOT/JOINT HIERARCHY?

The structures used by BVH and H-Anim are different. There is not a direct correspondence between them. Differences include:

  • The hierarchical nesting of parent-child relationships is different
  • Values are therefore in different coordinate frames (one might be relative to parent joint, the other absolute coordinates for the human)
  • Formats are quite different
  • (others will be listed here)

2. However many similarities also exist. Maybe they can be exploited?

Perhaps we can build a mapping between the two. If we perform rotation transformations, a conversion algorithm can be computed for each hierarchy and corresponding units.

Here is a possible set of correspondences between a BVH model and an X3D H-Anim LOA-1 model.

  • BVH :: X3D H-Anim
    • ROOT :: HAnimHumanoid
    • JOINT :: HAnimJoint
      • JOINT name:: HAnimJoint name
      • OFFSET :: HAnimJoint center
    • CHANNEL definitions :: HAnimJoint rotation, translation
      • MOTION array of Euler-degree rotations :: OrientationInterpolator MFRotation array with ROUTE connections
      • CHANNEL array of position translations :: PositionInterpolator MFVec3f array with ROUTE connections
    • End Site :: HAnimSite
    • Anything else?

TODO

  • map the BVH names to the approved H-Anim X3D names.

3. Once we have the skeleton mapping, we can work on the animation values.

Let's look at the BVH file next. Hypothesis:

  • BVH MOTION arrays providing Zrotation Xrotation Yrotation values

to

  • corresponding MFRotation arrays with 4-tuple SFRotation values

Example reconstruction follows. First is an extract from the BVH file, then a counting exercise of how many floats appear. 

 * ROOT Hips
   CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation
   * JOINT Chest
     CHANNELS 3 Zrotation Xrotation Yrotation
     * JOINT LeftCollar
       CHANNELS 3 Zrotation Xrotation Yrotation
       * JOINT LeftShoulder
         CHANNELS 3 Zrotation Xrotation Yrotation
         * JOINT LeftElbow
           CHANNELS 3 Zrotation Xrotation Yrotation
           * JOINT LeftWrist
             CHANNELS 3 Zrotation Xrotation Yrotation
     * JOINT RightCollar
       CHANNELS 3 Zrotation Xrotation Yrotation
       * JOINT RightShoulder
         CHANNELS 3 Zrotation Xrotation Yrotation
         * JOINT RightElbow
           CHANNELS 3 Zrotation Xrotation Yrotation
         * JOINT RightWrist
           CHANNELS 3 Zrotation Xrotation
     * JOINT Neck
       CHANNELS 3 Zrotation Xrotation Yrotation
       * JOINT Head
         CHANNELS 3 Zrotation Xrotation Yrotation
   * JOINT LeftHip
     CHANNELS 3 Zrotation Xrotation Yrotation
     * JOINT LeftKnee
      CHANNELS 3 Zrotation Xrotation Yrotation
      * JOINT LeftAnkle
        CHANNELS 3 Zrotation Xrotation Yrotation
   * JOINT RightHip
     CHANNELS 3 Zrotation Xrotation Yrotation
     * JOINT RightKnee
       CHANNELS 3 Zrotation Xrotation Yrotation
       * JOINT RightAnkle
       * CHANNELS 3 Zrotation Xrotation Yrotation

Defines a MOTION array consisting of numeric values: 

 3 floats [Hip location]
 3 floats [Hip rotation]
 3 floats [Chest rotation]
 3 floats [LeftCollar rotation]
 3 floats [LeftShoulder rotation]
 3 floats [LeftElbow rotation]
 3 floats [LeftWrist rotation]
 3 floats [RightCollar rotation]
 3 floats [RightShoulder rotation]
 3 floats [RightElbow rotation]
 3 floats [RightWrist rotation]
 3 floats [Neck rotation]
 3 floats [Head rotation]
 3 floats [LeftHip rotation]
 3 floats [LeftKnee rotation]
 3 floats [LeftAnkle rotation]
 3 floats [RightHip rotation]
 3 floats [RightKnee rotation]
 3 floats [RightAnkle rotation]
 = 19*3 = 54 floats

Each line of the MOTION data has 19 triples. Therefore we do have a good match.

4. Units

Length

  • BVH position units: author-defined, arbitrary - no fixed units, apparently (TODO confirm)
  • X3D position units: 3-tuple x-y-z meters relative to point between feet on floor

Some scaling may be needed to get from BVH to X3D, either by a Transform scale or by a UNIT statement.

Angle

  • BVH orientation units: (Zrotation Xrotation Yrotation) Euler angles in degrees. Note that order may vary. (TODO confirm)
  • X3D rotation units: SFRotation 4-tuple (normalized x-y-z axis vector, angle in radians)

Given these definitions, we will need to convert the BVH orientation to X3D rotation. Ordinarily in 3D graphics this is performed using a 3x3 or 4x4 transformation matrix.

5. Converting BVH Motion Arrays to X3D Interpolators

The goal is to determine an algorithm to take BVH MOTION arrays to create PositionInterpolator and OrientationInterpolator.

First some definitions, then conversions, step by step.

5a. Computation of duration value

BVH excerpt: 

 MOTION
 Frames: 482
 Frame Time: 0.016667

Note that these units are seconds. Frame time matches 60 frames per second.

The first entry in the time array is at time zero. Therefore there are (482 - 1) = 481 intervals between each frame.

Thus total duration = (# intervals)(frame duration) = (481 * 0.016667) = (481 / 60) = 8.01667 seconds.

Using this computation, the derived X3D arrays can then be driven by a TimeSensor and key values at appropriate frame-time increments.

X3D TimeSensor duration is expressed by a value like 

  cycleTime="8.01667"

5b. Interpolator key array

In X3D, an interpolator key is typically a fraction from 0 to 1.0.

  • take total time interval of BVH file, example 8.01667 seconds
  • divide by number of intervals, for example 481
  • resulting key array would have fractions incrementing by (1.0 / 481) = 0.016667 (looks familiar)
  • add increments, result is
 key = "0 1/481     2/481       3/481      ...etc... 1" or
 key = "0 0.002079  0.002079*2  0.002079*3 ...etc... 1"
 key = "0 0.002079  0.004158    0.006237   ...etc... 1"

Note that the X3D Interpolator keyValue array will have same length as rotation array

  • must convert each triple of Euler angles to a corresponding SFRotation (see preceding section 4)
  • result will be something like
 keyValue="0 1 0 0.0, 0 1 0 0.111, 0 1 0 0.123 ...etc..." etc.

5c. TimeSensor and ROUTE

TimeSensor fraction_changed output is ROUTEd to OrientationInterpolator set_fraction. Result: 

 <TimeSensor DEF="AnimationClock" cycleTime="8.01667" />
 
 <OrientationInterpolator DEF="X3dMotionArray"
        key = "0 0.002079  0.004158    0.006237   ...etc... 1"
    keyValue="0 1 0 0.0, 0 1 0 0.111, 0 1 0 0.123 ...etc..." />
 
 <ROUTE fromNode="AnimationClock" fromField="fraction_changed"
          toNode="X3dMotionArray" fromField="set_fraction"    />

Informational note: the time values corresponding to the key values above do not have to be included in the scene. Nevertheless they are computed by a browser as follows: 

 key = 0       1/481         2/481         3/481 ...         481/481
 time: 0 8.01667/481 8.01667*2/481 8.01667*3/481 ... 8.01667*481/481  ==> 0 0.016667  0.016667*2  0.016667*3  ... 8.01667

5d. Computing keyValue arrays

This section examines how BVH values are excerpted and converted into various keyValue arrays. In other words, we take two frames of a BVH file to produce PositionInterpolator and OrientationInterpolator nodes.

Below is an example of two unmodified, unmarked BVH frames: 

 BVH Motion
 Frame 0:
 1.662 31.427 60.304  -1.249 -4.859 -3.582  4.463 1.354 0.075   -13.732 3.052 3.999  
 95.677   1.705 -1.512   5.541 -3.491 0.339   1.259 -3.022 1.790   6.765 2.405 -4.446  
 -91.027 -7.187 4.910   -3.633 0.867 0.043   -2.879 0.120 -5.688   -1.132 -1.858 0.809 
 -2.969 -8.472 1.461   -1.304 3.919 -2.045   1.054 9.006 -0.191   2.695 -1.341 -0.615 
 0.361 4.452 4.756   0.484 8.095 0.193   -6.340 -0.815 1.224
 
 Frame 1:
 1.659 31.427 60.307  -1.268 -4.835 -3.588  4.487 1.352 0.080   -13.802 3.059 3.999
 95.651   1.737 -1.609   5.541 -3.521 0.340   1.298 -3.030 1.974   6.795 2.410 -4.418 
 -90.999 -7.145 4.917   -3.633 0.825 0.043   -2.862 0.151 -5.736   -1.141 -1.863 0.806  
 -2.988 -8.482 1.455   -1.283 3.890 -2.157   1.055 9.042 -0.193   2.781 -1.350 -0.638 
 0.381 4.430 4.692   0.484 8.102 0.194   -6.266 -0.819 1.206

Next we will use excerpts from the above BVH frame values to show example conversions into X3D PositionInterpolator and OrientationInterpolator.

  • The keyValue array for the PositionInterpolator are obtained from the first three values at all frames.
  • The keyValue array for each OrientationInterpolator per joint are obtained by Euler-to-SFRotation conversion from the corresponding three values at all frames for each corresponding joint.
  • We do not yet perform the Euler-to-SFRotation conversion, but we show where it will occur.

Color codes: translation values, joint 1 rotations, joint 2 rotations, and joint 3 rotations 

 BVH Motion
 Frame 0:
 1.662 31.427 60.304  -1.249 -4.859 -3.582  4.463 1.354 0.075   -13.732 3.052 3.999 ...

 Frame 1:
 1.659 31.427 60.307  -1.268 -4.835 -3.588  4.487 1.352 0.080   -13.802 3.059 3.999  ...
 
 <PositionInterpolator DEF='HumanoidRootTransInterp' key='0 1/481 2/481 3/481 ...' keyValue='1.662 31.427 60.304,  1.659 31.427 60.307, ...' />
 <OrientationInterpolator DEF='HumanoidRootRotInter' key='0 1/481 2/481 3/481 ...' keyValue='f(-1.249 -4.859 -3.582), f(-1.268 -4.835 -3.588), ...'/>
 <OrientationInterpolator DEF='sacroiliacRotInterpo' key='0 1/481 2/481 3/481 ...' keyValue='f(4.463 1.354 0.075), f(4.487 1.352 0.080), ... '/>
 <OrientationInterpolator DEF='l_hipRotInterpolator' key='0 1/481 2/481 3/481 ...' keyValue='f(-13.732 3.052 3.999), f(-13.802 3.059 3.999), ...'/>
 
 where
 f(phi,theta,psi) means euler-to-SFRotation conversion. (Might need to check order of BVH rotations to match phi,theta,psi).

Using other expressions, here is a pattern for the conversion mapping from BVH to X3D keyValue arrays.

The meaning of these BVH value labels are as follows:

  • "0-tz" means "frame time 0, translation along z axis"
  • "1-rotz2" means "frame time 1, single Euler-rotation angle about z axis, for joint 2"
 BVH Motion
 Frame 0:
 0-tx 0-ty 0-tz, 0-rotx0, 0-roty0, 0-rotz0, 0-rotx1, 0-roty1, 0-rotz1, 0-rotx2, 0-roty2, 0-rotz2, ...
 
 Frame 1:
 1-tx 1-ty 1-tz, 1-rotx0, 1-roty0, 1-rotz0, 1-rotx1, 1-roty1, 1-rotz1, 1-rotx2, 1-roty2, 1-rotz2, ...

 <PositionInterpolator  DEF='HumanoidRootTransInterp' key='0.0 1/481 2/481 3/481 ...' keyValue='0-tx 0-ty 0-tz, 1-tx 1-ty 1-tz, ...'/>
 <OrientationInterpolator DEF='HumanoidRootRotInterp' key='0.0 1/481 2/481 3/481 ...' keyValue='f(0-rotx0, 0-roty0, 0-rotz0), f(1-rotx0, 1-roty0, 1-rotz0), ...'/>
 <OrientationInterpolator DEF='sacroiliacRotInterp'   key='0.0 1/481 2/481 3/481 ...' keyValue='f(0-rotx1, 0-roty1, 0-rotz1), f(1-rotx1, 1-roty1, 1-rotz1), ... '/>
 <OrientationInterpolator DEF='l_hipRotInterp'        key='0.0 1/481 2/481 3/481 ...' keyValue='f(0-rotx2, 0-roty2, 0-rotz2), f(1-rotx2, 1-roty2, 1-rotz2), ...'/>

5e. Conversion algorithm observations

In essence this work tries to duplicate the BVH animation expressions using X3D.

As shown in the above example, there are differences in the orders of position and orientation values between BVH and X3D Interpolator nodes.

  • In BVH, each frame includes a position x-y-z value for the root joint, and then 3-tuple Euler-angle rotation values for each and every one of the joints.
  • In X3D Interpolator, each joint has all position and rotation values for each of the timed frames.
  • There appears to be one-to-one correspondences between the data in BVH and the keyValue arrays in X3D.
  • In effect this reordering simply resembles a matrix transposition (vertical colored columns for BVH are mapped to horizontal colored rows in X3D).

We see the following file size characteristics in this conversion of BVH data into X3D Interpolators:

  • BVH does not need key arrays, because all time intervals are equal.
  • X3D is somewhat more verbose since the same key arrays are duplicated for all joints. This duplication of key arrays can lead to larger file size, in text form, but data compression can eliminate the cost of array copies.
  • There appears to be approximately the same amount of rotation information in each representation.
  • BVH has no ROUTE definitions. Typically there is only one ROUTE statement for the output of each node for each H-Anim humanoid. (There may be multiple ROUTEs for each node if there are multiple humanoids.)

Additional notes:

  • Time-based frame-by-frame expression is appropriate for animation sequences such as BVH, i.e. a frame includes a time stamp, a single position, and orientation values of all joints.
  • X3D viewers only use axis-angle SFRotation values, not quaternions and not Euler angles. For example, in an X3D OrientationInterpolator, if (1 0 0 0.5) means x-axis rotation with 0.5 radian, then (1 1 1 0.5) means an arbitrary axis from (0 0 0) to (1 1 1) with 0.5 radian rotation about that axis, using the right-hand rule for direction.
  • Define the function f(phi,theta,psi) for converting a single BVH Euler-angle triplet into a single X3D SFRotation axis-angle value.
  As shown in the BVH hierarchy as below,
  ------------------------------------------
 HIERARCHY
 ROOT Hips
 {
  OFFSET 0.000000 0.000000 0.000000
  CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation
  JOINT Chest
  {
   OFFSET 0.000000 5.613096 0.000000
   CHANNELS 3 Zrotation Xrotation Yrotation
   JOINT LeftCollar
   {
     OFFSET 0.003804 10.354579 1.025227
     CHANNELS 3 Zrotation Xrotation Yrotation
     JOINT LeftShoulder
     {

In this BVH file, the order of one BVH Euler angle is z, x, y, while an X3D SFRotation includes the order of axes, x, y, z. So we must be careful that the order of angles in an Euler triple is different from the order of axes in an X3D SFRotation.

6. Conversion algorithm details

(TODO: verify)

The transformation from BVH Euler angles to SFRotation is considered as follow:

Simply speaking, we must transform a triple value (x-rot y-rot z-rot angles in degree) into a quadruple value (x y z w angle in radian). Therefore, our transformation algorithm is as follows:

1) Choose the largest rotation angle among the three Euler angles.

2) Define the axis value as 1 for the rotation axis with the largest rotation angle for the quadruple.

3) Convert the largest angle in degrees into angle in radian, and this will be w for the quadruple.

4) The remaining axis values are calculated from the proportion of the rotation angle against the largest rotation angle.

For example, consider Euler angles (in degrees) x=90, y=45, z=-180.

  • From the algorithm, SFRotation is x=0.5, y=0.25, z=-1, w=3.14
  • For another example, consider Euler angles x=30, y=30, z=30.
  • From the algorithm, SFRotation is x=1, y=1, z=1, w=3.14/6.

The following is the implementation code of the algorithm: 

 void HAnim::ConvertEulerToSFRotation(float& x, float& y, float& z, float& w)
 {
   #define piover180 0.01745329252f

   float fAbsoluteX = abs(x);
   float fAbsoluteY = abs(y);
   float fAbsoluteZ = abs(z);
   
   float fMaxAngle = 0;
   
   if(fAbsoluteX > fMaxAngle)
      fMaxAngle = fAbsoluteX;
   if(fAbsoluteY > fMaxAngle)
      fMaxAngle = fAbsoluteY;
   if(fAbsoluteZ > fMaxAngle)
      fMaxAngle = fAbsoluteZ; 

   float fRatio = 0.0f;
 
   if(fMaxAngle > 0.0001f)
      fRatio = 1.0f / fMaxAngle; 
  
   x = x * fRatio;
   y = y * fRatio;
   z = z * fRatio;
 
   w = fMaxAngle * piover180;
   w = abs(w);
}

How to convert

TODO: this section has information that complements the preceding sections.

We can combine the pieces to show corresponding animations in HAnim and BVH. We can build an application program that uses these mappings.

  • This conversion program operates off-line, not within an X3D scene, because X3D does not allow reading files.

Germane Wikipedia articles:

It looks like choosing the right matrix operations will perform this conversion. This requires careful composition of rotation matrix operations.

Retrieved from "https://web3d.org/wiki/index.php?title=Techniques&oldid=8176"