[x3d-public] FW: BVH to X3D Conversion bug

[Joseph D Williams]

From: John Carlson
Sent: Monday, November 7, 2022 5:01 PM
To: Joseph D Williams
Cc: Brutzman, Donald (Don) (CIV); X3D Graphics public mailing list
Subject: Re: [x3d-public] FW: BVH to X3D Conversion bug

➢ My albiet impractical approach was to bring your own skeleton, and add whatever motion there is in the bvh file to the Motion object or HAnimMotion element.  

Fine, very useful, getting the bvh skeleton to work is fine for experiment and comparing bvh motion with bvh skeleton against bvh motion with Hanim skeleton. Should be some comparable animations but since it may be important to demonstrate actual realistic body motions, the bvh armature is less proficient than the Hanim version. 

➢  I did not realize that BVH files contained their own skeleton.  

It may be mostly the same in lots of examples. I think that is really a first step, to figure out if the bvh skeleton will run some ‘standard’ animation, like the walk or run in archives. If the bch skeleton can’t do the more or less standard Walk example we have, then maybe bvh animation may not be useful. 

➢  Seems like one should Protoify a standard skeleton and pass the motion as a parameter.   

That is fine plan. Think of how to bring in a new animation, like you are walking, then want to wave, and those are two mostly independent animations. 

➢ But I didn’t even get a chance to adequately look at a bvh file beforehand.   May I suggest we pursue the Motion method, if we are all in agreement?

Please notice that the typical bvh consists of a skeleton and an animation routine of some certain length. 
The animation information consists of a constant time step and a set of keyframe data for each joint to be animated, at each time step. Since the bvh is probably aimed at video frame rates, this means lots of keytimes and lots of keyframe data – one keytime for each frame and one keyvalue for each joint at each frame. If one second is 30 frames, then for each animation one time value and joints times frames. Oh, and the bvh animation data probably uses xyz angles rather than axis-angle, or may be quaternions in gltf. 
When you convert this bvh animation to x3d timer-interpolator style, then the keytimes are all listed for every interpolator and also all keyvalues in axis-angle. If the bvh was 30 frames, then for each animation 30 keytimes and 30 keyvalues for each joint. Once you have this, though, at least it is in a form that is editable, changing animations and adding to the base. 

Recall how the bvh animations were probably made, by dragging limb to some point, saving that, moving to another, then setting an interpolater to work on the motion and storing it frame by frame in bvh for export to make a video at a fixed frame rate with the renderer actually not needing interpolator since all keytimes are set. Just what you need for a video render bot. Of course almost everybody knew that in order to exchange animations, the playback skeleton needed to be similar to the capture skeleton and any difference between the capture initial pose and the playback initial pose had to be known. 

Anyway, the big point is that in order to actually take control of the animation using your skeleton, you want it to be in x3d style rather than bvh style. Usually find way too many keyframes in bvh. More fun to learn how to edit out unneeded stuff and refine that than to have to deal with the excessive file sizes when you can author in realtime with interpolation rather than videotime.

➢ I realize that X3DOM/Xj3d may not support the Motion/mocap stuff in XML.   

Well, it seems like Don just told me that x3d player is supposed to support bvh like a native without the need for Hanim skeleton(-1) or x3d timer-interpolator expression. Maybe figure out how that is done. The spec tell how to do the xyz angle to axis-angle.

➢ 
➢ I’m not aware if these are currently supported.  I do know that X_ITE does.  I do know there is BVH handling in JavaScript.

➢ So it seems like we’re translating bvh to non-standard skeletons, instead of translating BVH to standard motion.

To me, the ideal is the application of a ‘standard’ animation to a ‘standard’ skeleton. This means extracting the bvh animation for use in the Hanim skeleton. 
The bvh file probably contains a sort of consensus skeleton that can be basis for a ‘standard’ Hanim skeleton, hopefully by some systematic process giving correct bone to joint connections. Since all bvh skeletons I have seen are -1 style, we just need some upgraded shoulder stuff. The bvh animation data is based on the skeleton in the file and if it is similar to the target Hanim skeleton, then the animation may be useful. 

➢ John

Hey John we are just getting to the fun part. 

Best of All, 
Joe


On Mon, Nov 7, 2022 at 5:16 PM Joseph D Williams <joedwil at earthlink.net> wrote:
Joe >> However the piro bvh example is Not Level 1, or any level because the bvh skeleton is (typically) Not a legal Hanim level of articulation. 
This skeleton may be typical in bvh archives because it is a legacy artifact from before we had a world standard realistic skeleton. Just look at it. Is that the way your back works? But, the bvh skeleton is simple for simple stuffs.
 
A point is that the bvh skeletons I have seen are not any Hanim level because the spine is connected in a different way. There are other minor difs also. 
 
In the following, it is first about how to use a Standard Hanim skeleton with the bvh animation. 
Please not simply activating the bvh capture skeleton with the bvh animation because the bvh skeleton is probably not a standard hierarchy Hanim skeleton. 
 
 
• As far as which approach is best – not currently an issue.  At present, am simply trying to convert legacy BVH files into the most compliant X3D HAnim possible.  That path can give us a LOT of content.  Still moving forward, so far…
 
Which is best is not an issue because bvh is not an authoring approach, it is a transport approach. For example, data from a defined bvh capture skeleton applied to a defined playback skeleton. The main problem is that the bvh skeleton is (probably) not any Hanim skeleton. 
I think we get usable content when the bvh animation is incorporated with _your_ “Standard” Hanim skeleton. 
 
The animation is fully  incorporated with your skeleton when the animation is set up using x3d animation styles with reasonable keytimes and keyframe data, and therefore can be used as a basis for detailed animation of _your_ skeleton. 
 
Another idea for the x3d user code would be to allow construction of an interpolator that just listed the constant keytime interval instead of every keytime. That is what the gltf prefers since the rules for setting key times is more simple than x3d. However, since the bvh is aimed at video, that still leaves the problem of mostly too much keyvalue data. 
 
If we can provide a reasonable conversion from the bvh skeleton to Hanim loa4, then the user can edit and simplify the key and keyvalue data to finally adapt the animation to the target skeleton and integrate it with other animations.
 
Thanks Again, 
Joe
 
Again, the reason the target playback is a ‘standard’ loa4 is that the player will not complain if there is a joint that is not animated. The author can always easily add animation of joints not controlled by the bvh import. 
 
From: Brutzman, Donald (Don) (CIV)
Sent: Sunday, October 30, 2022 12:21 PM
To: Joseph D Williams
Cc: X3D Graphics public mailing list; Brutzman, Donald (Don) (CIV)
Subject: RE: [x3d-public] FW: BVH to X3D Conversion bug
 
Joe you are correct historically – but we added a third level for HAnim motion node in X3D4.  Schema, DTD are up-to-date, examples corrected too.  Am correcting other tools and diagnostics as I find them.
 
• X3D4 Architecture, Clause 26 Humanoid Animation (HAnim) component
• Table 26.2 — Humanoid animation (HAnim) component support levels
• https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-DIS/Part01/components/hanim.html#t-supportLevels
 
As far as which approach is best – not currently an issue.  At present, am simply trying to convert legacy BVH files into the most compliant X3D HAnim possible.  That path can give us a LOT of content.  Still moving forward, so far…
 
Building an X3D widget to follow your note and control frame-by-frame timing of HAnimMotion advancement by TimeSensor will be interesting…
 
all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
X3D graphics, virtual worlds, Navy robotics https:// faculty.nps.edu/brutzman
 
From: x3d-public <x3d-public-bounces at web3d.org> On Behalf Of Joseph D Williams
Sent: Friday, October 28, 2022 7:04 PM
To: X3D Graphics public mailing list <x3d-public at web3d.org>
Subject: [x3d-public] FW: BVH to X3D Conversion bug
 
 
Brought out to public
 
From: Joseph D Williams
Sent: Friday, October 28, 2022 5:49 PM
To: John Carlson; Norbraten, Terry (CIV); X3D Graphics public mailing list
Cc: Brutzman, Donald (Don) (CIV)
Subject: RE: BVH to X3D Conversion bug
 
• The Pirouette.x3d has
• <component level='1' name='HAnim’/>
 
An error.
Hanim has basically two levels – 
skeleton with segment geometry and 
skeleton with skin. 
Check me on this. 
However the piro example is Not Level 1, or any level because the skeleton is Not a legal level of articulation. 
This skeleton may be typical in bvh archives because it is a legacy artifact from before we had a world standard realistic skeleton. Just look at it. Is that the way your back works? But, it is simple for simple stuffs.
 
If you play with this loa non-conformer, like for instance hook the animation up to any of the more or less standard and proven stand walk run jump twist etc. animations in the archive, then I am fairly sure you will see unexpected gestures. And, when one or some of these work in a ‘standard’ hierarchy skeleton, then add it to the “Standard” examples. 
 
Certainly, as you progress with authored hanim animation style, then you will wish you had started with current vs legacy skeleton, same as possible regret if you started with anything else than an loa4. 
 
However, this example certainly shows that out of any bvh collections, the native production is probably only a very rough starting point with lots of work left to adapt the captured motion to a usable playback character. 
 
So (from the Peanut Gallery), please do not run that non-conforming skeleton as a conforming hanim character anymore unless, If you really can’t get conforming x3d hanim skeleton as playback, then please don’t archive as hanim without proper bvh skeleton Disclaimer. 
Again, just convert to use the ‘standard’ loa4 as the target playback, at humanoid scale. Then, if interesting, the animations can be extracted for use in a legal skeleton and coded for realtime rather than videotime. Please don’t just simply recreate the bvh-encoded skeleton – convert directly to loa4 at human scale and then you can see what you really have.  
 
Please note one advantage of the typical video frame rate seen in these animation thingees, is that to make a video from these your player does not need to use interpolators since you give it the exact keyframe interval data. 
 
More fun with realtime hanim. 
 
Best regards,
Joe
 
From: John Carlson
Sent: Friday, October 28, 2022 2:53 PM
To: Joe D Williams; Norbraten, Terry (CIV)
Cc: Brutzman, Donald (Don) (CIV)
Subject: Re: BVH to X3D Conversion bug
 
Last i heard, HAnim is acceptable.  Joe can confirm, cced.
 
On Fri, Oct 28, 2022 at 10:00 AM Norbraten, Terry (CIV) <tdnorbra at nps.edu> wrote:
One issue is that the BVH converter is not translating the correct level for H-Anim. 
 
The Pirouette.x3d has
 
<component level='1' name='HAnim’/>
 
Should be
 
<component level='1' name=‘H-Anim’/>   <<< require the hyphen 
 
On Oct 28, 2022, at 05:59, Brutzman, Donald (Don) (CIV) <brutzman at nps.edu> wrote:
 
Thank you John for excellent logs and sleuthing.
 
Terry, the error immediately jumping out at me is the following.  At least one library is not getting bundled properly.
 
•        java.lang.ClassNotFoundException: javax.vecmath.Vector3d
 
Great discussion yesterday Terry, thanks.  I have created the pre-release file site at
 
•        https://sourceforge.net/projects/x3d/files/X3D-Edit%20Pre-Release%20Testing/
 
all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
X3D graphics, virtual worlds, Navy robotics https:// faculty.nps.edu/brutzman
 
From: John Carlson <yottzumm at gmail.com> 
Sent: Thursday, October 27, 2022 12:09 PM
To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; Norbraten, Terry (CIV) <tdnorbra at nps.edu>
Subject: Re: Trying new download--X3D-Edit 4
 
NPS WARNING: *external sender* verify before acting.
 
I have confirmed the Pirouette.bvh in archives and pirouette.bvh are the same, essentially, just different whitespace.   I will now try to see if X3D-Edit will load the archive version. 
 
Attached are the logs that are showing up in Windows 10 X3D-Edit dev/var/log.
 
I haven't looked into them yet, I will now move to Linux.
 
If you could send me the X3d-Edit steps how to convert BVH to X_ITE and X3DOM, that would be great.
 
Forward old email if necessary.
 
 
 
_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20221107/89f459de/attachment.html>