[x3d-public] STEP interpolation, was Blender > Exporting rig transforms to HAnim?

[Andreas Plesch]

On Mon, Jul 17, 2023 at 5:13 PM Michalis Kamburelis
<michalis.kambi at gmail.com> wrote:
>
> 1. Edge-cases:
>
> Yes, I confirm all 3 edge-cases you mentioned:
>
> 1. key[exactly at n] => keyValue[n]
> 2. key[< nMin] => keyValue[nMin]
> 3. key[> nMax] => keyValue[nMax]
>
> Inluding AD 1, yes, I checked. I'm not sure how much it matters what
> happens when input is *exactly* at given key (since possibility of
> floating-point errors means one should not rely on anything being
> *exactly equal*) but I see our code does logic to make it work.
> Linking to X3D spec and a related bugreport from you :)
> https://github.com/castle-engine/castle-engine/blob/master/src/scene/x3d/x3dnodes_miscellaneous_globals.inc#L338
> . This piece of code affects both linear and step interpolation.

Ok. Super.

> 2. Naming:
>
> > x3dom has a different DEF naming convention prefixing with 'glTF_' and
> using the node.name when available but I do not think it is necessary
> to align.
>
> Note that in Castle Game Engine / view3dscene, I did some thinking and
> changing implementation back-and-forth to find my optimal "best
> approach to rename nodes at import". I documented my decision and
> reasons on https://github.com/castle-engine/castle-engine/blob/master/doc/miscellaneous_notes/avoid_renaming_node_names_at_import.md
> . Basically, now we avoid doing *any* node renames on import. The
> goal: Ideally, if you call something "MyGroup" in Blender, you should
> be able to use in CGE `MyScene.Node('MyGroup')` to get a related
> transformation. Adding any prefixes/suffixes to "MyGroup" was not
> good, mostly because users generally didn't know about the rules of
> any prefixes/suffixes.

Good discussion. Inlines have their own namescope so prefixing to more
or less avoid clashes would not be necessary. Prefixing and postfixing
was just the safest, quick way to generate unique names. I think we
use json array index as the postfix.as glTF uses the index as a
reference. One could also use the full JSON path as a name, perhaps
combined with node.name. That may be a way to standardize.

Special consideration may go to nodes involved in animations since
their DEF names need to be robustly unique.

$ is legal in xml DEF names just not recommended.

> This is also the reason behind `$` you may have seen in some CGE
> names, like `CastleEncoded_Material$46$007`. When the glTF (or OBJ or
> Collada...) names have some weird characters, we actually want to keep
> these "weird characters" in X3D node names in CGE API. When
> reading/writing X3D node names, we "encode" the name if it has weird
> characters.
>
> So e.g. <Appearance DEF="CastleEncoded_Material$46$007"... >
>
> is seen from CGE API as TAppearanceNode with name 'Material.007' and
> can be found from Pascal like `MyScene.Node(TAppearanceNode,
> 'Material.007')`. This way you can use in CGE API the same name as you
> set in e.g. Blender (and it is exported to glTF).
>
> This is all FYI -- I agree that X3D browsers don't need to have these
> node naming strategies aligned (at least for now), and surely it makes
> sense for everyone to explore best approach for given use-case. I'm
> just saying what/why we do in CGE :)
>
> 3. keyValueQuaternions:
>
> Oh indeed, I forgot about this feature of CGE :)
>
> Basically since glTF rotations are already quaternions, and
> interpolating rotations (with slerp or similar) requires to convert
> them to quaternions anyway... for speed, I decided it makes no sense
> in CGE to convert glTF quaternions -> axis+angle, and then every time
> we need to interpolate do 1. axis+angle -> quaternions, 2. interpolate
> using slerp, and 3. convert quaternions -> axis+angle.
>
> Avoiding these conversions was not only faster, it also results in
> more straightforward code.
>
> Interesting, so I see that X3DOM took even more "quaternions
> internally everywhere" approach.

Yes, the keyValues are internally stored as quaternions (as are all
SFRotations).

> As for how CGE API exposes rotations:
>
> - We generally expose rotations as axis+angle, so just like X3D
> SFRotation/MFRotation.
>
>     I think X3D made this decision right, i.e. axis+angle is a nice
> rotation representation. It is easier to visualize for humans (i.e.
> you can read and visualize axis) than quaternions. And it avoids
> various Euler angles problems (Gimbal lock, various possible order of
> angles making Euler angles unclear to interpret without extra info).
>
> - We also have quaternions in CGE API naturally to employ when make
> sense (e.g. for slerp).
>
> - We managed to completely avoid in CGE API *ever* using or displaying
> rotations in Euler angles. I'm happy from this decision -- Euler
> angles, while popular, bring a lot of complication (various possible
> angle orders, gimbal lock) and IMHO little gain. Simple rotations are
> easy to express in axis+angle too.

Much agreed on all points. The only issue is that Euler angles are so
widely used. But most, perhaps all framework also deal with axis+angle
and can convert.

> This was again a lot of "FYI, in case you're curious", sorry for the
> long message :)

Thanks again. Regards, Andreas

> Regards,
> Michalis
>
> pon., 17 lip 2023 o 17:59 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> >
> > Hi Michalis,
> >
> > On Mon, Jul 17, 2023 at 9:23 AM Michalis Kamburelis
> > <michalis.kambi at gmail.com> wrote:
> > >
> > > > For STEP, I am using the lower value of the interval:
> > > ...
> > > > This corresponds to typical x3d interval use, and you may have done the same ?.
> > >
> > > Yes, that is indeed what Castle Game Engine / view3dscene is doing too.
> > >
> > > If input is between key[n] and key[n+1], then the output of STEP is keyValue[n].
> >
> > Super. There are also the edge cases:
> >
> > key[exactly at n] => keyValue[n]
> > key[< nMin] => keyValue[nMin]
> > key[> nMax] => keyValue[nMax]
> >
> > Just to confirm.
> >
> > > I started converting your testcase (
> > > https://github.com/andreasplesch/x3dom/blob/step_interpolator/test/regression-suite/test/cases/interpolators/interpolators.html
> > > ) into standalone X3D file. I noticed a few things special to X3DOM I
> > > think (aside from lowercase node / field names, which I understand is
> > > just necessary for HTML5 integration):
> > >
> > > - "alphaMode" and "alphaCutoff" specified at "PhysicalMaterial", while
> > > in X3D 4 they are on "Appearance"
> >
> > Yes, for historical reasons, because they only apply to PhysicalMaterial.
> > Appearance has alphaClipThreshold for regular Material in x3dom. There
> > was some reason why it was not not reused for PhysicalMaterial perhaps
> > due to shader uniforms or subtle usage differences.
> >
> > > - "PhysicalMaterial.unlit" boolean, while in X3D 4 we have separate
> > > node "UnlitMaterial" instead
> > >
> > > - baseColorFactor -> it is baseColor in X3D 4
> > >
> > > - metallicfactor, roughnessfactor -> just metallic, roughness in X3D 4
> >
> > Will be easy to rename.
> >
> > > - diffusefactor, specularfactor, glossinessfactor on PhysicalMaterial
> > > -> in X3D 4 we follow te glTF standard "metallic-roughness" workflow,
> > > so there's no no specular / glossinness.
> >
> > There used to be specular/glossiness for glTF1. We would probably just
> > keep it as there is also an extension.
> >
> > > In the end, it was a bit easier to take glTF from
> > > https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/InterpolationTest/glTF
> > > and open it in view3dscene, or convert to X3D from view3dscene. But I
> > > know this is not objective test, sure it was easier this way for me to
> > > just open in CGE/view3dscene :)
> >
> > Of course.
> > x3dom has a different DEF naming convention prefixing with 'glTF_' and
> > using the node.name when available but I do not think it is necessary
> > to align.
> > I noticed the $ character in the DEF names but see
> > https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#NamingConventions:
> > 'Most restrictive and most interoperable guidance regarding choice of
> > characters in names: can start with letter or underscore character,
> > but not a number or a colon. Can then include letters, numbers,
> > hyphen, underscore, or period characters. This interoperability is
> > important so that files can be converted equivalently between
> > encodings without validation problems.'
> >
> > Shapes do not get named
> >
> > > Quickly scanning view3dscene output, the one CGE/view3dscene-specific
> > > thing there is "ImageTexture.flipVertically" (we plan to add
> > > alternative to it, using texture transform for max conformance), the
> > > rest looks standard X3D 4.
> >
> > x3dom has boolean flipY which I believe corresponds to an OpenGl flag.
> > I also noticed the boolean OrientationInterpolator.keyValueQuaternions
> > . x3dom converts all rotations from axis-angle to Quaternions during
> > parsing, so this would save a little bit time there but not during the
> > animation. I would assume Euler angles would be a more common format
> > to accept from modeling software but that might be one step too far.
> >
> > Thank you for the nice list which will make it more manageable to work through.
> >
> > I found another animation example here:
> > https://github.com/mrdoob/three.js/pull/12907
> > and uploaded it here:
> > https://github.com/andreasplesch/Library/blob/gh-pages/Examples/gltf2/interpolation/InterpolationComparison.gltf
> >
> > Regards, Andreas
> >
> >
> >
> > >
> > > Regards,
> > > Michalis
> > >
> > >
> > > pon., 17 lip 2023 o 07:16 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> > > >
> > > > Hi Michalis,
> > > >
> > > > I am putting finishing touches on STEP and CUBICSPLINE interpolation modes.
> > > >
> > > > For STEP, I am using the lower value of the interval:
> > > >
> > > > key='0 1 2'
> > > > keyValue='10 11 12'
> > > >
> > > > value at key= 0.6 : 10
> > > > value at key= -0.1 : 10
> > > > value at key= 2 : 12
> > > > value at key= 2.1 : 12
> > > >
> > > > This corresponds to typical x3d interval use, and you may have done the same ?.
> > > >
> > > > I also added CUBICSPLINE to all interpolators except for
> > > > ColorInterpolator (could be added but seems strange). Doing that I
> > > > found that for CoordinateInterpolator and NormalInterpolator the
> > > > layout of keyValues with in and out tangents was most natural this
> > > > way:
> > > >
> > > > keyValue='FrameData1, FrameData2..FrameDataN'
> > > >
> > > > where FrameData is the data for a key frame and looks like:
> > > >
> > > > [inTangent1, inTangent2 .. inTangentM],
> > > > [value1, value2 .. valueM],
> > > > [outTangent1, outTangent2 .. outTangentM]
> > > >
> > > > where all entries are Vec3f.
> > > >
> > > > The total number of scalar values in keyValue is thus N * 3 * M * 3.
> > > >
> > > > I could not test CUBICSPLINE for Coordinate or NormalInterpolator yet
> > > > since I do not have a test example. Does any scene come to mind ?
> > > >
> > > > Perhaps I can augment the Dolphin example with some arbitrary tangents
> > > > for testing.
> > > >
> > > > I also converted the glTF Interpolation Test example to X3D:
> > > > https://github.com/andreasplesch/x3dom/blob/step_interpolator/test/regression-suite/test/cases/interpolators/interpolators.html
> > > >
> > > > It may be useful for testing Position and OrientationInterpolator without glTF.
> > > >
> > > > Andreas
> > > >
> > > > On Tue, Jun 27, 2023 at 10:34 AM Andreas Plesch <andreasplesch at gmail.com> wrote:
> > > > >
> > > > > Hi Michalis,
> > > > >
> > > > > Ok, let me work on adding a STEP interpolation mode to interpolators.
> > > > > Do you want to raise a spec. improvement suggestion ?
> > > > >
> > > > > As  a side note, STEP mode is also related to x3d sequencers. A
> > > > > ScalarInterpolator in STEP mode would be equivalent to a hypothetical
> > > > > ScalarSequencer, except for the next and previous fields. Hm, perhaps
> > > > > it makes sense to add next and previous fields to interpolators which
> > > > > would step through intervals.
> > > > >
> > > > > The CUBICSPLINE algorithm for glTF is fairly well documented as far as
> > > > > I remember. Feel free of course to look up the implementation in
> > > > > x3dom.
> > > > >
> > > > > It is worth noting that the existing SplineInterpolator nodes only
> > > > > allow for one (central) tangent per key and key value, whereas the
> > > > > common cubic spline algorithm allows for two tangents (in, and out)
> > > > > per key and key value. Those mathematically inclined may be able to
> > > > > show that the x3d algorithm for spline interpolations is equivalent to
> > > > > the cubic spline algorithm with identical in and out tangents.
> > > > >
> > > > > x3dom just mirrors glTF in that inTangent, value, outTangent occur in
> > > > > triplets in the MF keyValue field. An alternative interface would
> > > > > introduce keyInTangent and keyOutTangent fields which then would
> > > > > require sensible default values. Not sure.
> > > > >
> > > > > Some thoughts, Andreas
> > > > >
> > > > > PS:
> > > > > A StringSequencer would be useful as a mechanism to store and select
> > > > > alternative string values for display in Text nodes. I think the
> > > > > current strategy is to use an IntegerSequencer to select from Shapes
> > > > > in a Switch. A MFStringSequencer would be more concise and convenient.
> > > > > Hm, similarly there could be a StringTrigger event utility.
> > > > >
> > > > >
> > > > > On Tue, Jun 27, 2023 at 8:18 AM Michalis Kamburelis
> > > > > <michalis.kambi at gmail.com> wrote:
> > > > > >
> > > > > > Indeed glTF has additional interpolation options, and I think it would
> > > > > > make sense to add them to X3D.
> > > > > >
> > > > > > Looks like X3DOM and view3dscene/Castle Game Engine go in this
> > > > > > direction already :)
> > > > > >
> > > > > > 1. X3DOM has SFString "interpolation" field, default "LINEAR", see
> > > > > > https://doc.x3dom.org/author/Interpolation/X3DInterpolatorNode.html .
> > > > > > Andreas above explained how it supports "CUBICSPLINE" option.
> > > > > >
> > > > > > 2. view3dscene / Castle Game Engine allow to use "STEP" interpolation
> > > > > > explicitly.
> > > > > >
> > > > > >     For this, we add such field:
> > > > > >
> > > > > >     SFString [in,out]   interpolation  "LINEAR" # range: ["LINEAR"|"STEP"]
> > > > > >
> > > > > >     .. to all X3DInterpolatorNode descendants (so, all interpolator
> > > > > > nodes). This seems exactly like X3DOM, we just added "STEP", where
> > > > > > X3DOM added "CUBICSPLINE". In the end we should probably have all
> > > > > > ["LINEAR"|"STEP"|"CUBICSPLINE"].
> > > > > >
> > > > > > 3. Note that even without this field, with default linear
> > > > > > interpolation, it was possible to have effectively step interpolation,
> > > > > > as Andreas mentions. You can just duplicate keys in the middle, like
> > > > > >
> > > > > >     key [0 0.33 0.33 0.66 0.66]
> > > > > >     keyValue [10 10 20 20 30 30]
> > > > > >     # interpolation "LINEAR"
> > > > > >
> > > > > >     This is valid with X3D ("""keys shall be monotonically
> > > > > > non-decreasing""" according to spec).
> > > > > >
> > > > > >     It is equivalent to
> > > > > >
> > > > > >     key [0 0.33 0.66]
> > > > > >     keyValue [10 20 30]
> > > > > >     interpolation "STEP"
> > > > > >
> > > > > >     Using explicit "STEP" is slightly better, as you need 50% less
> > > > > > memory to store it, less computation (no need to do lerp or detect
> > > > > > this special case to optimize), and seems just simpler for authors --
> > > > > > it specifies intention explicitly, which makes sense, also if one
> > > > > > hopes a round-trip e.g. import to Blender and export to Blender.
> > > > > >
> > > > > >     Having such additional interpolation options seems common, glTF
> > > > > > and Blender too have them (
> > > > > > https://docs.blender.org/manual/en/latest/editors/graph_editor/fcurves/properties.html#editors-graph-fcurves-settings-interpolation
> > > > > > ).
> > > > > >
> > > > > > 4. Note that view3dscene / Castle Game Engine do not support
> > > > > > "CUBICSPLINE" interpolation yet, but we plan to.
> > > > > >
> > > > > >     For now we support curve interpolation by adding extra
> > > > > > intermediate keys to the linear interpolation -- so, we have
> > > > > > needlessly more keys / key values. But this is only a temporary
> > > > > > solution, we want to support "CUBICSPLINE" properly, like X3DOM.
> > > > > >
> > > > > > Regards,
> > > > > > Michalis
> > > > > >
> > > > > > niedz., 25 cze 2023 o 05:33 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> > > > > > >
> > > > > > > > Date: Sat, 24 Jun 2023 10:11:03 -0700
> > > > > > > > From: Joseph D Williams <joedwil at earthlink.net>
> > > > > > > > To: Michalis Kamburelis <michalis.kambi at gmail.com>
> > > > > > > > Cc: GPU Group <gpugroup at gmail.com>,  X3D Graphics public mailing list
> > > > > > > >         <x3d-public at web3d.org>
> > > > > > > > Subject: Re: [x3d-public] Blender > Exporting rig transforms to HAnim?
> > > > > > >
> > > > > > > > Which shows that yes, you have a list of the key times that just have to be ascending. Default is  linear interpolation between keyvalues. Now new to me, I like the STEP option where value not interpolated but stays the same until next keyvalue change. For that effect, gltf makes it easier. I want that STEP option in x3d! And then there is also the cubic spline with specified tangents. Me want that one too.
> > > > > > >
> > > > > > > In order to support glTF x3dom added a CUBICSPLINE with tangents
> > > > > > > option to interpolators, in the same form as glTF, eg. three values
> > > > > > > (in tangent, value, out tangent) per key. Parallel to glTF x3dom
> > > > > > > interpolators have a "interpolation" SFString field which can have
> > > > > > > "LINEAR" and "CUBICSPLINE" values. I suspect due to glTF support other
> > > > > > > viewers also already do something like that and could expose such a
> > > > > > > mode to regular X3D.
> > > > > > >
> > > > > > > A STEP option is already elegantly solved by X3D with duplicate keys
> > > > > > > with different values in series. That allows mixing step wise and
> > > > > > > linear interpolation in one animation. x3dom just translates glTF STEP
> > > > > > > to that format which is very straightforward. A dedicated STEP option
> > > > > > > would be 50% more efficient in storage but I am not sure if this is
> > > > > > > enough of a benefit to justify duplication of existing functionality.
> > > > > > > It may not or it may as x3dom already does it internally for glTF. It
> > > > > > > would be possible to add to x3d as well but perhaps not necessary.
> > > > > > >
> > > > > > > Andreas
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > x3d-public mailing list
> > > > > > > x3d-public at web3d.org
> > > > > > > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Andreas Plesch
> > > > > Waltham, MA 02453
> > > >
> > > >
> > > >
> > > > --
> > > > Andreas Plesch
> > > > Waltham, MA 02453
> >
> >
> >
> > --
> > Andreas Plesch
> > Waltham, MA 02453



-- 
Andreas Plesch
Waltham, MA 02453