[x3d-public] gltf inspired binary storage

[Joseph D Williams]

➢ . This is how in glTF they do something equivalent to
CoordinateInterpolar: define a number of "morph targets" and animate
between them. X3D CoordinateInterpolator is similar to a "morph
targets animation" with particular morph targets already premultiplied
with weights for the current animation.


Can we also compare this with Hanim Displacer operation (target displacements of mesh and weights driven by 0 to 1 scale input) ? 
Can the glTF change number of points in a shape during the course of an animation?
Thanks, Again, 
Joe


From: Andreas Plesch
Sent: Wednesday, January 19, 2022 3:33 PM
To: Michalis Kamburelis
Cc: X3D Graphics public mailing list
Subject: Re: [x3d-public] gltf inspired binary storage

Thanks, Michalis.

Yes, this tracks very closely the glTF specification. But there are probably tweaks that can be introduced to better integrate with x3d. 

I would expect some discussion around the current description. A first order question is that it may be more appropriate for the Interpolators to have new nodes such as BufferCoordinateInteprolator rather than an additional 'buffer' field. It was more convenient in x3dom to just add fields but it leads to competition between the key/keyValue fields and the BufferAccessor fields requiring clarifications.

Also, referring to nodes by index into an array of nodes tracks glTF but may be unusual for x3d.

In a way, the wiki description is a bit of initial spec. work but I am not really sure it is the best design.

For example, typically there is only one buffer file. Should therefore Buffer become a node rather than a field so there is a way to DEF/USE it ?

Thanks, Michalis, for your summary wiki. I honestly did not remember that I had put some thoughts into that before. Glad to see that the thinking stayed largely consistent.

Right, MorphTargets are the equivalent. The expectation is that all targets can be uploaded to the GPU and that then the interpolation can take place on the GPU, very fast. But it places a limit on the number targets/frames. x3dom currently does all on the CPU, with javascript. That is quite slow but fast enough for simpler models. I think for real GPU processing it would become necessary to migrate to Three.js, or Babylon.js, and write a x3d interpreter around those which is a massive task.

Best , Andreas

On Wed, Jan 19, 2022 at 3:58 PM Michalis Kamburelis <michalis.kambi at gmail.com> wrote:
Excellent, this is very valuable.

In general, I absolutely agree that in future X3D we should copy the
glTF approach of binary accessors+views. It's a flexible mechanism,
that maps nicely to GPU operations, and makes a huuuge speedup when
reading real-life models. (By "real-life models" I mean here "large 3D
models that are typically exported from 3D authoring software like
Blender", not coded by hand in a  text editor.)

I have a page on my wiki
https://github.com/michaliskambi/x3d-tests/wiki/Binary-meshes to
remind me of that :)

It's great that X3DOM takes a lead on this. Would you be willing to
work on PR to X3D specification to incorporate accessors+views in the
X3D standard? I would gladly help with review and implementing it on
my side (CGE/view3dscene) too. Because if you don't do it, then I'll
have to do this specification work sooner or later :) And from what I
see, X3DOM approach maps nicely to glTF, so it's just reasonable to
move X3DOM accessors+views idea to X3D specification.

P.S. Of note is that glTF actually has one more animation type (on top
of simple translation/rotation/scale that you mention), and this
animation type corresponds to X3D CoordinateInterpolator idea
more-or-less: glTF allows to define a number of "morph targets" (which
are equivalent to "Shape Keys" in Blender) and then to animate their
weights, using channel.path = weights in animation. See
https://www.khronos.org/registry/glTF/specs/2.0/glTF-2.0.html#animations
. This is how in glTF they do something equivalent to
CoordinateInterpolar: define a number of "morph targets" and animate
between them. X3D CoordinateInterpolator is similar to a "morph
targets animation" with particular morph targets already premultiplied
with weights for the current animation.

Regards,
Michalis

śr., 19 sty 2022 o 20:05 Brutzman, Donald (Don) (CIV)
<brutzman at nps.edu> napisał(a):
>
> Tweeted @Web3dConsortium with some nice screenshots.  Bravo!
>
>
>
> Improved #X3DOM binary storage of field values on GPU using (inspired by @glTF3D techniques) can efficiently store exceptionally large arrays such as #X3D CoordinateInterpolator, useful for large keyframe animations.
>
> http://web3d.org/pipermail/x3d-public_web3d.org/2022-January/016533.html
>
>
>
> https://twitter.com/Web3DConsortium/status/1483819271016824835
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
>
> X3D graphics, virtual worlds, Navy robotics https:// faculty.nps.edu/brutzman
>
>
>
> From: x3d-public <x3d-public-bounces at web3d.org> On Behalf Of Andreas Plesch
> Sent: Tuesday, January 18, 2022 7:40 PM
> To: X3D Graphics public mailing list <x3d-public at web3d.org>
> Subject: [x3d-public] gltf inspired binary storage
>
>
>
> Let me announce x3dom support for efficient support of large CoordinateInterpolator (and other) nodes by adopting gltf inspired binary storage of field values.
>
>
>
> Here is an example:
>
>
>
> regular CoordinateInterpolator (ca. 12MB):
>
>
>
> https://61e75f82d2db93000729e6a7--x3dom.netlify.app/examples/cases/coordinateInterpolator/coordinateInterpolator.html
>
>
>
> binary CoordinateInterpolator (ca. 5MB):
>
> https://61e75f82d2db93000729e6a7--x3dom.netlify.app/examples/cases/coordinateinterpolator/coordinateinterpolatorbuffer
>
>
>
> The binary version should load faster. There is no visual difference. There is also no performance difference after initial loading.
>
>
>
> This is useful for large keyframe animations.
>
>
>
> Background:
>
>
>
> glTF stores all data in buffers, eg. binary storage which can often be used directly on the GPU. The details on how the data is stored is provided by 'views' into the buffer and 'accessors' of views.
>
> x3dom implements glTF support not by translating these data structures into x3d field values but by implementing additional x3d nodes and fields. There is a 'BufferGeometry' node and additional fields such as BufferAccessor for Interpolators. This has the advantage that these additions can also be used outside of glTF.
>
> glTF only has support for translation, scale and rotation interpolation. In x3dom, buffer storage can now also be used for other interpolators. Here is a fuller description:
>
>
>
> https://github.com/x3dom/x3dom/wiki/Buffer-interpolators
>
>
>
> It is possible to convert regular Interpolators to buffer Interpolators. Here is a sample converter for CoordinateInterpolator:
>
>
>
> https://observablehq.com/@andreasplesch/x3dom-interpolator-buffer-converter
>
>
>
> In x3dom, there is also a BinaryGeometry node which is a precursor to the BufferGeometry node used for glTF support. Though non-standard, the BinaryGeometry node is very popular since it allows for faster and more efficient loading of large IFS. There is a converter (aopt) which generates BinaryGeometry. It should be possible to write another converter to BufferGeometry. This binary format is quite different from x3db, the Fast Infoset based binary encoding of x3d which requires heavy processing. With glTF support in x3d, there may be now more interest in glTF type binary storage of heavy models, perhaps leading to standardization.
>
>
>
> Any feedback or comment always welcome,
>
>
>
> Andreas
>
> --
>
> Andreas Plesch
> Waltham, MA 02453
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org



-- 
Andreas Plesch
Waltham, MA 02453

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220119/05be0fdc/attachment.html>