[x3d-public] How to implement glTF EXT_mesh_gpu_instancing?!

[Andreas Plesch]

Thanks, Holger. GPU instancing even with low end GPUs is extremely impressive.

Here is the teapot example with a rough implementation (and a proto for the new
fields):

https://andreasplesch.github.io/Library/Viewer/index.html?url=https://gist.githubusercontent.com/andreasplesch/1ee775f917593af42d0bb42a2c7f01ff/raw/8e166d8b12317c54ced6fa4e8da28eca2abc0abe/InstancedShapeProto.x3d

Checking how this could be supported, I ended up with another
potential path since webGL drawInstanced is already used for VR mode
in x3dom (one instance for each eye).

Instead of having the GPU iterate (internally and very fast) over the
same primitive with sequential (transformation) attributes, I tried to
just send a lot of geometry to the GPU by duplicating webgl buffer
contents in (much) larger buffers. Rendering happens in a single webgl
call which is critical for rendering performance, eg. effectively in a
single Shape with all instances included.

It takes a little because the buffers have to be duplicated and
transformed many times but then the 3000 teapots render fine. Another
disadvantage is that I hit javascript Array limitations with 10000
teapots. Rendering may be a little faster compared to drawInstanced
since the shader has to do a little less (no per instance
transformations at each frame).

Since most x3d geometries get translated to similar webgl buffers,
this path is fairly generic as well and would work for most shapes.

Node design: I made it a wrapping node which expects Shape children
but only for Proto convenience. Since GPU drawInstanced methods apply
per Shape, the shape level for new fields seems most appropriate.
Since Shape is a special case of InstancedShape may one of those is
not strictly necessary.

Since apparently not too many glTF renderers support the extension,
maybe there are not that many use cases. Perhaps most users prefer to
just actually generate many individual instances physically with
authoring in a single drawing entity.

Cheers, Andreas

On Sun, Apr 21, 2024 at 8:41 AM <x3d-public-request at web3d.org> wrote:
> Date: Sun, 21 Apr 2024 13:35:49 +0200
> From: Holger Seelig <holger.seelig at yahoo.de>
> To: X3D <x3d-public at web3d.org>
> Subject: [x3d-public] How to implement glTF EXT_mesh_gpu_instancing?!
> Message-ID: <5386E3E7-2A85-46F6-825B-1013528CEAA6 at yahoo.de>
> Content-Type: text/plain; charset="us-ascii"
>
> How to implement glTF EXT_mesh_gpu_instancing?!
>
> When looking through the glTF sample models, sooner or later you will stumble across an example and this extension.
>
> https://github.com/KhronosGroup/glTF/tree/main/extensions/2.0/Vendor/EXT_mesh_gpu_instancing
> https://github.com/KhronosGroup/glTF-Sample-Models/tree/main/2.0/SimpleInstancing
>
> The simplest idea to implement this, which I did, was to generate a lot of Transform nodes (one for each instance) and clone the shapes or geometry nodes. But this runs counter to the name of the extension (GPU). Especially if you create thousands of transform nodes for a shape with a geometry with only a few polygons. The extension is made to clone a lot of objects, e.g. a leaf on a tree, which then reappears in many places on the tree, only with different transformations, and everything should be calculated in one pass by the GPU.
>
> The second idea I had was to somehow incorporate the transformations (translation, rotation, scale) for each instance into the X3DGeometryNode nodes. However, this would have meant changing all of these nodes. So I discarded this for the time being and postponed it.
>
> Then I came up with the idea of why add it to the geometry nodes when you could invent a new shape node. So the idea of "InstancedShape" grew:
>
> Documentation of InstancedShape in X_ITE with example at the bottom of the page (2500 instances, 2,160,040 triangles):
> https://create3000.github.io/x_ite/components/x-ite/instancedshape/
>
> InstancedShape : X3DShapeNode {
>    # ... inherited fields
>    MFVec3f [in,out] translations [] #
>    MFRotation [in,out] rotations [] # list of rotations, one for each instance
>    MFVec3f [in,out] scales [] # list of scales, one for each instance
>    # List of translations, rotations, scales; one for each instance.
>    # If only one value is provided, it is provided to each instance equivalently.
> }
>
> The advantage is that the geometry nodes do not have to be changed, and the same functions could be used to implement these nodes that were already available for ParticleSystem nodes with some trivial additions.
>
> Examples to see InstancedShape node in action, all files are in glTF or GLB format converted to X3D:
>
> Simple Instancing (125 instances, 1,500 triangles):
> https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/KhronosGroup/glTF-Sample-Models/master/2.0/SimpleInstancing/glTF/SimpleInstancing.gltf
>
> Teapots Galore (958 instances, 7,725,314 triangles):
> https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/KhronosGroup/glTF/main/extensions/2.0/Vendor/EXT_mesh_gpu_instancing/samples/teapots_galore/teapots_galore.gltf
>
> Fur Instancing (10,000 instances, 3,200,000 triangles):
> https://create3000.github.io/x_ite/playground/?url=https://create3000.github.io/media/glTF/FurInstancing/FurInstancing.glb
>
> 3D Tiles (10 instances, 120 triangles):
> https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/CesiumGS/3d-tiles-samples/main/glTF/GpuInstancesMetadata/GpuInstancesMetadata.gltf
>
> Forest (100 instances, 21,400 triangles):
> https://create3000.github.io/x_ite/playground/?url=https://raw.githubusercontent.com/bertt/cesium_3dtiles_samples/master/samples/1.1/forest/forest.glb
>
> How do you do that, do you? Maybe we could consider this for a future X3D version.
>
> Make X3D even better!
> Best regards,
> Holger
>
> --
> Holger Seelig
> Leipzig, Germany
>
> holger.seelig at yahoo.de
> https://create3000.github.io/x_ite/
>

-- 
Andreas Plesch
Waltham, MA 02453