[x3d-public] Render optimizations

Tue Aug 27 08:47:17 PDT 2024

Hi Holger,

very cool ! Let's think about a comparative performance test, say a
(few?) thousand, individual spheres (or chess sets) in a Group and in
a StaticGroup, in two scenes, one supposed to bog down the browser and
the other one fast.

Here is a first attempt with 64 chess sets, > 3000 Shapes and > 90 M
primitives (triangles):

https://create3000.github.io/x_ite/playground/?url=https://gist.githubusercontent.com/andreasplesch/4ca3ebe9a51ebc8d28515b5e13229d54/raw/3302b0fc69165d446546bbdcb43a61411e78aba0/StaticGroupStressTest.x3d

It renders but StaticGroup compilation induces Chrome to give up on my
system. Probably not the best test.

Cheers, -Andreas

On Tue, Aug 27, 2024 at 6:19 AM Holger Seelig <holger.seelig at yahoo.de> wrote:
>
> With the current version of X_ITE (v10.4.1) I have now implemented exactly what was suggested here by Andreas. A StaticGroup node will now completely remove all transformations and bake them into the geometry. Furthermore, assuming the Appearance node is exactly the same (cloned) and the geometry nodes are compatible, the geometry nodes will be merged into a single one.
>
> This works very well with all geometry types (point, line, 2D, 3D).
>
> A basic test file is also available. This test file will create 10 static nodes from the previous 22 nodes:
> https://create3000.github.io/x_ite/playground/?url=https://create3000.github.io/Library/Tests/Components/Grouping/StaticGroup.x3d
>
> Another test file using an Inline node with a glTF sample (1.5 million triangles). This test file will create 15 static nodes from the previous 49 nodes:
> https://create3000.github.io/x_ite/playground/?url=https://create3000.github.io/Library/Tests/Components/Grouping/StaticGroupWithInline.x3d
>
> Best regards,
> Holger
>
> --
> Holger Seelig
> Leipzig, Germany
>
> holger.seelig at yahoo.de
> https://create3000.github.io/x_ite/
>
> Am 16.08.2024 um 17:59 schrieb Holger Seelig <holger.seelig at yahoo.de>:
>
> Thank you Andreas for this idea. It is an interesting possibility for optimizing geometries. There is nothing like this built into X_ITE yet, but there are already optimizations for StaticGroup and if I were to build something like this (geometry merge), it would be in StaticGroup, as it seems very easy to implement there at first glance.
>
> Best regards,
> Holger
>
> --
> Holger Seelig
> Leipzig, Germany
>
> holger.seelig at yahoo.de
> https://create3000.github.io/x_ite/
>
> Am 15.08.2024 um 16:05 schrieb Michalis Kamburelis via x3d-public <x3d-public at web3d.org>:
>
> Castle Game Engine can be told to do this at run-time, it's called
> DynamicBatching.
>
> It's just a checkbox really, see
> https://castle-engine.io/apidoc/html/CastleViewport.TCastleViewport.html#DynamicBatching
> . In Castle Model Viewer you can activate it with "View -> Dynamic
> Batching".
>
> Doing it at run-time has naturally benefits and drawbacks:
>
> - The benefit of doing it at run-time is that we can "glue" any
> shapes, not necessarily within StaticGroup.
>
>    We even work "cross-scene", that is in Castle Game Engine you
> typically display multiple models (X3D, glTF, etc.) in your viewport
> and we can "glue" into one shapes from various scenes. See e.g. news
> on https://castle-engine.io/wp/2023/06/30/big-renderer-improvements-correct-and-automatic-blending-sorting-more-powerful-batching-now-cross-scene-easier-and-more-reliable-occlusion-culling-and-occlusion-sorting/
> when we introduced "cross-scene" batching.
>
>    Doing it at run-time also means we can do it after frustum culling
> has eliminated shapes outside of frustum. So by "gluing many shapes
> into one" we don't negate the benefits of frustum culling.
>
> - The drawback is that effectively DynamicBatching does some
> additional comparisons and processing at run-time. This consumes time,
> and can (in theory) defeat the gains of batching. But it practice it
> (almost) never happens -- the comparisons are rather fast.
>
> Sometimes it's a huge huge gain. As you say, in some models there's a
> big opportunity to effectively have 1 draw call instead of thousands
> -- which translates to big performance gains. And sometimes it's zero
> gain. I haven't found a case when it's "negative gain", that is when
> the extra work at run-time done comparing / merging actually
> outweights the benefits, but I'm sure it exists (I can construct an
> artificial example of this in my head, but does it happen in real-life
> usage?).
>
> All in all, this is a significant feature with some maintenance costs,
> but it is worth it (on *some* cases).
>
> We plan to introduce also StaticBatching to do it at load time. For
> this we plan to invent a way to tell "this model in completely static"
> -- we don't want to rely on X3D StaticGroup nodes, because
>
> - Many X3D models don't use them (even when it would make sense). It
> requires authoring tools support also (e.g. Blender -> X3D exporter to
> honor some "static" setting from Blender).
>
> - And other 3D formats (like glTF) don't have a StaticGroup equivalent.
>
> So we plan to instead have a "Static" checkbox at TCastleScene
> component ( https://castle-engine.io/viewport_and_scenes ).
>
> The implementation contains hardcoded a number of rules / comparisons
> (to compare and merge things that are important visually, and ignore
> everything else). See
> https://github.com/castle-engine/castle-engine/blob/master/src/scene/castleinternalbatchshapes.pas
> . It's of course not perfect, right now it doesn't merge *everything*
> possible. It is coded carefully, i.e. we're not sure can 2 shapes be
> merged -> we don't merge (possibly losing performance, but not risking
> rendering bugs).
>
> To compare, Unity also has both static and dynamic batching. They
> probably don't use X3D underneath :), but still the concept underneath
> is likely similar to what CGE is doing and to what you describe. They
> also have a checkbox "Static" at GameObjects, to help with this.
>
> Regards,
> Michalis
>
> wt., 13 sie 2024 o 23:12 Andreas Plesch via x3d-public
> <x3d-public at web3d.org> napisał(a):
>
>
> I think there may be an opportunity for some relatively simple tools
> to achieve meaningful rendering improvements using the declarative
> nature of X3D.
>
> The idea is that it is much more efficient for (at least for GL based)
> rendering to send a single draw call with a large object compared to
> many draw calls with smaller objects. In effect, if it is possible to
> combine multiple Shapes into a single larger Shape, it can have huge
> effects on rendering speeds. For example, it is possible to easily
> render hundreds of thousands of points in a single cloud while it may
> be nearly impossible to render hundreds of thousands of Shapes with a
> few points each.
>
> StaticGroup is designed to help a browser with such optimizations and
> I was wondering if there are existing tools which already do this,
> perhaps in a preprocessing step.
>
> The simplest, while useful tool I can think of is this:
>
> Inside a StaticGroup, it should be possible to identify Shapes which
> use the same Appearance (by DEF/USE). We group these Shapes by the
> drawing primitive they would use (points, lines, triangles). Combining
> the geometries into a single geometry would be most involved but only
> considering IndexedFace and TriangleSets should be already useful.
> Transforms and TextureTransforms would need to be flattened but this
> can be neatly separated. Finally, a single Shape with the shared
> Appearance and the combined geometry can replace multiple Shapes.
>
> The underlying reason for such a tool is that (machine and human)
> generators and editors tend to produce many Shapes because it is a bit
> harder to keep track of them and organize them into single, larger
> objects.
>
> So any thoughts or pointers to existing X3D optimization tools will be
> very welcome,
>
> Andreas
>
> --
> Andreas Plesch
> Waltham, MA 02453
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
>
>

--
Andreas Plesch
Waltham, MA 02453