[x3d-public] gltf and x3d

[Michalis Kamburelis]

2017-12-08 21:04 GMT+01:00 Andreas Plesch <andreasplesch at gmail.com>:
> x3dom currently supports gltf 1.0 content with an ExternalGeometry and
> an ExternalShape node. The ExternalGeometry node picks out just a
> named geometry from a gltf scene, and the ExternalShape either a
> single named node with geometry and material, or the complete scene,
> as a group.
>
> Lights,  cameras (viewpoints), animations, skinning, morphing are all ignored.
>
> gltf 2.0 is aggressively pushed to phase out gltf 1.0 because it has a
> better design and has required material support in the form of
> PBR-metallic-roughness.  A lot of content is already available in gltf
> 2.0 format. So the question becomes if x3d should start to
> interoperate with gltf directly or just in the form of translators.
>
> The main benefit to support gltf for x3dom was to gain fast, efficient
> loading of large meshes in a binary format (binary gltf or .glb is
> very similar to src). It is possible to use blender for just geometry
> modelling, and then to export as gltf 2.0. There are other and will be
> more gltf exporting modellers as well. So this is still a valuable
> goal and ExternalGeometry may persist.
>
> If this is the case, we may start to ask if and how gltf geometry
> support should be formalized in the standard. Presumably, gltf would
> have to become an ISO standard first ?
>
> It seems also nice to be able to place fully textured gltf models into
> a x3d scenegraph (say a car) and then use x3d nodes to interact or
> animate. But such ExternalShape integration requires quite a lot of
> functionality. The x3d browser will have to know how to shade PBR
> materials and how to use texture maps for all related parameters
> similar to common surface shader nodes in view3dscene. Presumably
> there will have to be then also consideration of how x3d lights work
> with both, PBR and X3D materials. Since PBR materials would have to
> able to be rendered, PBR materials may also be exposed outside of gltf
> ExternalShape content, with a PhysicalMaterial node. However, in
> x3dom, I think, it would be easier at first to restrict PBR to just
> gltf content since it would use dedicated shader code anyways.
>
> For ExternalShape use cases, a translator from gltf to x3d with
> ExternalGeometry and PhysicalMaterial nodes may be also be a solution.
>
> I think embedded keyframe animations would need a translator to
> Interpolators rather than direct support.
>
> I do not know enough about skinning and morphing and its relation to
> HAnim to know if translation would be possible. One (key) aspect
> aspect may be that morphing seems to be designed as deformation in the
> shader. Not sure how collisions are then typically handled.
>
> So thinking through this a bit, it may already be useful to start
> supporting ExternalGeometry for gltf 2.0 in x3dom which may be
> feasible.
>
> Any feedback, comments, questions would be very welcome,
>

For what it's worth, I very definitely plan to add support for glTF 2
to Castle Game Engine / view3dscene in 2018:) I wrote some words about
it on https://castle-engine.sourceforge.io/planned_features.php .

It will be supported just like other model formats (Collada, Wavefront
OBJ, Spine JSON...):

- You will be able to use the "Inline" node to include the glTF file
inside a larger X3D scene.

- Or "Anchor" to jump to glTF from X3D.

- Or you can "just" load glTF model (in view3dscene, or in any engine
game, using our TCastleScene Object Pascal class). So there is no need
to wrap glTF in X3D, if you don't need it, you can just straight open
glTF.

The way it works: At loading we internally convert everything to a
graph of X3D nodes. So, as a side-effect, you will also get a
converter glTF -> X3D (usable explicitly e.g. through view3dscene
"Save As ..." menu item, or "tovrmlx3d" tool). See
https://castle-engine.sourceforge.io/x3d_extensions.php#section_ext_inline_for_all
for a short description how it already works for other formats. This
approach is used by *all* 3D and 2D model formats supported by Castle
Game Engine / view3dscene (
https://castle-engine.sourceforge.io/view3dscene.php#section_features
), internally everything is a graph of X3D nodes. This approach works
very nicely :), so I know that I will very much want to use the same
approach for glTF.

I like glTF a lot, Khronos does a good job designing a 3D format, so
eventually I want to support all the glTF features:

- multiple meshes,
- delivering mesh data efficiently, that is: I want to read from glTF
binary data and copy it straight to OpenGL VBO, without
"deconstructing" all the data into separate Coordinate,
TextureCoordinate, Color, XxxAttribute (for shaders) nodes,
- PBR materials,
- animations.

I realize that some of this will require adding new nodes to X3D, so I
will just do it (or use existing extensions, where possible). E.g. I
will almost definitely implement PhysicalMaterial that you referenced
lately (thanks for that!). So, as another side effect, one will also
be able to use PBR from X3D.

I have not yet done a research about everything. I welcome all
information how to best (without losing any functionality or
efficiency) convert all important glTF concepts to X3D :)

Regards,
Michalis