[x3d-public] X3D meeting minutes 1 February 2018: strategy for mapping glTF 2.0 to X3Dv4 (Brutzman, Donald (Don) (CIV))

Andreas Plesch andreasplesch at gmail.com
Fri Feb 1 15:29:08 PST 2019


Thanks for the great summary of an accurate discussion around glTF 2.0.

The eye candy is all glTF 2.0 meshes and PBR materials. They are all
available from the Khronos glTF sample models repo, and were converted from
Collada. The Duck looks like the Nurbs duck but the others are likely
authored in one of the main authoring systems.

In addition, the X3D box is regular X3D, to show how both shapes coexist in
the same light (head light, lighting PBR material and X3D material).

The x3dom inline glTF support is focused on the black box model. But as
Michalis points out there is not that much difference between the black box
and the transcoded model since the black box often ultimately relies on
transcoding anyways,

In x3dom, PBRMaterial is already available as an exposed node and could
work with regular X3D geometries. But this is not tested. ( I should try
it. )

Similarly, BufferGeometry (replacing ExternalGeometry) is already available
as an exposed node for binary geometry and could work with regular X3D
appearance material. But this is also not tested. ( I should try it. )

ExternalShape is not updated to glTF 2, and likely will not.

glTF animations are just regular TimeSensor, Interpolator, ROUTE combos.
This works well. As earlier explained (have to dig up the list messages)
the Interpolator nodes are extended to allow binary key and keyValue
values, to be able to directly use glTF binary data. This is done via
BufferAccessor nodes, renamed from BufferGeometryAccessor because they are
now used for interpolators as well, not just Geometry (and could be used
for other binary data). Also, for easier implementation, 'linear',
'cubicspline', and 'step' modes were added to Interpolators in a field.

glTF animations are not formally exposed but can be accessed via the DEF
names of the involved nodes (which have to be discovered). So there is the
issue of how to define control over embedded animations, perhaps similar to
video media control.

glTF PBR materials should not have a huge performance impact these days
over regular X3D materials. It is all in the shader and texture (various
maps) use which is very efficient. It would be a component in a profile, no
?

The only item to add to the lighting discussion is that it is unclear if a
environment light with PBR should be combined with other light sources
since the PBR environment (cubemap) light implementations typically assume
that it is the only light to consider for energy balance as I understand
it. x3dom currently does not allow this combination but AFAIK most other
engines do not care and just add up contributions of all light sources.
Maybe that is a question to ask in the three.js forum.

Also, the difference in rendering of the same glTF model in different
engines is (mostly) due to use of different lighting. The idea is that the
glTF spec. is tight enough to get very similar renderings under the same
conditions. But there are enough differences in the implementations of PBR
to get slight variations, and pixel by pixel equivalence is not a goal, I
believe. For example, for environment lights, aka image based lights, a
BRDF lookup map is typically involved to determine contributions of viewing
angle and roughness to specular color.The map differs slightly between
shaders. Three.js does not use such a lookup map but a functional form that
it believes is better suited. But all other engines use a map. It is all in
the open.

I believe the exploding glTF scene could also work in X-ITE. It would be
interesting to find out and what adjustments may be necessary.

We should be able to converge on a naming convention for a TimeSensor
derived from a glTF. The x3dom names changed a bit, I am curious what the
castle names are.

oops that became long, thanks for reading,

Cheers,

-Andreas


> Date: Fri, 1 Feb 2019 17:55:08 +0000
> From: "Brutzman, Donald (Don) (CIV)" <brutzman at nps.edu>
> To: X3D Graphics public mailing list <x3d-public at web3d.org>
> Subject: [x3d-public] X3D meeting minutes 1 February 2018: strategy
>         for mapping glTF 2.0 to X3Dv4
> Message-ID: <30083b0e-7156-296c-ea2d-b5296f1ae702 at nps.edu>
> Content-Type: text/plain; charset="utf-8"
>
> Attendees:  John Carlson, Anita Havele, Michalis Kamburelis, Vince
Marchetti, Nicholas Polys, Dick Puk, Don Brutzman.
>
> Major progress today, building on recent meetings.  Group approach is
really productive.
>
> ----------------------------------------------------------------------
>
> 1. Wakeup eye candy and immediate deep dive: interesting discussion on
>
> On 2/1/2019 6:13 AM, Andreas Plesch wrote:
> > The exploding gltf's scene below works also pretty well on an older mid
range cell phone which I just tested. -Andreas
> >
> > On Wed, Jan 30, 2019 at 12:08 PM Andreas Plesch <andreasplesch at gmail.com
<mailto:andreasplesch at gmail.com>> wrote:
> >
> >     Since we are getting closer to x3dom merging in gltf2 inline
support in its dev. version, here a first example of multiple gltf inlines,
using an older x3dom scene:
> >
> >
https://rawcdn.githack.com/andreasplesch/x3dom/b34506419c8877884501143b122cbcbd287abb43/test/regression-suite/test/cases/routes/routes.html
>
> We think these models are essentially glTF meshes (though duck avocado
boom-box etc. might have originated in prior lifetimes as nurbs).
>
> Vince notes that there are two ways to integrating glTF into X3D.  First
is to treat glTF as "black box" of hidden data that simply displays all or
part of a glTF model, with the content of the node inaccessible to the rest
of the scene graph (no ROUTEs or inspection, etc.).  The other approach is
to convert/rewrite/transcode the glTF content as native X3D scene-graph
content.
>
> References:
>         X3DOM documentation and node index (somewhat out of date)
>         https://doc.x3dom.org/developer
>         https://doc.x3dom.org/author/nodes.html
>
>         ExternalShape and ExternalGeometry (from SRC era)
>         https://doc.x3dom.org/author/Shape/ExternalShape.html
>         https://doc.x3dom.org/author/Geometry3D/ExternalGeometry.html
>
>         BinaryGeometry
>         https://doc.x3dom.org/author/Geometry3D/BinaryGeometry.html
>
> More recently, the mappings are changing, actively merging dev webvr
branch into master, tracking on the github mailing list. Many moving parts,
no deeper links provided here today.
>
>         https://github.com/x3dom/x3dom
>
> Michalis elaborated that conversions of glTF to X3D can occur internally
in the player, X3DOM is using those nodes, and every player will likely
perform this conversion internally.  So the key question might be whether
we advance those nodes to X3Dv4.
>
> Nicholas observed that some things might be some things to expose, for
example transcoding to an X3D PBRMaterial.  There is a run-time namespace
issue, and a conversion correspondence issue.  This might be especially
important for animation.
>
> Michalis reported that he has accomplished similar types of tradeoffs
when importing .spine JSON format into Castle Game Engine.
>
>         http://esotericsoftware.com/spine-json-format
>
> ----------------
> Scroll down to an incredibly informative message from Andreas. 1st
example shows static model conversion,
> https://github.com/michaliskambi/x3d-tests/wiki/Binary-meshes
> - for glTF materials: PhysicalMaterial
> - for glTF mesh: BufferGeometry and BufferGeometryAccessor
>
> 2nd example has animations (of Transform), so it's just standard
PositionInterpolator, OrientationInterpolator etc., but instead of explicit
numbers in key="..." and keyValue="...", it refers to a binary data, using
again the BufferGeometryAccessor node inside.
>
> Note that we have TimeSensor with name "clockANI0" inside -- this is how
you would run glTF animation from the outside.
> ----------------
>
> John shared gltf JSON Schema (v4) link for the below:
>
https://github.com/KhronosGroup/glTF/blob/c02fd64e1fc15985d84cbbfacb536709f7c22e28/specification/2.0/schema/mesh.primitive.schema.json
>
> ----------------
>
> Confirming no surprises geometry constructs... Existing glTF meshes modes:
>
>
https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#primitivemode
> Allowed values:
> 0 POINTS
> 1 LINES
> 2 LINE_LOOP
> 3 LINE_STRIP
> 4 TRIANGLES
> 5 TRIANGLE_STRIP
> 6 TRIANGLE_FAN
>
> Continuing: a big issue with PBR material (and indeed meshes) is that a
scene author might certainly want to use those advanced materials again.
So this might be a strong motivator for having some kind of path to capture
PBR material within an X3D scene graph.  However mandating such a
correspondence could have a negative impact on performance.  So we don't
want to be heavy-handed about it.
>
> However we do need to provide expressibility of glTF 2 capabilities in
X3Dv4, making a variety of import and support capabilities possible.
>
> Dick notes that if there is insufficient definition, then variable
results might occur.
>
> Wondering if progress has advanced sufficiently to define what nodes are
needed in X3Dv4 to express glTF capabilities: mesh, material, animation,
humanoid skeletons.  X3D has much of that already.  The deltas are
significant and probably our "short set" of what is needed.
>
> ----------------------------------------------------------------------
>
> 2. Lighting.  Of note is that glTF does not define lighting sources, but
it does use lighting equations to define their characteristics.  The glTF
lighting equations appear as part of the glTF material definitions.
>
>  From Michalis: There are two extensions to glTF (outside of glTF
specification) to define light sources. One of them defines simple
point/directional light sources (largely compatible with X3D), one defines
environment light source.
>
> Materials in glTF:
>
https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#materials
>
> Lights in glTF: Links from
>
https://github.com/michaliskambi/x3d-tests/wiki/How-to-add-PBR-to-X3D%3F#what-about-lights
>
> -
https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Khronos/KHR_lights_punctual
> . It defines point, directional and spot lights, largely compatible with
X3D lights but has a different recommended attenuation law, and required
non-rendering/culling beyond the max. distance. The extension is supported
by the glTF blender exporter. There is no ambient light. (Thanks go to
Andreas Plesch for this information, posted mail on x3d-public.)
>
> -
https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based
> . This defines image-based lights. Yeah, basically some numbers and a
cubemap (which we can already express in X3D).
>
> It thus appears possible that we could formally extend X3D Lighting Model
to match these equations.
>
> Cross-check: often we see differences in how glTF players render the same
content.  Why?  Is that due to different lighting arrangements/locations by
different players, or a fundamental problem?
>
> Michalis thinks that differences many of us have observed might easily be
explained by different lighting approaches in model portrayal, not in
lighting equations per se.
>
> ----------------------------------------------------------------------
>
> 3. Inventory:  what do we need to add to to X3D v4 Specification to fully
support glTF capabilities?
>
> Here are links to X3D v3.3 specification, we would start modifying v4
specification in github.
>
> * 11 Rendering component for meshes, define mappings or directly
supporting glTF geometry buffers for high performance.  Look at Coordinate
node and lines.
>
> * 13 Geometry3D component includes IndexedFaceSet, deserves close
inspection/comparison.
>
> * 12 Shape component: likely add PhysicalMaterial (or upgrade
Material/TwoSidedMaterial), possibly ExternalShape.
>
> * 17 Lighting component: 17.2.2 Lighting model equations, existing
lighting nodes, possibly a prior-compatibility mode with earlier lighting
model.
>         (Michalis points out that new lighting nodes are likely not
needed)
>
> No immediate need yet seen for modifying animation, since X3D has a rich
animation model already.  Perhaps future work on geometry buffers, if glTF
starts doing that.
>
> * 26  Humanoid animation (HAnim) component: desirable to continue mapping
to HAnim, especially since X3D will soon support HAnim 2.0 and BVH-style
motion animation.
>
> No other work noted in the rest of the 41-total X3D components.
>
> All of the above X3D specification work seems do-able and driven by X3D
glTF implementation work in progress: X3DOM, X_ITE, Castle Game Engine.
Any others - FreeWrl perhaps?
>
> ----------------------------------------------------------------------
>
> 4. Next steps for X3Dv4, prioritized:
>
> a. X3Dv4 Specification (github)
> b. Examples: Andreas scene, glTF spec examples.
> c. Implementations: iterate until glTF-TO-X3D mappings, X3D specification
and examples are stable.
> d. Encoding and validation work will wait until any new nodes are stable.
>
> Open question: is it possible to accomplish for the X3D Working Group to
accomplish these tasks by Web3D 2019 and SIGGRAPH?
>
> ----------------------------------------------------------------------
>
> Wow.  So much work (years actually) coming to a focused point that
proposes exactly where X3Dv4 is going with respect to glTF.
>
> Summary: looks like we have a fully actionable strategy for mapping glTF
2.0 to X3Dv4.
>
> No doubt more discussion and analysis will continue for us to correctly
achieve these worthy goals.
>
> Thanks to many many people getting us to this point!  The work
continues...
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br
brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
 +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics
http://faculty.nps.edu/brutzman

-- 
Andreas Plesch
Waltham, MA 02453
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190201/26010bdb/attachment-0001.html>


More information about the x3d-public mailing list