[x3d-public] X3D meeting minutes 1 February 2018: strategy for mapping glTF 2.0 to X3Dv4 (Brutzman, Donald (Don) (CIV))
Andreas Plesch
andreasplesch at gmail.com
Sat Feb 2 08:21:48 PST 2019
Hi Michalis,
just using the animation name makes sense. Autoexporting and then
IMPORT AS can resolve name collisions. Unfortunately, x3dom does not
have EXPORT/IMPORT. It relies on DOM access to inlined nodes via id
attributes which are given a prefix defined as an Inline field
(namespacename). namespacename could also be used as a DEF prefix.
I wonder if there could be mechanism to discover what is exported
without knowing in advance EXPORT names. Inlines self-advertising
their exported content.
A few edges cases are:
https://github.com/KhronosGroup/glTF/blob/master/specification/2.0/README.md#reference-animation
says that a name is not required. What name should be used then ?
The name can contain any character including spaces which are not
allowed in DEF names. How to sanitize ?
Animation of gltf skins will have to wait a while. I have an idea on
how to go about this, probably by first moving HAnim skinning to the
shader. There is a prototype but it requires more work, in particular
using float textures instead of uniform arrays to upload the joint,
weight data per vertex. Then, displacers also should move to the
shader. This will match pretty closely what glTF provides, so it will
be probably tempting to have new helper X3D nodes to map directly to
glTF, instead of transcoding to HAnim although this remains to be
seen.
I suppose for blending animations, eg. mixing interpolator output by
weight, there are additional processing steps. Do you use an actual
script or just internal computing ?
Blending may be common enough to justify a new Mixer node. Something like:
Mixer : X3DChildNode {
SFFloat [in] set_fraction [0,1]
SF<type> [in,out] mix1Value
SF<type> [in,out] mix2Value
SFNode [in,out] metadata NULL [X3DMetadataObject]
SF<type> [out] value_changed
}
value_changed = set_fraction * mix1Value + (1 - set_fraction) * mix2Value
The mix values would be typically output from interpolators of the
same kind, and fraction the blending strength.
-Andreas
On Fri, Feb 1, 2019 at 8:18 PM Michalis Kamburelis
<michalis.kambi at gmail.com> wrote:
>
> Thanks for the comments! Lots of good information, very informative.
>
> To address the question at the end: How does Castle Game Engine name
> the TimeSensor nodes created by converting animations from other
> formats (like Spine JSON or glTF) to X3D?
>
> The answer is that I don't rename them, at all. When an animation is
> called "Walk" in the source model, I create a "DEF Walk TimeSensor {
> ... }" in X3D graph, and "EXPORT Walk" to make it visible to the
> outside X3D scene.
>
> Rationale: For a short time I was adding a prefix ("Animation_Walk"),
> but it seemed that adding (and then removing) this prefix at various
> places in the code is an unnecessary complication (you need to
> remember if you have prefixed or unprefixed animation name). It is
> much easier to just tell engine users "the animation name == the same
> thing as TimeSensor X3D node name == the same name you set in your
> content-creation software == the same thing you see in view3dscene
> Animation menu".
>
> It also means that I can use the same API for playing animations from
> X3D files, that use any names for TimeSensor. X3D authors have a
> simple message: "your TimeSensor name is your animation name".
>
> <lots of background information below, sorry>
>
> Disclaimer: This is how I handle animations converted from Spine JSON,
> which is exactly how I plan to convert animations from glTF 2.
> CGE does *not yet* support glTF 2 animations, but I plan to support
> them in the same way as described above, in February. Only animations
> of transformations (for the beginning), not skinned animations.
>
> Spine JSON defines a 2D model with skeletal animation, this model has
> a set of animations, and it is up to the player to actually run a
> chosen animation by name, with looping or not. So it's similar to glTF
> in this regard.
>
> In Castle Game Engine, the primary Pascal API call for this is
> TCastleScene.PlayAnimation
> https://castle-engine.io/manual_scene.php#section_play_animation . In
> it's basic version, it just takes an animation name (TimeSensor name)
> and a Boolean specifying is it looping. As a bonus, it can also
> specify to play animation backwards, do animation blending (cross-fade
> from previous animation), play animation from the middle etc.
> Internally, TCastleScene.PlayAnimation just sets TimeSensor fields to
> achieve the desired effect.
>
> </lots of background information, sorry>
>
> Regards,
> Michalis
>
> sob., 2 lut 2019 o 00:30 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> >
> > Thanks for the great summary of an accurate discussion around glTF 2.0.
> >
> > The eye candy is all glTF 2.0 meshes and PBR materials. They are all available from the Khronos glTF sample models repo, and were converted from Collada. The Duck looks like the Nurbs duck but the others are likely authored in one of the main authoring systems.
> >
> > In addition, the X3D box is regular X3D, to show how both shapes coexist in the same light (head light, lighting PBR material and X3D material).
> >
> > The x3dom inline glTF support is focused on the black box model. But as Michalis points out there is not that much difference between the black box and the transcoded model since the black box often ultimately relies on transcoding anyways,
> >
> > In x3dom, PBRMaterial is already available as an exposed node and could work with regular X3D geometries. But this is not tested. ( I should try it. )
> >
> > Similarly, BufferGeometry (replacing ExternalGeometry) is already available as an exposed node for binary geometry and could work with regular X3D appearance material. But this is also not tested. ( I should try it. )
> >
> > ExternalShape is not updated to glTF 2, and likely will not.
> >
> > glTF animations are just regular TimeSensor, Interpolator, ROUTE combos. This works well. As earlier explained (have to dig up the list messages) the Interpolator nodes are extended to allow binary key and keyValue values, to be able to directly use glTF binary data. This is done via BufferAccessor nodes, renamed from BufferGeometryAccessor because they are now used for interpolators as well, not just Geometry (and could be used for other binary data). Also, for easier implementation, 'linear', 'cubicspline', and 'step' modes were added to Interpolators in a field.
> >
> > glTF animations are not formally exposed but can be accessed via the DEF names of the involved nodes (which have to be discovered). So there is the issue of how to define control over embedded animations, perhaps similar to video media control.
> >
> > glTF PBR materials should not have a huge performance impact these days over regular X3D materials. It is all in the shader and texture (various maps) use which is very efficient. It would be a component in a profile, no ?
> >
> > The only item to add to the lighting discussion is that it is unclear if a environment light with PBR should be combined with other light sources since the PBR environment (cubemap) light implementations typically assume that it is the only light to consider for energy balance as I understand it. x3dom currently does not allow this combination but AFAIK most other engines do not care and just add up contributions of all light sources. Maybe that is a question to ask in the three.js forum.
> >
> > Also, the difference in rendering of the same glTF model in different engines is (mostly) due to use of different lighting. The idea is that the glTF spec. is tight enough to get very similar renderings under the same conditions. But there are enough differences in the implementations of PBR to get slight variations, and pixel by pixel equivalence is not a goal, I believe. For example, for environment lights, aka image based lights, a BRDF lookup map is typically involved to determine contributions of viewing angle and roughness to specular color.The map differs slightly between shaders. Three.js does not use such a lookup map but a functional form that it believes is better suited. But all other engines use a map. It is all in the open.
> >
> > I believe the exploding glTF scene could also work in X-ITE. It would be interesting to find out and what adjustments may be necessary.
> >
> > We should be able to converge on a naming convention for a TimeSensor derived from a glTF. The x3dom names changed a bit, I am curious what the castle names are.
> >
> > oops that became long, thanks for reading,
> >
> > Cheers,
> >
> > -Andreas
> >
> >
> > > Date: Fri, 1 Feb 2019 17:55:08 +0000
> > > From: "Brutzman, Donald (Don) (CIV)" <brutzman at nps.edu>
> > > To: X3D Graphics public mailing list <x3d-public at web3d.org>
> > > Subject: [x3d-public] X3D meeting minutes 1 February 2018: strategy
> > > for mapping glTF 2.0 to X3Dv4
> > > Message-ID: <30083b0e-7156-296c-ea2d-b5296f1ae702 at nps.edu>
> > > Content-Type: text/plain; charset="utf-8"
> > >
> > > Attendees: John Carlson, Anita Havele, Michalis Kamburelis, Vince Marchetti, Nicholas Polys, Dick Puk, Don Brutzman.
> > >
> > > Major progress today, building on recent meetings. Group approach is really productive.
> > >
> > > ----------------------------------------------------------------------
> > >
> > > 1. Wakeup eye candy and immediate deep dive: interesting discussion on
> > >
> > > On 2/1/2019 6:13 AM, Andreas Plesch wrote:
> > > > The exploding gltf's scene below works also pretty well on an older mid range cell phone which I just tested. -Andreas
> > > >
> > > > On Wed, Jan 30, 2019 at 12:08 PM Andreas Plesch <andreasplesch at gmail.com <mailto:andreasplesch at gmail.com>> wrote:
> > > >
> > > > Since we are getting closer to x3dom merging in gltf2 inline support in its dev. version, here a first example of multiple gltf inlines, using an older x3dom scene:
> > > >
> > > > https://rawcdn.githack.com/andreasplesch/x3dom/b34506419c8877884501143b122cbcbd287abb43/test/regression-suite/test/cases/routes/routes.html
> > >
> > > We think these models are essentially glTF meshes (though duck avocado boom-box etc. might have originated in prior lifetimes as nurbs).
> > >
> > > Vince notes that there are two ways to integrating glTF into X3D. First is to treat glTF as "black box" of hidden data that simply displays all or part of a glTF model, with the content of the node inaccessible to the rest of the scene graph (no ROUTEs or inspection, etc.). The other approach is to convert/rewrite/transcode the glTF content as native X3D scene-graph content.
> > >
> > > References:
> > > X3DOM documentation and node index (somewhat out of date)
> > > https://doc.x3dom.org/developer
> > > https://doc.x3dom.org/author/nodes.html
> > >
> > > ExternalShape and ExternalGeometry (from SRC era)
> > > https://doc.x3dom.org/author/Shape/ExternalShape.html
> > > https://doc.x3dom.org/author/Geometry3D/ExternalGeometry.html
> > >
> > > BinaryGeometry
> > > https://doc.x3dom.org/author/Geometry3D/BinaryGeometry.html
> > >
> > > More recently, the mappings are changing, actively merging dev webvr branch into master, tracking on the github mailing list. Many moving parts, no deeper links provided here today.
> > >
> > > https://github.com/x3dom/x3dom
> > >
> > > Michalis elaborated that conversions of glTF to X3D can occur internally in the player, X3DOM is using those nodes, and every player will likely perform this conversion internally. So the key question might be whether we advance those nodes to X3Dv4.
> > >
> > > Nicholas observed that some things might be some things to expose, for example transcoding to an X3D PBRMaterial. There is a run-time namespace issue, and a conversion correspondence issue. This might be especially important for animation.
> > >
> > > Michalis reported that he has accomplished similar types of tradeoffs when importing .spine JSON format into Castle Game Engine.
> > >
> > > http://esotericsoftware.com/spine-json-format
> > >
> > > ----------------
> > > Scroll down to an incredibly informative message from Andreas. 1st example shows static model conversion,
> > > https://github.com/michaliskambi/x3d-tests/wiki/Binary-meshes
> > > - for glTF materials: PhysicalMaterial
> > > - for glTF mesh: BufferGeometry and BufferGeometryAccessor
> > >
> > > 2nd example has animations (of Transform), so it's just standard PositionInterpolator, OrientationInterpolator etc., but instead of explicit numbers in key="..." and keyValue="...", it refers to a binary data, using again the BufferGeometryAccessor node inside.
> > >
> > > Note that we have TimeSensor with name "clockANI0" inside -- this is how you would run glTF animation from the outside.
> > > ----------------
> > >
> > > John shared gltf JSON Schema (v4) link for the below:
> > > https://github.com/KhronosGroup/glTF/blob/c02fd64e1fc15985d84cbbfacb536709f7c22e28/specification/2.0/schema/mesh.primitive.schema.json
> > >
> > > ----------------
> > >
> > > Confirming no surprises geometry constructs... Existing glTF meshes modes:
> > >
> > > https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#primitivemode
> > > Allowed values:
> > > 0 POINTS
> > > 1 LINES
> > > 2 LINE_LOOP
> > > 3 LINE_STRIP
> > > 4 TRIANGLES
> > > 5 TRIANGLE_STRIP
> > > 6 TRIANGLE_FAN
> > >
> > > Continuing: a big issue with PBR material (and indeed meshes) is that a scene author might certainly want to use those advanced materials again. So this might be a strong motivator for having some kind of path to capture PBR material within an X3D scene graph. However mandating such a correspondence could have a negative impact on performance. So we don't want to be heavy-handed about it.
> > >
> > > However we do need to provide expressibility of glTF 2 capabilities in X3Dv4, making a variety of import and support capabilities possible.
> > >
> > > Dick notes that if there is insufficient definition, then variable results might occur.
> > >
> > > Wondering if progress has advanced sufficiently to define what nodes are needed in X3Dv4 to express glTF capabilities: mesh, material, animation, humanoid skeletons. X3D has much of that already. The deltas are significant and probably our "short set" of what is needed.
> > >
> > > ----------------------------------------------------------------------
> > >
> > > 2. Lighting. Of note is that glTF does not define lighting sources, but it does use lighting equations to define their characteristics. The glTF lighting equations appear as part of the glTF material definitions.
> > >
> > > From Michalis: There are two extensions to glTF (outside of glTF specification) to define light sources. One of them defines simple point/directional light sources (largely compatible with X3D), one defines environment light source.
> > >
> > > Materials in glTF:
> > > https://github.com/KhronosGroup/glTF/tree/master/specification/2.0#materials
> > >
> > > Lights in glTF: Links from
> > > https://github.com/michaliskambi/x3d-tests/wiki/How-to-add-PBR-to-X3D%3F#what-about-lights
> > >
> > > - https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Khronos/KHR_lights_punctual
> > > . It defines point, directional and spot lights, largely compatible with X3D lights but has a different recommended attenuation law, and required non-rendering/culling beyond the max. distance. The extension is supported by the glTF blender exporter. There is no ambient light. (Thanks go to Andreas Plesch for this information, posted mail on x3d-public.)
> > >
> > > - https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based
> > > . This defines image-based lights. Yeah, basically some numbers and a cubemap (which we can already express in X3D).
> > >
> > > It thus appears possible that we could formally extend X3D Lighting Model to match these equations.
> > >
> > > Cross-check: often we see differences in how glTF players render the same content. Why? Is that due to different lighting arrangements/locations by different players, or a fundamental problem?
> > >
> > > Michalis thinks that differences many of us have observed might easily be explained by different lighting approaches in model portrayal, not in lighting equations per se.
> > >
> > > ----------------------------------------------------------------------
> > >
> > > 3. Inventory: what do we need to add to to X3D v4 Specification to fully support glTF capabilities?
> > >
> > > Here are links to X3D v3.3 specification, we would start modifying v4 specification in github.
> > >
> > > * 11 Rendering component for meshes, define mappings or directly supporting glTF geometry buffers for high performance. Look at Coordinate node and lines.
> > >
> > > * 13 Geometry3D component includes IndexedFaceSet, deserves close inspection/comparison.
> > >
> > > * 12 Shape component: likely add PhysicalMaterial (or upgrade Material/TwoSidedMaterial), possibly ExternalShape.
> > >
> > > * 17 Lighting component: 17.2.2 Lighting model equations, existing lighting nodes, possibly a prior-compatibility mode with earlier lighting model.
> > > (Michalis points out that new lighting nodes are likely not needed)
> > >
> > > No immediate need yet seen for modifying animation, since X3D has a rich animation model already. Perhaps future work on geometry buffers, if glTF starts doing that.
> > >
> > > * 26 Humanoid animation (HAnim) component: desirable to continue mapping to HAnim, especially since X3D will soon support HAnim 2.0 and BVH-style motion animation.
> > >
> > > No other work noted in the rest of the 41-total X3D components.
> > >
> > > All of the above X3D specification work seems do-able and driven by X3D glTF implementation work in progress: X3DOM, X_ITE, Castle Game Engine. Any others - FreeWrl perhaps?
> > >
> > > ----------------------------------------------------------------------
> > >
> > > 4. Next steps for X3Dv4, prioritized:
> > >
> > > a. X3Dv4 Specification (github)
> > > b. Examples: Andreas scene, glTF spec examples.
> > > c. Implementations: iterate until glTF-TO-X3D mappings, X3D specification and examples are stable.
> > > d. Encoding and validation work will wait until any new nodes are stable.
> > >
> > > Open question: is it possible to accomplish for the X3D Working Group to accomplish these tasks by Web3D 2019 and SIGGRAPH?
> > >
> > > ----------------------------------------------------------------------
> > >
> > > Wow. So much work (years actually) coming to a focused point that proposes exactly where X3Dv4 is going with respect to glTF.
> > >
> > > Summary: looks like we have a fully actionable strategy for mapping glTF 2.0 to X3Dv4.
> > >
> > > No doubt more discussion and analysis will continue for us to correctly achieve these worthy goals.
> > >
> > > Thanks to many many people getting us to this point! The work continues...
> > >
> > > all the best, Don
> > > --
> > > Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
> > > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
> > > X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> >
> > --
> > Andreas Plesch
> > Waltham, MA 02453
> > _______________________________________________
> > x3d-public mailing list
> > x3d-public at web3d.org
> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
--
Andreas Plesch
Waltham, MA 02453
More information about the x3d-public
mailing list