[x3d-public] beyond Blinn-Phong: PBR

Wed Jan 30 09:08:35 PST 2019

Since we are getting closer to x3dom merging in gltf2 inline support in its
dev. version, here a first example of multiple gltf inlines, using an older
x3dom scene:

https://rawcdn.githack.com/andreasplesch/x3dom/b34506419c8877884501143b122cbcbd287abb43/test/regression-suite/test/cases/routes/routes.html

Using multiple levels of instancing, the scene has ca. 180 shapes
consisting of ca. 500K triangles. It plays well on my desktop but I did not
try on mobile.

It also shows a regular x3d material (the x3d box) next to gltf PBR
materials (the duck, fish, boombox and avocado).

-Andreas

On Sat, Jan 26, 2019 at 3:39 PM Andreas Plesch <andreasplesch at gmail.com>
wrote:

> On Sat, Jan 26, 2019, 8:14 AM Joseph D Williams <joedwil at earthlink.net
> wrote:
>
>>
>>    - All glTF syntax have a large binary component (the 'buffer') which
>>    contains the bulk of the data (geometry, animation, textures).
>>
>>
>>
>> Thanks Andreas, all real nice coverage.
>>
>>
>>
>>    - Should there be a way for a X3D scene to control animations in an
>>    inline glTF ?
>>
>>
>>
>> Is there an inline gltf that contains enough data to generate and control
>> animations?
>>
>> How about if the inline can perform like a video, start/stop/showmetadata?
>>
> I think the video clip analogy could be a useful concept. In fact, gltf
> viewers often have the typical media controls to play animations. There may
> be additional interface to select which clip is played (run, jump ..,
> pumping, circling), and at which speed (overriding the authored speed).
>
>> An inline gltf may be a container consisting of several gltf files made
>> up from a list somewhere?
>>
> A gltf currently cannot reference or include other gltfs. I think I saw
> that suggested for the future, but the general feel is that it is focused
> on single assets rather than complex scenes.
>
> hm, what would be the difference of having multiple inlines with single
> gltfs ?
>
> Then this gltf mass also skips validation and has its own runtime?
>>
>> And can talk to the parent scene using data and commands with standard
>> and custom interfaces?
>>
>> Do some gltf need its own sandbox?
>>
>>
>>
>> I think I still look at gltf resources simply as inert data containers
>> you are using for some sort of convenience or protection in your
>> application of realtime interactive immersive 3D for the WorldWildWeb. .
>>
> Yes, a 3d image or video. There should not be an expectation to modify the
> gltf content, just control included animations.
>
>> Some gltf structures are appropriate for identification as being usable
>> as x3d native binary assets because the data can be identified as a trusted
>> external asset by x3d authortime and possibly used directly without further
>> processing by runtime. When those overlap you can at least get tooltips and
>> hopefully actual feedback.
>>
>> At the first level the gltf must be a convenient highly optimized
>> container for one or more strongly typed x3d field values to be plugged
>> directly into one or more basic x3d containers. In this, the gltf is
>> available at authortime, just that it is a ‘external’ binary thing that
>> does not need to be human-readable. Well, sure, some metadata.
>>
>>
>>
>> But hey, the big problem is not that the data structures of the
>> authortime and runtime has diverged, just that they may be basically
>> different due to different considerations. Maybe see Humanoid where the x3d
>> authortime uses each joint as a container for the skin animation vertex
>> bindings, while a gltf may seem to prefer using each skin vertex as the
>> container for the bound joints at one step which gives no indication
>> concerning from where the instant data was derived. The first is in this
>> case a clear authortime consideration, the second may be a preferred best
>> practice data structure implemented by a ‘standard’ or consensus runtime.
>> The first is based on how a human can organize a complex interactive object
>> from the top down while the second may expose one deep aspect of one
>> discrete step in a complex process the vizualization machine takes to
>> produce one tiny part of one flash of one frame.
>>
>>
>>
>> However, when data is the same, why not transport some of it between the
>> gltf environment which is optimized for runtime and x3d which is optimized
>> for authortime? Fine, when the data fits. Let’s do it.
>>
> Much of gltf can be translated into x3d plus pbr material with an offline
> translator (to be). Direct gltf support for Inline raises questions which
> can be separated out.
>
> Now that x3dom has pretty good gltf inline support I am interested to see
> how it integrates with interactive and scene x3d features. First would be
> just having multiple gltfs in a scene.
>

>
> Andreas
>
>>
>>
>> Best Regards,
>>
>> Joe
>>
>>
>>
>> *Subject: *Re: [x3d-public] beyond Blinn-Phong: PBR
>>
>>
>>
>> A response below
>>
>>
>>
>> Message: 2
>> Date: Mon, 21 Jan 2019 02:27:53 -0800
>> From: Joseph D Williams <joedwil at earthlink.net>
>> To: Andreas Plesch <andreasplesch at gmail.com>,  Michalis Kamburelis
>>         <michalis.kambi at gmail.com>
>> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>
>> Subject: Re: [x3d-public] beyond Blinn-Phong: PBR
>> Message-ID: <E1glWo5-0007qm-6l at elasmtp-dupuy.atl.sa.earthlink.net>
>> Content-Type: text/plain; charset="utf-8"
>>
>>
>> ? ? glTF supports animation of transformation components but does not
>> define exactly when an animation is played,
>>
>> So, if I remember, as if I just awoke from a long sleep, then the gltf is
>> to provide a secure and reliable way to transport carefully defined data
>> structures into and out of either an unknown authortime or a known runtime.
>> Maybe I am focusing of the binary forms for various gltf syntax.
>>
>>
>>
>> Yes, glTF is supposed to be transport format to be used inbetween
>> authortime and runtime. Its design goal is to avoid friction for the
>> runtime to render on the screen using a limited subset of OpenGL (webGL) at
>> the expense of authortime legibility. All glTF syntax have a large binary
>> component (the 'buffer') which contains the bulk of the data (geometry,
>> animation, textures).
>>
>>
>>
>> glTF can be validated. But it does not have built-in security to guard
>> against tampering and such. Systems need to provide their own security
>> layer around glTF where required.
>>
>>
>>
>> gltf took collada and the rest into the deepest parts of best practice
>> highly optimized data structures of mesh and field simulations in a form
>> the best practice realtime 3D graphics processing tools wanted to get it --
>> Validated trusted secure binary the runtime can swallow without chewing.
>>
>> The runtime can import this binary directly because the runtime is a
>> ?standard? runtime using  ?standard? known best-practice runtime data
>> structures. These data structures are defined by the preferred forms for
>> binary data used directly by geometry and animation processors. The gltf
>> incidentally has an authortime text form to provide systematic
>> human-readable editing and validation, but the human-readable form is just
>> to allow data specialists to define carefully denominated strongly typed
>> definitions for the binary form actually used by features of the runtime.
>>
>>
>>
>> The geometry, animation and shading processor targeted are GPU shading
>> programs. Although initially targeted at GLSL, wide support has shown that
>> there is enough abstraction to allow other platforms as well although I do
>> not know how much effort this required.
>>
>>
>>
>> The x3d deals mainly with the authortime form and does not define any
>> runtime data structures except for runtime external and internal scenegraph
>> event flows. An x3d tool would be expected to produce its internal runtime
>> structures from the ?standard? x3d user code according to its needs. The
>> great purpose of gltf is to provide a known binary form that accurately
>> reflects known expectations of the runtime.
>>
>> A comprehensive x3d authortime tool might have an accessory to directly
>> import and use certain text forms of gltf if the data types and structures
>> are the same. Likewise, an x3d runtime might be able to directly import and
>> use certain binary forms. Or, any combination thereof.
>>
>>
>>
>> There are the parallel options of translating glTF to x3d which is
>> currently possible for geometry and transform animations, and treating glTF
>> as inline, inert, jpg-like content. A translator could always be used at
>> authortime and provides most flexibility. glTF inline has the advantage
>> that it can be easier to implement and preserves 'swallowing without
>> chewing'. New x3d nodes such as PhysicalMaterial or BufferView/Accessor for
>> binary data can be imagined as internal helpers or as spec. worthy x3d
>> additions.
>>
>>
>>
>> Since the gltf is defined and supported by the same folks who developed
>> the concepts behind realistic accelerated realtime 3D Graphics, from which
>> x3d is derived, then we might expect that gltf would be of great use to x3d.
>>
>>
>>
>> glTF has wide industry support. Incidentally, PBR was not part of glTF
>> 1.0, which required content to provide its own shaders.
>>
>>
>>
>> ? ? but does not define exactly when an animation is played,
>>
>> So far, the gltf does not define an interactive runtime that I have seen
>> (please send a link if there is), although I think I have seen authoring
>> that defines sets of gltf data and seen runtimes that use sets of gltf
>> files to produce an animated scene.
>>
>>
>>
>> You are right. Animation is seen as clips made available to the runtime
>> in the context of a larger scene. glTF just provides the storage. The
>> runtime needs to provide interactivity and logic.
>>
>>
>>
>> Should there be a way for a X3D scene to control animations in an inline
>> glTF ? If so how ? Using standardized DEF  names for (virtual) TimeSensor,
>> Interpolator nodes from the glTF ?
>>
>>
>>
>> Thanks and Best,
>> Joe
>>
>>
>>
>>
>>
>> From: Andreas Plesch
>> Sent: Sunday, January 20, 2019 6:31 PM
>> To: Michalis Kamburelis
>> Cc: X3D Graphics public mailing list
>> Subject: Re: [x3d-public] beyond Blinn-Phong: PBR
>>
>> Timo Sturm provided these nodes. The main work is to get to the gltf data
>> to a special shader which in itself is not complex. I think the goal is to
>> proceed to an overdue x3dom release after consolidating into development
>> release with what we have. There would be some time to make adjustments to
>> node signatures perhaps focusing on PhysicalMaterial.
>>
>> The flipped Y axis for texture coordinate is handled in the fragment
>> shader:
>>
>>
>> https://github.com/x3dom/x3dom/blob/webVR/src/shader/ShaderPBRMaterial.js#L124
>>
>> The 1 - coord.y should not have a noticeable performance impact. There is
>> currently no way to access the gltf texture coordinates from the x3d scene.
>>
>> glTF supports animation of transformation components but does not define
>> exactly when an animation is played, just how long a cycle is. x3dom just
>> plays and loops them but could opt to play once or not play and just
>> provide the equivalent Timesensor, Interpolator and ROUTE nodes for the
>> scene to use as it wants. On the other hand there is no way to EXPORT from
>> a gltf inline.
>>
>> -Andreas
>>
>> On Sun, Jan 20, 2019 at 3:21 PM Michalis Kamburelis <
>> michalis.kambi at gmail.com> wrote:
>> Andreas Plesch <andreasplesch at gmail.com> wrote:
>> >
>> > For glTF inline support, PBR shading will be required. The glTF spec.
>> has an example implementation for PBR but for anybody who wants to dive in
>> more, there is the free
>> >
>> > http://www.pbr-book.org
>> >
>> > It seems somewhat artificially inflated and is based on ray tracing but
>> explains the foundational PBR concepts in depth, and in code.
>> >
>> > I just wanted to share that resource but will point out that x3dom will
>> have a PhysicalMaterial node and an EnvironmentLight node which are used by
>> the glTF loader.
>> >
>>
>> Cool! I also want to define PhysicalMaterial and EnvironmentLight in
>> Castle Game Engine, for interoperability with glTF. It will be good to
>> converge on a single specification for these nodes, and eventually add
>> it to the X3D 4 specification :) We have the same goals (achieve PBR
>> in X3D, and achieve it in a way that makes glTF 2.0 -> X3D conversion
>> straightforward), so I'm sure this will be possible.
>>
>> I don't have an exact specification yet (my rough ideas are on
>> https://github.com/michaliskambi/x3d-tests/wiki/How-to-add-PBR-to-X3D%3F
>> , this page collects thoughts and info from our talks on x3d-public :)
>> ).
>>
>> Regards,
>> Michalis
>>
>>
>>
>> --
>> Andreas Plesch
>> Waltham, MA 02453
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL: <
>> http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190121/2066856d/attachment.html
>> >
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>
>>
>> ------------------------------
>>
>> End of x3d-public Digest, Vol 118, Issue 60
>> *******************************************
>>
>>
>>
>> --
>>
>> Andreas Plesch
>> Waltham, MA 02453
>>
>>

-- 
Andreas Plesch
Waltham, MA 02453
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190130/a0604fa8/attachment-0001.html>