[x3d-public] consistency of glTF renders improving, comparable

[Don Brutzman]

Summary: consistency of glTF renders improving and now comparable.

Sandy tweeteth:
https://twitter.com/sressler/status/1303806443993411586

> Looks great...perhaps you might call it "conformance testing" isn't that what it is? (and that's the normal term in the standards world)

 From @modleviewer:
https://twitter.com/modelviewer/status/1303728820583718913

> We've updated our fidelity comparisons of the top web renderers of @glTF3D to demonstrate how well converged the industry is now: https://modelviewer.dev/fidelity/
> Now you can have confidence that when authoring a #glTF it will look right anywhere on the web, including in #AR with #WebXR.

glTF comparison page of interest:

https://modelviewer.dev/fidelity

> Render Fidelity Comparison Results
> The purpose of glTF is to standardize Physically-Based Rendering (PBR) materials such that you can be confident your model will appear as intended in any lighting environment in any renderer. This is a very ambitious goal, as real-time rendering at this level of quality is still very much an area of active research with improvements being made constantly. This site demonstrates where we are on that path to convergence and highlights areas that could still use improvement. We are comparing the most popular web renderers: three.js (represented by <model-viewer>), filament.js, and babylon.js. If any other renderers would like to be included, please open a PR adding them to the render-fidelity-tools package.
> 
> In addition, we are showing a ground-truth render provided by dspbr-pt, which is a very slow path-tracing renderer based on three.js. The advantage it provides is simplicity: without the speed requirement it can be made clear and readable and as it runs with more and more samples it approaches the physical result with far fewer approximations than are necessary in real-time renderers.
> 
> The first set of scenarios are for demonstrating material capabilities, so these are primarily fidelity tests. Following these are simpler cases that test glTF spec conformance, but are not particularly interesting for fidelity. At the bottom are the Furnace Test and the Directional Light Test. The Furnace Test checks energy conservation and the correct result is the whole 7x7 grid of metal-rough spheres being invisible (indistinguishable from the background). The top (metal) and bottom (dielectric) rows are passing, while the mixtures lose some energy. The Directional Light Test checks the worst-case scenario for environmental lighting approximations: the case where a single bright pixel represents a directional light. The Filament version uses an actual directional light, so this is the ground truth.
> 
> In <model-viewer>, we do not consider rendering changes to be breaking changes, as our quality is incrementally improving with every release. However, the difference between the renders you see here bounds how much you can expect our rendering to change going forward. Note the largest differences are in the handling of transparent materials, as this is quite difficult to get right, so the largest changes will likely be here.
> 
> We show a simple logarithmic metric on an average of the pixel differences, excluding the transparent background. This is not a perceptual metric, so take it with a grain of salt. It is mostly there to help us identify regressions.

Screenshot examples attached.  Thanks Sandy!

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ModelViewerFidelityTest1.png
Type: image/png
Size: 201146 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200912/dd38f590/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ModelViewerFidelityTest2.png
Type: image/png
Size: 176895 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200912/dd38f590/attachment-0003.png>