[x3d-public] draft EnvironmentLight signature needed
Don Brutzman
brutzman at nps.edu
Fri Feb 5 07:51:39 PST 2021
Thanks for steady refinement Michalis. Am removing 'radius' and will clear the TODO.
On 2/5/2021 2:29 AM, Michalis Kamburelis wrote:
>
>
> New "rotation" field indeed is good, following glTF.
>
> I do not think we need "radius". "EnvironmentLight" doesn't have a
> natural position (conceptually it's an infinite sphere around each
> shape that gets the light). It also wouldn't look sensible to have
> part of some shape within, part outside the radius of the
> EnvironmentLight. This one is more naturally controlled by scoping,
> IMHO.
>
> Not sure what you mean by "TODO avoid any variations from glTF IBL at
> this stage?" -- the current spec in practice matches glTF IBL :)
>
> Regards,
> Michalis
>
> pt., 5 lut 2021 o 08:46 Don Brutzman <brutzman at nps.edu> napisał(a):
>>
>> Dick and I worked further on EnvironmentLight today. We wanted it reviewable by others, and so need to define node/field functionality.
>>
>> [1] Mantis 1336: EnvironmentLight - Khronos glTF extension for image based lighting
>> https://www.web3d.org/member-only/mantis/view.php?id=1336
>>
>> [2] X3D4 WD3 17.4.2 EnvironmentLight
>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-WD3/Part01/components/lighting.html#EnvironmentLight
>>
>> To match glTF, I removed /direction/ and added back /rotation/ since such a value might easily exist in a glTF (or other) model getting converted, no need to complicate handling any further.
>>
>> Don't we need to add a /radius/ field similar to PointLight and SpotLight? I've added it for consistency, to reduce unintended side effects of a light, and to facilitate further comment.
>>
>> Wondering if we should reduce or avoid any variations from glTF IBL at this point?
>>
>> [3] https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based
>>
>> [4] https://github.com/michaliskambi/x3d-tests/wiki/Image-Based-Lighting-(EnvironmentLight-node)
>>
>> [5] https://doc.x3dom.org/author/Lighting/PhysicalEnvironmentLight.html
>>
>> Please take a look and see what you think, let's finish this draft for now. Thanks.
>>
>>
>> Current revision:
>> ===================================================
>> 17.4.2 EnvironmentLight
>> Editors NOTE This preliminary node definition indicates intent to implement Image Based Lighting (IBL) as defined by Khronos glTF specification. Current activity by X3D practitioners is focused on converged design, implementation and evaluation using the open-source Castle Game Engine and X3DOM browsers. Further improvements are expected.
>>
>> EnvironmentLight : X3DLightNode {
>> SFFloat [in,out] ambientIntensity 0 [0,1]
>> SFColor [in,out] color 1 1 1 [0,1]
>> SFNode [in,out] diffuse NULL [X3DSingleTextureNode] # .DDS format
>> SFNode [in,out] diffuseTexture NULL [X3DEnvironmentTextureNode]
>> MFFloat [in,out] diffuseCoefficients []
>> SFBool [in,out] global FALSE
>> SFFloat [in,out] intensity 1 [0,∞)
>> SFNode [in,out] metadata NULL [X3DMetadataObject]
>> SFBool [in,out] on TRUE
>> SFFloat [in,out] radius 100 [0,∞)
>> SFRotation [in,out] rotation 0 0 1 0 [-1,1] or (-∞,∞)
>> SFBool [in,out] shadows FALSE
>> SFFloat [in,out] shadowIntensity 1 [0,1]
>> SFNode [in,out] specularTexture NULL [X3DEnvironmentTextureNode]
>> }
>>
>> The EnvironmentLight node supports image-based lighting (IBL) by specifying light-source intensity around a given location (i.e. the environment) as a cube map. EnvironmentLight defines both specular radiance and diffuse irradiance.
>>
>> TODO avoid any variations from glTF IBL at this stage?
>>
>> The diffuseTexture and specularTexture fields define diffuse and specular lighting modifications, respectively.
>>
>> The diffuseCoefficients field provides a 3 x 9 array of float values that declares spherical harmonic coefficients for irradiance up to l=2, corresponding to glTF irradianceCoefficients field. The diffuseCoefficients field overrides the diffuseTexture field if both are provided.
>> ===================================================
>>
>> On 1/31/2021 12:39 PM, Don Brutzman wrote:
>>> Thanks for all review, your note makes consistent sense Michalis. Revision applied to draft spec and committed to github.
>>>
>>> My understanding from the other conversation thread, plus your summary here, is that continuing scrutiny is primarily focused on the nature of texture data being provided... so the fields currently listed in node signature may be nearly complete.
>>>
>>> Am intentionally avoiding addition of node/field functional definitions until confirmed ready, in order to avoid possibility of unintended uncertainties/misunderstandings getting propagated at this stage of maturity.
>>>
>>> Looking forward to you and other implementers getting this important glTF capability matched correctly in X3D4. Onward we go. 8)
>>>
>>> =======================================================
>>> 17.4.2 EnvironmentLight
>>>
>>> Editors NOTE This preliminary node definition indicates intent to implement Image Based Lighting (IBL) as defined by Khronos glTF specification.
>>>
>>> Current activity by X3D practitioners is focused on converged design, implementation and evaluation using the open-source Castle Game Engine and X3DOM browsers. Further improvements are expected.
>>>
>>> EnvironmentLight : X3DLightNode {
>>> SFFloat [in,out] ambientIntensity 1 [0,1]
>>> SFColor [in,out] color 1 1 1 [0,1]
>>> SFNode [in,out] diffuse NULL [X3DSingleTextureNode] # .DDS format
>>> SFVec3f [in,out] direction 0 0 -1 (-∞,∞)
>>> SFBool [in,out] global FALSE
>>> SFFloat [in,out] intensity 1 [0,∞)
>>> SFNode [in,out] metadata NULL [X3DMetadataObject]
>>> SFBool [in,out] on TRUE
>>> SFBool [in,out] shadows FALSE
>>> # draft fields
>>> SFNode [in,out] diffuseTexture NULL [X3DEnvironmentTextureNode]
>>> MFFloat [in,out] diffuseCoefficients [] # used when diffuseTexture = NULL
>>> # must be 3 x 9 array of floats
>>> SFNode [in,out] specularTexture NULL [X3DEnvironmentTextureNode]
>>> }
>>>
>>> TODO. Node and field descriptions will be added when a converged design is implemented and tested.
>>> =======================================================
>>>
>>>
>>> On 1/31/2021 5:07 AM, Michalis Kamburelis wrote:
>>>>
>>>> That's way too much new fields, and most of X3DOM fields you added are
>>>> about shadows (not IBL) :)
>>>>
>>>> I would say boldly that the fields from X3DOM you added (from
>>>> PhysicalEnvironmentLight) should be removed, and only the ones I
>>>> proposed (from CGE) should remain. Reasons:
>>>>
>>>> - The fields "shadowXxx" and "zNear" / "zFar" are about shadow maps.
>>>> We already decided to go with simple shadows fields (for all X3D light
>>>> nodes). So, let's not mix this issue with discussion about shadows :)
>>>>
>>>> - The field "specular" (X3DSingleTextureNode, and required by prose to
>>>> be DDS) from X3DOM is superseded by IMHO better idea of
>>>> "specularTexture" (X3DEnvironmentTextureNode). Such "specularTexture"
>>>> allows to also use DDS (or KTX) for cubemaps, by using
>>>> ImageCubeMapTexture. It also allows to provide 6x 2D images by
>>>> ComposedCubeMapTexture (which is what glTF IBL extension requires).
>>>>
>>>> Regards,
>>>> Michalis
>>>>
>>>>
>>>> niedz., 31 sty 2021 o 01:26 Don Brutzman <brutzman at nps.edu> napisał(a):
>>>>>
>>>>> On 1/30/2021 7:38 AM, Don Brutzman wrote:
>>>>>> Michalis, we are looking to provide a preliminary draft node signature for EnvironmentLight.
>>>>>>
>>>>>> If you can create one that encompasses both approaches, that will best notify reviewers at ISO national bodies on specification-alignment work in progress that is admittedly incomplete.
>>>>>>
>>>>>> Hoping to finalize Monday, I can fill all gaps. Thanks for all of your good preparation and rough-sketch outline now.
>>>>>
>>>>> Here is first draft, checked into github.
>>>>>
>>>>> =========================================
>>>>> 17.4.2 EnvironmentLight
>>>>>
>>>>> Editors NOTE This preliminary node definition indicates intent to implement Image Based Lighting (IBL) as defined by Khronos gltF specification.
>>>>>
>>>>> * https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based
>>>>> * https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0
>>>>>
>>>>> Current activity by X3D practitioners is focused on implementation and evaluation using the open-source Castle Game Engine and X3DOM browsers. Further improvements are expected.
>>>>>
>>>>> * https://github.com/michaliskambi/x3d-tests/wiki/Image-Based-Lighting-(EnvironmentLight-node)
>>>>> * https://doc.x3dom.org/author/Lighting/PhysicalEnvironmentLight.html
>>>>>
>>>>> EnvironmentLight : X3DLightNode {
>>>>> SFFloat [in,out] ambientIntensity 1 [0,1]
>>>>> SFColor [in,out] color 1 1 1 [0,1]
>>>>> SFNode [in,out] diffuse NULL [X3DSingleTextureNode] # .DDS format
>>>>> SFVec3f [in,out] direction 0 0 -1 (-∞,∞)
>>>>> SFBool [in,out] global FALSE
>>>>> SFFloat [in,out] intensity 1 [0,∞)
>>>>> SFNode [in,out] metadata NULL [X3DMetadataObject]
>>>>> SFBool [in,out] on TRUE
>>>>> SFBool [in,out] shadows FALSE
>>>>>
>>>>> # draft X3DOM fields PhysicalEnvironmentLight
>>>>> SFFloat [in,out] shadowIntensity 1 [0,1]
>>>>> SFInt32 [in,out] shadowCascades 1 [0,∞)
>>>>> SFInt32 [in,out] shadowFilterSize 1 [0,∞)
>>>>> SFFloat [in,out] shadowIntensity 1 [0,1]
>>>>> SFInt32 [in,out] shadowMapSize 1 [0,1024]
>>>>> SFFloat [in,out] shadowOffset 0 [0,∞)
>>>>> SFFloat [in,out] shadowSplitFactor 1 [0,1]
>>>>> SFFloat [in,out] shadowSplitOffset 1 [0,1]
>>>>> SFNode [in,out] specular NULL [X3DSingleTextureNode] # .DDS format
>>>>> SFFloat [in,out] zFar -1 -1, [0,∞)
>>>>> SFFloat [in,out] zNear -1 -1, [0,∞)
>>>>>
>>>>> # draft Castle Game Engine fields EnvironmentLight
>>>>> SFNode [in,out] diffuseTexture NULL [X3DEnvironmentTextureNode]
>>>>> MFFloat [in,out] diffuseCoefficients [] # used when diffuseTexture = NULL
>>>>> # must be 3 x 9 array of floats
>>>>> SFNode [in,out] specularTexture NULL [X3DEnvironmentTextureNode]
>>>>> }
>>>>>
>>>>> TODO. Node and field descriptions will be added when a converged design is implemented and tested.
>>>>>
>>>>> =========================================
>>>>>
>>>>> Perhaps sufficient for submission... Tracking as
>>>>>
>>>>> * Mantis 1336: EnvironmentLight - Khronos glTF extension for image based lighting
>>>>> https://www.web3d.org/member-only/mantis/view.php?id=1336
>>>>>
>>>>> Superficial review welcome, no need to make any design decisions or perform implementation efforts at this stage.
>>>>>
>>>>>
>>>>> Related thread:
>>>>>
>>>>> * [x3d-public] Comment of X3D V4, glTF, and Lighting
>>>>> http://web3d.org/pipermail/x3d-public_web3d.org/2021-January/014459.html
>>>>>
>>>>> Recent background email thread (which was not previously archived on email list) follows.
>>>>>
>>>>> On 1/29/2021 8:31 AM, Andreas Plesch wrote:
>>>>>>
>>>>>> Good point about SHCubeMapTexture. SH in computer graphics really is
>>>>>> only used and useful for IBL, it appears.
>>>>>>
>>>>>> And 27 * 4 is 108 not 112. It was too late, something misfired.
>>>>>>
>>>>>> Cheers, -Andreas
>>>>>>
>>>>>> On Fri, Jan 29, 2021 at 9:13 AM Michalis Kamburelis
>>>>>> <michalis.kambi at gmail.com> wrote:
>>>>>>>
>>>>>>> Andreas,
>>>>>>>
>>>>>>> Thank you -- I agree with everything In particular I also saw the
>>>>>>> spherical harmonics (instead of a cubemap texture) used for "diffuse"
>>>>>>> in glTF IBL extension as something a bit non-standard. Most IBL
>>>>>>> implementations I saw take a cubemap texture as input (whether it
>>>>>>> means 1x DDS/KTX or 6x 2D image files). But as you say. this also
>>>>>>> probably performs very well.
>>>>>>>
>>>>>>> This strenghtens my idea that we should support both options in X3D
>>>>>>> EnvironmentLight.
>>>>>>>
>>>>>>> As for your approach of SHCubeMapTexture:
>>>>>>>
>>>>>>> - on one hand,I like it a lot, because it makes these 2 problems
>>>>>>> independent I mean, then EnvironmentLight could just refer to
>>>>>>> X3DEnvironmentTextureNode, and it's an independent decision whether we
>>>>>>> add SHCubeMapTexture.
>>>>>>>
>>>>>>> - OTOH, it makes implementation harder. Right now all
>>>>>>> X3DEnvironmentTextureNode descendants result in the same thing for GPU
>>>>>>> - a cubemap sampler. So e.g. shaders don't care about the difference
>>>>>>> between ImageCubeMapTexture, GeneratedCubeMapTexture,
>>>>>>> ComposedCubeMapTexture . If we add SHCubeMapTexture as new
>>>>>>> X3DEnvironmentTextureNode, then it has to be supported everywhere
>>>>>>> where we allow a cubemap (not only at EnvironmentLight), and browsers
>>>>>>> have to be prepared that this "different cubemap implementation"
>>>>>>> requires different handling in shaders.
>>>>>>>
>>>>>>> Thank you for the references, reading
>>>>>>>
>>>>>>> Regards,
>>>>>>> Michalis
>>>>>>>
>>>>>>> pt., 29 sty 2021 o 02:15 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
>>>>>>>>
>>>>>>>> Yes, x3dom has Timo's PhysicalEnvironmentLight. It only accepts
>>>>>>>> cubemaps in the form of DDS files.
>>>>>>>>
>>>>>>>> This has the advantage that it is simple and allows MIP levels and HDR.
>>>>>>>>
>>>>>>>> X3DEnvironmentTextureNode as in Michalis' proposal also allows DDS
>>>>>>>> through ImageCubeMapTexture. This signature is more flexible
>>>>>>>> (requiring more implementation work), and more similar to glTF which
>>>>>>>> expects separate images for each side of the cube map.
>>>>>>>> An x3dom implementation would require first implementing
>>>>>>>> ImageCubeMapTexture and quite a bit of replumbing.
>>>>>>>>
>>>>>>>> The coefficients for irradiance are for spherical harmonics which
>>>>>>>> allow for an extremely efficient representation of sampling levels of
>>>>>>>> a spherical image. Apparently, these coefficients are easy to use in
>>>>>>>> the shader and require only a few lines of computation. There are
>>>>>>>> almost certainly sample implementations. It sounds like a nice option
>>>>>>>> but I wonder how common the use of spherical harmonic coefficients for
>>>>>>>> IBL really is. Maybe it is the standard now ?
>>>>>>>>
>>>>>>>> The main motivation for the glTF spec.extension to prefer 27
>>>>>>>> coefficients over a cubemap must be efficiency, eg. savings in file
>>>>>>>> size. 27 * 4 bytes is only 112 bytes. However, a comparable cubemap
>>>>>>>> image should be also pretty small, relative to other content. 3 levels
>>>>>>>> of spherical harmonics may correspond to only a 8x8 pixel image, times
>>>>>>>> 6 for each face. 384 pixels maybe times 12 bytes is only 4.5 kbytes or
>>>>>>>> so, uncompressed. So I am not sure if the added complexity is worth
>>>>>>>> the savings.
>>>>>>>>
>>>>>>>> While generating the coefficients from a cube map seems a bit
>>>>>>>> involved, using them in the shader to find a color for a given
>>>>>>>> direction is apparently pretty straightforward. So these probably
>>>>>>>> should be supported. The harder part may be how to exactly define
>>>>>>>> these coefficients in the spec.
>>>>>>>>
>>>>>>>> Although coefficients are probably only useful for IBL, they are from
>>>>>>>> a logical standpoint an implementation of X3DEnvironmentTextureNode.
>>>>>>>> So one option would be to add a field MFFloat "SHcoefficients" to
>>>>>>>> Image or ComposedCubeMapTexture or have a new node "SHCubeMapTexture"
>>>>>>>>
>>>>>>>> Hm, searching the github repo found this:
>>>>>>>>
>>>>>>>> https://github.com/KhronosGroup/glTF/pull/1850
>>>>>>>>
>>>>>>>> which is discussed as version 2 of the extension. Probably worth
>>>>>>>> studying. KTX2 as container image format may need to be supported.
>>>>>>>>
>>>>>>>> https://github.com/KhronosGroup/glTF/pull/1377 is the original discussion.
>>>>>>>>
>>>>>>>> https://github.com/KhronosGroup/glTF/pull/1612#issuecomment-495737070
>>>>>>>> has the math for SH.
>>>>>>>>
>>>>>>>> More generally, I am not sure how strongly the glTF extension should
>>>>>>>> guide an X3D EnvironmentLight node. It is only an extension and most
>>>>>>>> glTF models do not use it expecting the renderer to provide their own
>>>>>>>> idea of IBL.
>>>>>>>>
>>>>>>>> -Andreas
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 28, 2021 at 7:30 AM Michalis Kamburelis
>>>>>>>> <michalis.kambi at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> X3DOM, as far as I know, implements
>>>>>>>>> https://doc.x3dom.org/developer/x3dom/nodeTypes/PhysicalEnvironmentLight.html
>>>>>>>>> . Which follows the original design of Timo Sturm, and it was the
>>>>>>>>> first approach to IBL in glTF (it was also used in first glTF sample
>>>>>>>>> viewer implementation of IBL), with "diffuse" being a cubemap.
>>>>>>>>>
>>>>>>>>> The new glTF approach to IBL is a bit different (
>>>>>>>>> https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based
>>>>>>>>> ). In particular in new glTF approach, the "diffuse" (which is now
>>>>>>>>> called "irradiance") is defined by coefficients, not a cubemap
>>>>>>>>> texture.
>>>>>>>>>
>>>>>>>>> This is an important change, and it's exactly what I describe on
>>>>>>>>> https://github.com/michaliskambi/x3d-tests/wiki/Image-Based-Lighting-(EnvironmentLight-node)
>>>>>>>>> We should find (and test!) a way to reconcile these approaches, or
>>>>>>>>> just use the new (from glTF extension) one. So that the new
>>>>>>>>> EnvironmentLight
>>>>>>>>>
>>>>>>>>> 1. is comfortable for authors,
>>>>>>>>> 2. matches glTF extension for IBL,
>>>>>>>>> 3. is implementable in X3D browsers without much pain.
>>>>>>>>>
>>>>>>>>> And note that we should do more than glTF. EnvironmentLight in X3D
>>>>>>>>> should work with both PhysicalMaterial and Material (because it can,
>>>>>>>>> and because other lights also do), on both normal X3D shapes and on
>>>>>>>>> X3D shapes imported from glTF.
>>>>>>>>>
>>>>>>>>> I welcome Andreas (and other implementors) comments about this. Maybe
>>>>>>>>> we can design a new EnvironmentLight and make it work in e.g. X3DOM
>>>>>>>>> sooner?
>>>>>>>>>
>>>>>>>>> I emphasize, I do *not* require that EnvironmentLight must be
>>>>>>>>> implemented in CGE/view3dcene before it is added to the spec. I am not
>>>>>>>>> special, I don't want you to wait for me with something
>>>>>>>>>
>>>>>>>>> But I *do* require that EnvironmentLight (in the form matching X3D
>>>>>>>>> spec, and matching glTF IBL extension) must be implemented in *some*
>>>>>>>>> browser, and we must have *some* testcases.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Michalis
>>>>>>>>>
>>>>>>>>> czw., 28 sty 2021 o 06:57 Don Brutzman <brutzman at nps.edu> napisał(a):
>>>>>>>>>>
>>>>>>>>>> Thanks for status updates Andreas, very helpful and promising.
>>>>>>>>>>
>>>>>>>>>> Michalis, given this additional interesting status:
>>>>>>>>>>
>>>>>>>>>> On 1/27/2021 6:44 PM, Andreas Plesch wrote:
>>>>>>>>>>> X3DOM also has a working EnvironmentLight node for image based
>>>>>>>>>>> lighting (IBL). I believe it only affects PhysicalMaterial nodes which
>>>>>>>>>>> are used internally for glTF models.
>>>>>>>>>>
>>>>>>>>>> If you think we can have a reasonable "first draft" approximation of EnvironmentLight field signature and functional definitions, it is likely preferable to get this into the CD specification (thereby inviting continued comment and implementation discussions) rather than waiting until you are able to pursue your implementation.
>>>>>>>>>>
>>>>>>>>>> The glTF definition looks reasonably well defined and stable, and if they change we will match. If you want, I'll try to draft it.
>>>>>>>>>>
>>>>>>>>>> Seems like there is little downside and some good positives if we put that draft node in...
>>>>>>>>>>
>>>>>>>>>> Please advise what you think, thanks for reconsidering it.
>>>>>
>>>>> all the best, Don
>>>>> --
>>>>> Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
>>>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
>>>>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
>>>
>>> all the best, Don
>>
>> all the best, Don
>> --
>> Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list