[x3d-public] beyond Blinn-Phong: PBR

Andreas Plesch andreasplesch at gmail.com
Mon Feb 11 07:23:58 PST 2019


Michalis,

well, the alternative would be add a SFInt32 'channel' field to
X3DTextureCoordinate and X3DTextureTransform which then would be
inherited by all derived nodes.

It would have all the same advantages including sharing.

MultiTextureXXX nodes would be only used for MultiTexture textures.

BTW, the spec. does not spell out what to do if a
MultiTextureCoordinate node is used as a choice inside another
MultiTextureCoordinate node which is allowed by the signature and
presumably schema. It is a minor and unrelated point since it is clear
that this just leads to undefined behavior.

The implicit mapping for MultiTexture between textures and texture
coordinates/transforms by their index in their arrays feels
underspecified to me but this also unrelated.

The main disadvantage of introducing a 'channel' field is that the
Geometry.texcoord and Appearance.textureTransform fields would need to
become MFNode fields and that there is a perceived overlap with
MultiTextureCoordinate. It would be also necessary to define if
'channel' is used with MultiTexture (ignore) and what to do if the
same channel number is used for multiple texcoords ( undefined, or
last one wins).

The main advantage is for authors. Adding another set of texture
coordinates would mean just adding another TextureCoordinate node. And
then referring to its channel wherever the coordinates are used. It
would be a more natural transition from a single set of texcoords to
multiple sets. Also it is conceptually easier to understand for
authors that all MultiTexture* nodes work together and only together,
as they were originally designed.

Hm, I was actually ready to say that precedence trumps all and that
therefore just expanding the intended use of
MultiTextureCoordinate/Transform is appropriate but I am not so sure
anymore :)

It is true, I believe, that elevating MutliTextureCoordinate/Transform
for use with other nodes requires the least amount of spec. work.

-Andreas


On Mon, Feb 11, 2019 at 5:04 AM Michalis Kamburelis
<michalis.kambi at gmail.com> wrote:
>
>  Andreas,
>
> Whether we use MultiTextureCoordinate (with new xxxTextureCoordinateId
> fields in materials):
>
> I'm after using them, because for me, MultiTextureCoordinate and
> MultiTextureTransform nodes "stand on their own" :) That is, I think
> they are not "inseparable" from the MultiTexture node. They can be
> "decoupled" from MultiTexture node without any problems.
>
> The way I see them, MultiTextureCoordinate and MultiTextureTransform
> nodes allow to define a set of texture coordinates on a geometry.
> These texture coordinates can be used in various fashion -- by
> rendering with MultiTexture, by rendering with CommonSurfaceShader, by
> rendering with custom shader code (where textures may be provided as a
> number of uniform values, ImageTexture slots to a shader node, or as
> one MultiTexture node, or a mixture of it). So it makes sense that
> they would also be useful with the textures defined inside the new
> (extended) Material and PhysicalMaterial.
>
> I know that I'm interpreting here X3D specification more "broadly"
> than what http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/texturing.html
> says :) But the practice confirms that MultiTextureCoordinate and
> MultiTextureTransform nodes make sense for these additional use-cases.
> So I would prefer to keep using them, instead of inventing new nodes.
>
> Some nice features of existing MultiTextureCoordinate approach:
>
> - It allows to put both explicit tex coords (TextureCoordinate,
> TextureCoordinate3D, TextureCoordinate4D) and implicit coordinates
> (TextureCoordinateGenerator, in CGE also ProjectedTextureCoordinate)
> inside.
>
> - It can be placed everywhere where we have texCoord. (CGE even allows
> it on primitives,
> https://castle-engine.io/x3d_implementation_texturing_extensions.php#section_ext_tex_coord
> , although I introduced MultiGeneratedTextureCoordinate for this --
> it's like MultiTextureCoordinate but only allows implicit coordinates,
> by TextureCoordinateGenerator or ProjectedTextureCoordinate ).
>
> - They allow for useful sharing. The X3DTextureCoordinateNode node and
> friends *do not* specify their index inside (instead, the index is
> implicit by their position inside MultiTextureCoordinate). This allows
> to use the same coordinates at different indexes in various geometry
> nodes.
>
> Regards,
> Michalis
>
> niedz., 10 lut 2019 o 23:35 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> >
> > I looked up what x3dom is currently doing with regards to multiple
> > sets of texture coordinates. In short, I think they are only supported
> > for glTF although xxxTextureCoordinateId fields exist for
> > SurfaceShader textures (which are derived from a wrapper abstract
> > texture node dedicated for surface shader), and although there is a
> > MultiTextureCoordinate node. But these fields and nodes do not seem to
> > be used.
> >
> > For glTF, Timo introduced a "channel" field for X3DTextureNode which
> > indicates which set of texture coordinates should be used. So this is
> > equivalent to a xxxTextureCoordinateId field. Currently, this channel
> > field can only hold 0 or 1 since only a maximum of two sets are
> > supported.
> >
> > But in x3dom there is currently no way to actually define a second set
> > of texture coordinates outside of glTF since MultiTextureCoordinate is
> > effectively ignored.
> >
> > I am actually not sure if MultiTextureCoordinate should be used for
> > this purpose since it is designed to be used be with MultiTexture
> > which is dedicated to blending diffuse and specular textures in
> > various ways. And here it would be used more generally for
> > SurfaceShader and PhysicalMaterial. So for spec. purposes it comes
> > down to asking if it is easier to expand the meaning of an existing
> > node (but keep its signature), or to define a new node or field with a
> > similar purpose. So, for example, one could add a channel field to
> > TextureCoordinate and make texcoord a MFNode field, or have a
> > TextureCoordinateChannels node.
> >
> > I also noticed that the spec. for MultiTextureCoordinate does not
> > really define how to map a set of texture coordinates to a texture in
> > MultiTexture. It seems it is implied it is by index in the MFNode
> > arrays but one has to infer that. So xxxTextureCoordinateId, I think,
> > is really xxxTextureCoordinateIndex
> >
> > Cheers,
> >
> > -Andreas
> >
> >
> > >
> > > :)
> > >
> > > niedz., 10 lut 2019, 20:50 użytkownik Michalis Kamburelis <michalis.kambi at gmail.com> napisał:
> > >>
> > >> 1. As for TextureCoordinates: Each xxxTexture coordinate should be accompanied by a xxxTextureCoordinateId , which is an index into node's MultiTextureCoordinate (in geometry texCoord). I deliberately omitted this for simplicity in my previous mail :), but in reality these are needed.
> > >>
> > >> This is the approach that CommonSurfaceShader is using ( https://castle-engine.io/x3d_implementation_texturing_extensions.php#section_ext_common_surface_shader ) and glTF. CGE is already using it when converting glTF to X3D with CommonSurfaceShader, so normal map can reuse or not reuse texture coords of the base texture.
> > >>
> > >> So, this is something we already can do easily. MultiTextureCoordinate is simple and cooperates nicely with this approach.
> > >>
> > >> 2. As for reUSEing the nodes: Hm, good point. If you reuse textures, then you probably reuse diffuse textures and specular textures and normalmaps at the same time. All these textures must be prepared to work on the shapes you're applying them on (e.g. having separate areas for each shape, if you're reusing the texture to achieve texture atlas optimization).
> > >>
> > >> Maybe, instead of Appearance.normalMap, it is better to just put normalMap (and normalMapCoordinateId) in a base material abstract class (like X3DOneSidedMaterialNode) and derive both Material and PhysicalMaterial from it.
> > >>
> > >> P.S. And change normalMap/normalMapCoordinateId to normalTexture/normalTextureCoordinateId, these are more consistent. CommonSurfaceShader also calls it normalTexture, I guess for the same reason.
> > >>
> > >> 3. P.P.S. I deliberately avoided thinking for now what to do with TwoSidedMaterial ;) Introducing TwoSidedPhysicalMaterial, for consistency, feels uncomfortable -- it would be a node with many fields. Possibly new node like GenericTwoSidedMaterial should be added, with two slots for back and front  X3DOneSidedMaterialNode, with the requirement that they should be the same class (that is, both front and back should be Material, or they both should be PhysicalMaterial). I'm not convinced about the usefulness of TwoSidedMaterial in practice (while it seems very useful for authors, 3D software I know doesn't support it, so in practice you just create a separate mesh with flipped normals, and a different material, which nullifies the need for TwoSidedMaterial). So I don't have much of an opinion here yet.
> > >>
> > >> Anyway, TwoSidedMaterial is why we currently have X3DMaterialNode in spec.
> > >>
> > >> Regards,
> > >> Michalis
> > >>
> > >> niedz., 10 lut 2019, 17:10 użytkownik Andreas Plesch <andreasplesch at gmail.com> napisał:
> > >>>
> > >>> Super input. Just a few remarks.
> > >>>
> > >>> On Fri, Feb 8, 2019 at 5:48 PM Michalis Kamburelis <michalis.kambi at gmail.com> wrote:
> > >>> >
> > >>> > Thanks for many good ideas! I agree with everything, I only want to
> > >>> > add a note to the below paragraph:
> > >>> >
> > >>> > Andreas Plesch <andreasplesch at gmail.com> wrote:
> > >>> > > A node design choice was to add the texture maps as fields to
> > >>> > > PhyicalMaterial rather than Appearance. I think Timo's reasoning was
> > >>> > > that this way all new functionality can be contained in one new node.
> > >>> > > But this is a bit of a departure. Other designs are certainly
> > >>> > > possible.
> > >>> >
> > >>> > I see advantages of this design, where a material node allows to
> > >>> > configure every property by a constant factor (scalar, vector) and
> > >>> > optionally to multiply it by a texture.
> > >>> >
> > >>> > CommonSurfaceShader also uses this design, CGE docs (linking to X3DOM
> > >>> > and InstantReality docs):
> > >>> > https://castle-engine.io/x3d_implementation_texturing_extensions.php#section_ext_common_surface_shader
> > >>> >
> > >>> > E.g. in CommonSurfaceShader you have
> > >>> >
> > >>> >   SFVec3f diffuseFactor
> > >>> >   SFNode  diffuseTexture
> > >>> >   SFVec3f emissiveFactor
> > >>> >   SFNode  emissiveTexture
> > >>> >   SFVec3f specularFactor
> > >>> >   SFNode  specularTexture
> > >>> >   .. and so on
> > >>> >
> > >>> > (I'm simplifying a bit, in reality you need at least a way to provide
> > >>> > texture coordinate index to each texture slot, so there are more
> > >>> > fields.)
> > >>>
> > >>> > In my view, we should follow this approach to the consistent end :) So
> > >>> > PhysicalMaterial would have
> > >>> >
> > >>> >   baseColorFactor
> > >>> >   baseColorTexture
> > >>> >   .. and so on
> > >>> >   (or maybe just baseColor, baseTexture)
> > >>>
> > >>> It is called basecolortexture in glTF but baseTexture would be following x3d conventions.
> > >>>
> > >>> > and to the regular Material we add
> > >>> >
> > >>> >   diffuseTexture
> > >>> >   specularTexture
> > >>> >   emissiveTexture
> > >>> >   .. and so on
> > >>> >
> > >>> > So all factors (scalars/vectors) have a counterpart texture, and the
> > >>> > texture field is present right next to the non-texture (scalar/vector)
> > >>> > value.
> > >>>
> > >>> > Advantages:
> > >>> >
> > >>> > - This is simple for authors. """Question: What does the texture in
> > >>> > slot xxxTexture do? Answer: The same thing as xxxFactor, but it's a
> > >>> > texture so it allows to vary this material property over a surface.
> > >>> > See the xxx treatment in lighting equations."""
> > >>> >
> > >>> > - It is simple to implement. Plug the texture xxxTexture into the
> > >>> > shader at the same place where you use xxxFactor.
> > >>> >
> > >>> > Compare this to the current approach of X3D:
> > >>>
> > >>> Perhaps somebody has insight into the underlying reasoning behind the current approach.
> > >>>
> > >>> In a way, the new Materials are on the level of the current Appearance.
> > >>>
> > >>> Structurally, the main question may be what works best for reuse of Appearance, Material and Texture.
> > >>>
> > >>> Currently, reusing Material gives you the option to use a different diffuse texture for the same set colors. This does not seem to be a very common requirement. With the new design this flexibility disappears since colors (as factors) and existing maps are reused. This is the common requirement.
> > >>>
> > >>> > - The lighting equations say to use the Apperance.texture for diffuse,
> > >>> > http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/lighting.html#Lightingmodel
> > >>> > . (And to treat RGB and grayscale textures differently, which is
> > >>> > another point I find bad -- RGB textures should "replace" by default,
> > >>> > while RGB should "modulate", according to the spec.)
> > >>> >
> > >>> > - The multi-texturing specification says that when MultiTexture is
> > >>> > present, the "MultiTexture.mode" rules (and the default is "modulate",
> > >>> > regardless of RGB or grayscale,
> > >>> > http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/texturing.html#MultiTexture
> > >>> > ). And the "MultiTexture.source" determines whether the texture
> > >>> > affects the diffuse or specular calculation.
> > >>> >
> > >>> > There are a couple of inconsistencies here. And it's not implemented
> > >>> > completely by X3D browsers, as far as I tested.
> > >>> >
> > >>> > And it still doesn't provide all the necessary flexibility. E.g. you
> > >>> > can only modify "diffuse" and "specular" by "MultiTexture.source", you
> > >>> > cannot modify "emissive" color, and it's unclear what modifies the
> > >>> > "transparency". Although some of these things could be easily fixed in
> > >>> > the multi-texturing spec, but I think that overall the approach of
> > >>> > PhysicalMaterial/CommonSurfaceShader is better.
> > >>> >
> > >>> > I discussed some of the above in """How does this relate to the
> > >>> > existing X3D multi-texturing nodes?""" on
> > >>> > https://github.com/michaliskambi/x3d-tests/wiki/How-to-add-PBR-to-X3D%3F
> > >>> > . In short, I think we have to keep "Appearance.texture" supported,
> > >>> > and when it's present it should override "Material.diffuseTexture" or
> > >>> > "PhysicalMaterial.baseColorTexture" (depending whether
> > >>> > "Appearance.material" is Material or PhysicalMaterial). But new models
> > >>> > should be advised to instead add textures inside Material or
> > >>> > PhysicalMaterial.
> > >>> >
> > >>> > And the normalmap texture (that is not related to lighting equations)
> > >>> > should be specified outside of the material, in "Appearance.normalMap"
> > >>> > field. This is also consistent with glTF, that places "normalTexture"
> > >>> > outside of the "pbrMetallicRoughness" block, recognizing that
> > >>> > normalmaps make sense for all lighting models, since "varying normal
> > >>> > vectors over a surface" makes sense for all algorithms that look at
> > >>> > surface normals. CGE has "Appearance.normalMap" already.
> > >>>
> > >>> Although I agree that normal maps are generated independently of the lighting model, the use of a particular normal map is strongly coupled to use of a corresponding set of other maps. I do not see a situation where you would want to reuse a PhysicalMaterial and its maps with a different normal map. In fact, reusing a PhysicalMaterial should not require repeating the use of the same normal map (outside of PhysicalMaterial).
> > >>>
> > >>> This does not leave much space for Appearance to act in, really only TextureTransform I believe. This may be appropriate as variations in TextureTransforms produces variations in the appearance of the same PBR material.
> > >>>
> > >>> We may also have to discuss the option to use more than one set of texture coordinates for the same geometry, for different maps. I think glTF requires that two UV sets are supported. I think x3dom supports this for glTF but may not expose it for X3D. Let me check.
> > >>>
> > >>> Cheers,
> > >>>
> > >>> Andreas
> > >>>
> > >>> >
> > >>> > Sorry for a long train of thought :) Hopefully this is informative.
> > >>> >
> > >>> > Regards,
> > >>> > Michalis
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Andreas Plesch
> > >>> Waltham, MA 02453
> >
> >
> >
> > --
> > Andreas Plesch
> > Waltham, MA 02453



-- 
Andreas Plesch
Waltham, MA 02453



More information about the x3d-public mailing list