[x3d-public] Need for colorspaceConversion field in ImageTexture, glTF compatibility

Tue Aug 29 10:53:20 PDT 2023

Following up after some work:

On Thu, Aug 24, 2023 at 11:15 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
>
> On Thu, Aug 24, 2023 at 9:14 AM Michalis Kamburelis
> <michalis.kambi at gmail.com> wrote:
> >
> > 1. As for colorspaceConversion:
> >
> > I don't think we want to avoid adding X3D field for this :)
>
> Well, it is worth thinking through options if it means smarter
> solutions can be discovered.
>
> > I'm all after adding X3D field to account for this -- that is, say
> > explicitly whether ICC profile should be honored or not, so that all
> > cases are satisfied. I'm cool with colorspaceConversion name.
>
...snip

After some thought I do think that a new field is needed for
ImageTexture (and MovieTexture, and Background ?).
"colorspaceConversion" or "colorSpaceConversion" or perhaps something
shorter like "colorProfile"  ?

x3dom will probably use the webgl texture unpacking parameter.

> > 2. As for attenuation and spot cutoff: Indeed glTF math is a bit different.

..snip

I have implemented the gltf punctual light extension for x3dom. For
now, it just uses the gltf angular attenuation math for all
PhysicalMaterial shapes.

See https://github.com/x3dom/x3dom/issues/1259 for notes and details.

But I do think that the smoother transition on the outer edge of the
spotlight will be attractive for shapes with regular material as well,
and we do not want to have different lighting for the same spot light
in scenes where both material types are used (although this should not
be recommended anyways). At the same time there is a need for backward
compatibility.

To keep things as simple as possible, a boolean "smoothDropoff" or
"punctual", or a SFString "dropoffMode" field for spot light is
needed. Since there are already at least three modes of angular
attenuation (x3d, gltf spec., "smoothstep") which are relevant,
"dropoffMode" seems most suitable.

Valid values would be one of  [LINEAR, COSINESQ, SMOOTHSTEP] with
LINEAR the default. Invalid values would cause fallback to LINEAR or
lead to browser defined behaviour.

Since X3D reserves the word "attenuation" for distance attenuation, I
thought of "dropoffMode" as dropoff is the word used in the spec.
currently but there may be better options.

For distance attenuation, I second Michalis' proposal to sacrifice a
small bit of visual backward compatibility in exchange for simplicity
by adopting the glTF numerator term in general, after testing the XD
default case of radius=100 which is probably meant to be a large value
relative to the scene size. For example, I would not expect any
noticeable visual impact on the vast majority of web3d example scenes.

Andreas

> Andreas
>
> > Regards,
> > Michalis
> >
> >
> > śr., 23 sie 2023 o 18:40 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> > >
> > > If we prefer to avoid introducing a new field, another idea may be to
> > > just require in the spec. that embedded color/gamma profiles must be
> > > ignored in case a texture image is used for one of the
> > > PhysicalMaterial texture fields:
> > >
> > > https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-IS.proof/Part01/components/shape.html#PhysicalMaterial
> > >
> > > How regular Material is dealing with such color profiles in textures
> > > could be left an open, unaddressed question, if such ambiguity keeps
> > > being acceptable.
> > >
> > > Another rendering detail for glTF compatibility is the slightly
> > > different ways X3D and glTF define light attenuation away from a point
> > > or spot light. glTF can include such lights if a corresponding glTF
> > > extension is supported, and since X3D already has such lights it is
> > > pretty straightforward to support this glTF extension.
> > > Distance attenuation in glTF corresponds to an X3D attenuation field
> > > value of [0 0 1] for the most part but has an additional term which
> > > makes the transition very close to the radius cutoff smooth while in
> > > X3D it is a bit more abrupt.
> > > Angular attenuation for spot lights between the full beam and the
> > > outer cutoff angle is proportional to the angle differences in X3D
> > > while in glTF it is proportional to the difference of the cosine of
> > > the angles. This also produces a smooth transition at the outer edge
> > > rather than the  sharper boundary in X3D.
> > > These attenuation rules are fixed for X3D and so needing to use them
> > > leads to slight incompatibilities for glTF renderings.
> > > From a technical viewpoint, since for glTF in X3D different shaders
> > > are needed anyways due to PBR, these shaders could also follow the
> > > glTF attenuation rules.
> > > This would mean in effect that if Lights are used with a
> > > PhysicalMaterial, a second set of attenuation rules would be followed.
> > > I am considering doing this for x3dom.
> > > Another option would be to have a new field for Lights such as
> > > "attenuationMode" [classic, glTF] which allows for choosing between
> > > these behaviours. A sharper, more dramatic outer boundary for spot
> > > lights is needed sometimes for PBR as well.
> > >
> > > Andreas
> > >
> > > On Wed, Aug 23, 2023 at 10:39 AM Michalis Kamburelis
> > > <michalis.kambi at gmail.com> wrote:
> > > >
> > > > 1. I agree it would be nice to add an X3D extension to honor the ICC profiles.
> > > >
> > > >     I understand that the functional outcome should be:
> > > >
> > > >     - By default it should imply "honor embedded ICC profiles".
> > > >
> > > >     - But when we convert content from glTF -> X3D, converters could
> > > > set it to "do not honor ICC color profiles".
> > > >
> > > >     Then everyone is happy -- we can honor ICC profiles in X3D, but we
> > > > can also follow glTF spec.
> > > >
> > > > 2. As for CGE/view3dscene:
> > > >
> > > > We simply for now do not support ICC profiles embedded in PNG or JPG
> > > > images. We ignore them when reading PNG from ImageTexture. We ignore
> > > > them when converting textures embedded in glTF (which may be embedded
> > > > PNG image with embedded ICC profiles) into PixelTexture.
> > > >
> > > > So we follow glTF spec literally, but that's just our priorities and
> > > > our current implementation. I agree that X3D in general should allow
> > > > to use ICC profiles from PNG or JPG.
> > > >
> > > > To explain more:
> > > >
> > > > CGE/view3dscene converts textures from glTF into
> > > >
> > > > A. MovieTexture (if they have URI in glTF that has a movie extension, like .mp4)
> > > >
> > > > B. ImageTexture (if they have other URI in glTF, like .png or .jpg)
> > > >
> > > > C. PixelTexture (if they don't have URI in glTF, but content is
> > > > embedded in the glTF buffer)
> > > >
> > > > Indeed, C means that we decompress PNG -> simple PixelTexture, losing
> > > > ICC profile from PNG / JPG, if any. But for our implementation, even A
> > > > or B means that we ignore ICC profile, because we just ignore it
> > > > always.
> > > >
> > > > As for PixelTexture:
> > > >
> > > > Indeed the representation of PixelTexture (in classic or XML encoding
> > > > of X3D) is not efficient.
> > > >
> > > > But the in-memory representation of PixelTexture in CGE/view3dscene is
> > > > efficient, so for now we chose this way. Things are decompressed in
> > > > PixelTexture, but they need to be decompressed anyway to upload to GPU
> > > > (at least decompressed from PNG / JPG).
> > > >
> > > > In the future we will likely add an option to always write
> > > > PixelTexture (memory) -> into ImageTexture, using data URI (
> > > > https://castle-engine.io/manual_network.php#section_data ) to embed
> > > > the contents as PNG. This would achieve optimal (simple) in-memory
> > > > representation and efficient encoding in X3D files too. It would make
> > > > sense both for converting glTF->X3D and for optimizing X3D files (e.g.
> > > > making X3D->X3D conversion).
> > > >
> > > > Regards,
> > > > Michalis
> > > >
> > > >
> > > >
> > > > wt., 22 sie 2023 o 03:09 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> > > > >
> > > > > I found that Castle actually ignores embedded color profiles in
> > > > > texture images in glTFs as required and sidesteps the problem by
> > > > > generating new PixelTextures from these images.
> > > > >
> > > > > This works in principle but is not practical for large textures which
> > > > > today can have 4kx4k pixels, 16 * (10+1) MCharacters = 170MBytes as
> > > > > PixelTexture.
> > > > >
> > > > > I guess one could recompress these raw images to say png. If an X3D
> > > > > optimizer could do that it might be a viable solution.
> > > > >
> > > > > But a flag field to turn off color space conversion from embedded ICC
> > > > > profiles would be more elegant and very useful for custom shaders
> > > > > which expect consistent input.
> > > > >
> > > > > -Andreas
> > > > >
> > > > > > Date: Mon, 21 Aug 2023 01:22:59 +0200
> > > > > > From: Holger Seelig <holger.seelig at yahoo.de>
> > > > > > To: X3D <x3d-public at web3d.org>
> > > > > > Subject: [x3d-public] Need for colorspaceConversion field in
> > > > > >         ImageTexture
> > > > > >
> > > > > > There is still the problem that the issue of ICC color profiles is not addressed here, although it is an urgent issue. For people who have not yet dealt with this problem, it may seem like it is not important, but it is of interest to professional graphic designers.
> > > > > >
> > > > > > When it comes to texturing, there is nothing about ICC color profiles in the X3D spec:
> > > > > >
> > > > > > https://www.google.com/search?client=firefox-b-d&q=ICC+site%3Ahttps%3A%2F%2Fwww.web3d.org%2Fdocuments%2Fspecifications%2F19775-1%2FV4.0%2FPart01%2F
> > > > > >
> > > > > > But in glTF spec, they are very aware of that:
> > > > > >
> > > > > > > ... images SHOULD NOT contain ... ICC profiles. Such features, if present, MUST be ignored by client implementations.
> > > > > >
> > > > > > But we in X3D do, and the problem occurs if we want to try to convert glTF to X3D, which is possible and straightforward, but we need options to do this in the right way, and in a way everyone is satisfied.
> > > > > >
> > > > > > Best regards,
> > > > > > Holger from LE
> > > > >
> > > > > --
> > > > > Andreas Plesch
> > > > > Waltham, MA 02453
> > > > >
> > > > > _______________________________________________
> > > > > x3d-public mailing list
> > > > > x3d-public at web3d.org
> > > > > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> > >
> > >
> > >
> > > --
> > > Andreas Plesch
> > > Waltham, MA 02453
>
>
>
> --
> Andreas Plesch
> Waltham, MA 02453

--
Andreas Plesch
Waltham, MA 02453