[x3d-public] Need for colorspaceConversion field in ImageTexture, glTF compatibility

[Andreas Plesch]

If we prefer to avoid introducing a new field, another idea may be to
just require in the spec. that embedded color/gamma profiles must be
ignored in case a texture image is used for one of the
PhysicalMaterial texture fields:

https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-IS.proof/Part01/components/shape.html#PhysicalMaterial

How regular Material is dealing with such color profiles in textures
could be left an open, unaddressed question, if such ambiguity keeps
being acceptable.

Another rendering detail for glTF compatibility is the slightly
different ways X3D and glTF define light attenuation away from a point
or spot light. glTF can include such lights if a corresponding glTF
extension is supported, and since X3D already has such lights it is
pretty straightforward to support this glTF extension.
Distance attenuation in glTF corresponds to an X3D attenuation field
value of [0 0 1] for the most part but has an additional term which
makes the transition very close to the radius cutoff smooth while in
X3D it is a bit more abrupt.
Angular attenuation for spot lights between the full beam and the
outer cutoff angle is proportional to the angle differences in X3D
while in glTF it is proportional to the difference of the cosine of
the angles. This also produces a smooth transition at the outer edge
rather than the  sharper boundary in X3D.
These attenuation rules are fixed for X3D and so needing to use them
leads to slight incompatibilities for glTF renderings.
>From a technical viewpoint, since for glTF in X3D different shaders
are needed anyways due to PBR, these shaders could also follow the
glTF attenuation rules.
This would mean in effect that if Lights are used with a
PhysicalMaterial, a second set of attenuation rules would be followed.
I am considering doing this for x3dom.
Another option would be to have a new field for Lights such as
"attenuationMode" [classic, glTF] which allows for choosing between
these behaviours. A sharper, more dramatic outer boundary for spot
lights is needed sometimes for PBR as well.

Andreas

On Wed, Aug 23, 2023 at 10:39 AM Michalis Kamburelis
<michalis.kambi at gmail.com> wrote:
>
> 1. I agree it would be nice to add an X3D extension to honor the ICC profiles.
>
>     I understand that the functional outcome should be:
>
>     - By default it should imply "honor embedded ICC profiles".
>
>     - But when we convert content from glTF -> X3D, converters could
> set it to "do not honor ICC color profiles".
>
>     Then everyone is happy -- we can honor ICC profiles in X3D, but we
> can also follow glTF spec.
>
> 2. As for CGE/view3dscene:
>
> We simply for now do not support ICC profiles embedded in PNG or JPG
> images. We ignore them when reading PNG from ImageTexture. We ignore
> them when converting textures embedded in glTF (which may be embedded
> PNG image with embedded ICC profiles) into PixelTexture.
>
> So we follow glTF spec literally, but that's just our priorities and
> our current implementation. I agree that X3D in general should allow
> to use ICC profiles from PNG or JPG.
>
> To explain more:
>
> CGE/view3dscene converts textures from glTF into
>
> A. MovieTexture (if they have URI in glTF that has a movie extension, like .mp4)
>
> B. ImageTexture (if they have other URI in glTF, like .png or .jpg)
>
> C. PixelTexture (if they don't have URI in glTF, but content is
> embedded in the glTF buffer)
>
> Indeed, C means that we decompress PNG -> simple PixelTexture, losing
> ICC profile from PNG / JPG, if any. But for our implementation, even A
> or B means that we ignore ICC profile, because we just ignore it
> always.
>
> As for PixelTexture:
>
> Indeed the representation of PixelTexture (in classic or XML encoding
> of X3D) is not efficient.
>
> But the in-memory representation of PixelTexture in CGE/view3dscene is
> efficient, so for now we chose this way. Things are decompressed in
> PixelTexture, but they need to be decompressed anyway to upload to GPU
> (at least decompressed from PNG / JPG).
>
> In the future we will likely add an option to always write
> PixelTexture (memory) -> into ImageTexture, using data URI (
> https://castle-engine.io/manual_network.php#section_data ) to embed
> the contents as PNG. This would achieve optimal (simple) in-memory
> representation and efficient encoding in X3D files too. It would make
> sense both for converting glTF->X3D and for optimizing X3D files (e.g.
> making X3D->X3D conversion).
>
> Regards,
> Michalis
>
>
>
> wt., 22 sie 2023 o 03:09 Andreas Plesch <andreasplesch at gmail.com> napisał(a):
> >
> > I found that Castle actually ignores embedded color profiles in
> > texture images in glTFs as required and sidesteps the problem by
> > generating new PixelTextures from these images.
> >
> > This works in principle but is not practical for large textures which
> > today can have 4kx4k pixels, 16 * (10+1) MCharacters = 170MBytes as
> > PixelTexture.
> >
> > I guess one could recompress these raw images to say png. If an X3D
> > optimizer could do that it might be a viable solution.
> >
> > But a flag field to turn off color space conversion from embedded ICC
> > profiles would be more elegant and very useful for custom shaders
> > which expect consistent input.
> >
> > -Andreas
> >
> > > Date: Mon, 21 Aug 2023 01:22:59 +0200
> > > From: Holger Seelig <holger.seelig at yahoo.de>
> > > To: X3D <x3d-public at web3d.org>
> > > Subject: [x3d-public] Need for colorspaceConversion field in
> > >         ImageTexture
> > >
> > > There is still the problem that the issue of ICC color profiles is not addressed here, although it is an urgent issue. For people who have not yet dealt with this problem, it may seem like it is not important, but it is of interest to professional graphic designers.
> > >
> > > When it comes to texturing, there is nothing about ICC color profiles in the X3D spec:
> > >
> > > https://www.google.com/search?client=firefox-b-d&q=ICC+site%3Ahttps%3A%2F%2Fwww.web3d.org%2Fdocuments%2Fspecifications%2F19775-1%2FV4.0%2FPart01%2F
> > >
> > > But in glTF spec, they are very aware of that:
> > >
> > > > ... images SHOULD NOT contain ... ICC profiles. Such features, if present, MUST be ignored by client implementations.
> > >
> > > But we in X3D do, and the problem occurs if we want to try to convert glTF to X3D, which is possible and straightforward, but we need options to do this in the right way, and in a way everyone is satisfied.
> > >
> > > Best regards,
> > > Holger from LE
> >
> > --
> > Andreas Plesch
> > Waltham, MA 02453
> >
> > _______________________________________________
> > x3d-public mailing list
> > x3d-public at web3d.org
> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org



-- 
Andreas Plesch
Waltham, MA 02453