[x3d-public] New volume rendering extension proposals

[Ander Arbelaiz]

As it is defined, I am not sure if it could be applied in other use cases.
Sprite atlases use a very similar approach but they do not use a regular
grid.... But hopefully, yes there could be another potential use case for
this structure.

Yes, in X3DOM is defined in the texturing component. It is a 2D texture
which emulates a 3D texture, but a 2D texture nonetheless. It inherits the
2D texture properties.

BR,


El mar., 2 jul. 2019 a las 1:41, Brutzman, Donald (Don) (CIV) (<
brutzman at nps.edu>) escribió:

> New question: Dick and I are looking at how to best integrate
> ImageTextureAtlas into X3Dv4 Specification.
>
> Wondering, does it have uses besides volume rendering?  Perhaps it is more
> appropriate to put this in Texturing component?
>
> For example, that is where X3DOM puts it:
>
>         X3DOM documentation: ImageTextureAtlas
>         https://doc.x3dom.org/author/Texturing/ImageTextureAtlas.html
>
>
>
> On 6/12/2019 2:18 AM, Ander Arbelaiz wrote:
> > Dear X3D WG members,
> >
> > I would like to propose a series of extensions/additions to the X3D
> Volume Rendering Component.
> >
> > These extensions were first presented on the Web3D2017 conference.
> > Community-driven Extensions to the X3D Volume Rendering Component <
> https://www.researchgate.net/publication/317122169_Community-driven_extensions_to_the_X3D_volume_rendering_component>(full
> PDF copy available)
> >
> > On this paper the following additions were proposed:
> >
> > 1. New Proposed Nodes:
> > 1.1 ImageTextureAtlas node
> > 1.2 MPRVolumeStyle node
> >
> > 2. New Proposed Additions:
> > 2.1 quality attribute to target device computation capabilities
> > 2.2 allowViewpointInside attribute for inside exploration of volume data
> > 2.3 sceneDepth attribute for hybrid rendering
> > 2.4 cap attribute for Clipping
> >
> > So far, from those proposals I have already contribute in the X3DOM
> library with the following:
> >
> > *1.1 ImageTextureAtlas node
> > *
> > *1.2 MPRVolumeStyle*
> > *2.2 allowViewpointInside attribute for inside exploration of volume
> data*
> >
> > Therefore, I think these additions (1,1, 1.2, 2.2) can be discussed for
> inclusion in future revisions of the X3D standard.
> >
> > I will prepare some public live examples as soon as I can, and will post
> them in the mailing list.
> >
> > I understand a more formal spec definition will be required. @Don
> Brutzman <mailto:brutzman at nps.edu> @Nicholas Polys <mailto:npolys at vt.edu>,
> any help and guidance on this regard will be appreciated. When required I
> could present/discuss these extensions on any next X3D WG meeting.
> >
> > An initial description of these extensions is at the end of this post.
> >
> > BR,
> >
> >
> >
> >     1.1 ImageTextureAtlas node
> >
> > Volumetric data, specially those obtained from a MRI or a CT scan, can
> be seen as a set of 2D image slices in an array. This data is usually
> stored in a 3D Texture. Unfortunately, in the web context, WebGL 1.0 does
> not support 3D texture data types. X3DTexture3DNode derived nodes can not
> be implemented with this API. An alternative structure is required for
> compatibility with the web.
> >
> > We propose a new ImageTextureAtlas node which allows to represent the 3D
> volume data by composing all the 2D slices into a single 2D texture.
> Instead of adding a Z dimension to the texture, all 2D slices are arranged
> in a matrix configuration into one image.
> >
> > Listing below shows the basic X3D definition of the proposed node.
> >
> > ImageTextureAtlas : X3DTexture2DNode
> >
> > SFNode [in,out] metadata NULL [X3DMetadataObject]
> >
> > SFBool [in,out] repeatS TRUE
> >
> > SFBool [in,out] repeatT TRUE
> >
> > SFNode [in,out] textureProperties NULL [TextureProperties]
> >
> > MFString [in,out] url [] [URI]
> >
> > SFInt32 [in,out] numberOfSlices 0 [0,∞)
> >
> > SFInt32 [in,out] slicesOverX 0 [0,∞)
> >
> > SFInt32 [in,out] slicesOverY 0 [0,∞)
> >
> > SFBool [in,out] hideChildren TRUE
> >
> > SFString [in,out] channels "R"
> >
> > SFString [in,out] sortOrder "NONE" ["ROW", "CHANNEL", “NONE”]
> >
> > }
> >
> >
> > The following attributes are inherited by X3DTexture2DNode: metadata,
> repeatS, repeatT, textureProperties and url.
> >
> > The numberOfSlices attribute indicates the number of slices or the
> dimension in Z axis. SlicesOverX attribute indicates the number of slices
> along X axis or columns, while the SlicesOverY indicates the number of
> slices along Y axis or number of rows in the matrix configuration. This
> values must be provided by the user, since they cannot be deduced by the
> input image.
> >
> > The channels field defines in which color channel of the texture volume
> data is being stored. The default behavior is to store the volume data in
> the Red color channel by specifying the "R" value. When, a larger volume is
> required to be converted into an ImageTextureAtlas additional channels can
> be specified with "R", "G", "B", "A" characters. For instance, to store up
> to three times more data. A "RGBA" value in the channel field will indicate
> that all color channels of the texture are being used to store the data.
> >
> > Once multiple color channels are defined, the order in which 2D slices
> are tiled into the atlas must be set in the sortOrder field. The default
> behavior is to tile the 2D slices that represent the Z axis of the volume
> data in "ROW” (row-major) order. For each slice of the ImageTextureAtlas
> the next slice in the Z axis is the contiguous slice in the row of the
> matrix of slices. When the sortOrder is set to "CHANNEL" the next slice of
> the atlas in the Z axis direction is stored in the the contiguous color
> channel.
> >
> > A 2D image representing an ImageTextureAtlas of the MRI ventricles
> (Volvis)dataset (color inverted and contrast enhanced) as a set of 2D
> slices tiled into a matrix configuration
> >
> >
> > Not only to represent raw volume data, the ImageTextureAtlas can also be
> used for gradient volume data and segmented volume data.
> >
> > Surface normals are specified to be provided as an additional 3D
> texture. Using the same approach as before, we can create an
> ImageTextureAtlas using the color channels of the 2D texture to store the
> vector information of gradient data. We encode the gradient vector for each
> pixel in the atlas in the RGB color channels {R: X, G: Y, B: Z}.
> >
> > Listing below shows an example declaration of the gradient data.
> >
> > <ImageTextureAtlas containerField="surfaceNormals" url="gradient.png"
> slicesOverX="8" slicesOverY="8" channels=”RGB” sortOrger=”NONE”>
> </ImageTextureAtlas>
> >
> >
> > The containerField and URL attributes are the only modifications
> required for the ImageTextureAtlas declaration. The containerField
> attribute is used to target the volume data voxels field in a
> X3DVolumeDataNode or gradient data surfaceNormals field in a
> X3DVolumeStyleNode. This also requires, for the X3DVolumeStyleNode type
> nodes with a surfaceNormals attribute, to accept a X3DTexture2DNode as an
> input argument.
> >
> > The following Listing definition shows how X3DVolumeData derived nodes,
> such as the VolumeData, should allow a 2D texture input to accept the
> ImageTextureAtlas as a parameter.
> >
> > VolumeData : X3DVolumeDataNode {
> >
> > SFVec3f [in,out] dimensions 1 1 1 (0,∞)
> >
> > SFNode [in,out] metadata NULL [X3DMetadataObject]
> >
> > SFNode [in,out] renderStyle NULL [X3DVolumeRenderStyleNode]
> >
> > SFNode [in,out] voxels NULL [X3DTexture3DNode, X3DTexture2DNode]
> >
> > SFVec3f [] bboxCenter 0 0 0 (-∞,∞)
> >
> > SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1
> >
> > }
> >
> >
> > The SegmentedVolumeData node allows the user to discern regions of the
> volume and apply different rendering styles to each one. The segmented
> region must be labeled per voxel.
> >
> > Textures provide a mechanism to input data to the GPU, updating directly
> the visualization. In the Web platform we can take advantage of the 2D
> canvas API to create or modify images that can be copied to the GPU. This
> is of special interest to reflect dynamic changes in the data and, for
> instance, to perform the construction of the atlas in the browser.
> >
> > Listing below shows an ImageTextureAtlas node declaration with a canvas.
> The hideChildren attribute hides the atlas image visibility from the user,
> but makes it available to be modified with JavaScript.
> >
> > <ImageTextureAtlas containerField="voxels" url="" slicesOverX="8"
> slicesOverY="8" hideChildren="true">
> >
> >   <canvas id="v" style="width:2048px; height:2048px;">
> >
> >   </canvas>
> >
> > </ImageTextureAtlas>
> >
> >
> >     1.2 MPRVolumeStyle node
> >
> > The Multi-Planar Reconstruction (MPR) is a wide spread rendering
> technique. Essentially, it enables the user to define arbitrary planes
> slicing through the volume data. The rendering algorithm resamples the
> volume data to reconstruct the desired plane. Usually, the following planes
> are defined: Axial, Sagital, Coronal and Oblique.
> >
> > The following Listing shows the proposed X3D MPR volume rendering style
> definition.
> >
> > MPRVolumeStyle : X3DVolumeRenderStyleNode  {
> >
> > SFBool [in,out] enabled TRUE
> >
> > SFNode [in,out] metadata NULL [X3DMetadataObject]
> >
> > SFNode [in,out] transferFunction NULL [X3DTexture2DNode,
> X3DTexture3DNode]
> >
> > SFBool [in,out] forceOpaque TRUE
> >
> > MFNode [in,out] planes NULL [MPRPlane]
> >
> > }
> >
> >
> > An additional interesting enhancement to MPR is the ability to add a
> transfer function (TF) in order to illustrate or filter regions in the
> reconstructed planes. It is used to correlate the visual output with
> obtained results. The proposed MPR node defines a /transferFunction
> /attribute with the same functionality as the one already defined in the
> OpacityMapVolumeStyle. This node also defines a planes attribute to allow
> the user to declare arbitrary planes.
> >
> > The following Listing presents an arbitrary plane definition for the
> MPRVolumeStyle node. The /normal /attribute defines the normal vector of
> the plane, while the /position /attribute establishes the position of the
> plane from the origin of the volume in the normal direction.
> >
> > MPRPlane : X3DNode {
> >
> > SFBool [in,out] enabled TRUE
> >
> > SFNode [in,out] metadata NULL [X3DMetadataObject]
> >
> > SFVec3 [in,out] normal 0 0 1
> >
> > SFFloat [in,out] position 0.0 [0,1]
> >
> > }
> >
> >
> >     2.2 allowViewpointInside attribute for inside exploration
> >
> > Allowing to place the viewers virtual camera inside the volume provides
> a new perspective to analyze the data. Some regions of the volume data can
> be occluded even after filtering the data with a transfer function (TF). We
> suggest to define a new attribute to enable or disable this behavior for
> performance purposes.
> >
> > The following Listing shows the definition with the proposed attribute.
> >
> > X3DVolumeDataNode : X3DChildNode, X3DBoundedObject
> >
> > SFVec3f [in,out] dimensions 1 1 1 (0,∞)
> >
> > SFNode [in,out] metadata NULL [X3DMetadataObject]
> >
> > SFVec3f [] bboxCenter  0 0 0 (-∞,∞)
> >
> > SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1
> >
> > SFBool [in,out] allowViewPointInside TRUE
> >
> > }
> >
> >
> > Our proposal is to declare a new boolean attribute allowViewPointInside
> to all nodes inherited by the X3DVolumeDataNode. By default we suggest to
> allow the inspection of the volume data and if the user does not require to
> do so, it can be explicitly disabled. This feature allows to avoid
> additional computation.
> >
> > --
> > <http://www.vicomtech.org>
> > Ander Arbelaiz Aranzasti
> > Researcher | Investigador
> >
> > aarbelaiz at vicomtech.org <mailto:aarbelaiz at vicomtech.org>
> > +[34] 943 30 92 30Industry and Advanced Manufacturing | Industria y
> Fabricación Avanzada
> >
> > <https://www.linkedin.com/company/vicomtech> <
> https://www.youtube.com/user/VICOMTech> <https://twitter.com/@Vicomtech>
> > member of: <http://www.graphicsmedia.net/>
> > Legal Notice - Privacy policy <
> http://www.vicomtech.org/en/proteccion-datos>
>
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics
> http://faculty.nps.edu/brutzman
>


-- 
<http://www.vicomtech.org>
Ander Arbelaiz Aranzasti
Researcher | Investigador

aarbelaiz at vicomtech.org
+[34] 943 30 92 30Industry and Advanced Manufacturing | Industria y
Fabricación Avanzada

<https://www.linkedin.com/company/vicomtech>
<https://www.youtube.com/user/VICOMTech>  <https://twitter.com/@Vicomtech>
member of:  <http://www.graphicsmedia.net/>
Legal Notice - Privacy policy <http://www.vicomtech.org/en/proteccion-datos>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190708/ccb75e90/attachment-0001.html>