[x3d-public] New volume rendering extension proposals

[Ander Arbelaiz]

Dear X3D WG members,

I would like to propose a series of extensions/additions to the X3D Volume
Rendering Component.

These extensions were first presented on the Web3D2017 conference.
Community-driven Extensions to the X3D Volume Rendering Component
<https://www.researchgate.net/publication/317122169_Community-driven_extensions_to_the_X3D_volume_rendering_component>
(full
PDF copy available)

On this paper the following additions were proposed:

1. New Proposed Nodes:
1.1 ImageTextureAtlas node
1.2 MPRVolumeStyle node

2. New Proposed Additions:
2.1 quality attribute to target device computation capabilities
2.2 allowViewpointInside attribute for inside exploration of volume data
2.3 sceneDepth attribute for hybrid rendering
2.4 cap attribute for Clipping

So far, from those proposals I have already contribute in the X3DOM library
with the following:


*1.1 ImageTextureAtlas node*
*1.2 MPRVolumeStyle*
*2.2 allowViewpointInside attribute for inside exploration of volume data*

Therefore, I think these additions (1,1, 1.2, 2.2) can be discussed for
inclusion in future revisions of the X3D standard.

I will prepare some public live examples as soon as I can, and will post
them in the mailing list.

I understand a more formal spec definition will be required. @Don Brutzman
<brutzman at nps.edu> @Nicholas Polys <npolys at vt.edu>,  any help and guidance
on this regard will be appreciated. When required I could present/discuss
these extensions on any next X3D WG meeting.

An initial description of these extensions is at the end of this post.

BR,



1.1 ImageTextureAtlas node
Volumetric data, specially those obtained from a MRI or a CT scan, can be
seen as a set of 2D image slices in an array. This data is usually stored
in a 3D Texture. Unfortunately, in the web context, WebGL 1.0 does not
support 3D texture data types. X3DTexture3DNode derived nodes can not be
implemented with this API. An alternative structure is required for
compatibility with the web.

We propose a new ImageTextureAtlas node which allows to represent the 3D
volume data by composing all the 2D slices into a single 2D texture.
Instead of adding a Z dimension to the texture, all 2D slices are arranged
in a matrix configuration into one image.

Listing below shows the basic X3D definition of the proposed node.

ImageTextureAtlas : X3DTexture2DNode

SFNode [in,out] metadata NULL [X3DMetadataObject]

SFBool [in,out] repeatS TRUE

SFBool [in,out] repeatT TRUE

SFNode [in,out] textureProperties NULL [TextureProperties]

MFString [in,out] url [] [URI]

SFInt32 [in,out] numberOfSlices 0 [0,∞)

SFInt32 [in,out] slicesOverX 0 [0,∞)

SFInt32 [in,out] slicesOverY 0 [0,∞)

SFBool [in,out] hideChildren TRUE

SFString [in,out] channels "R"

SFString [in,out] sortOrder "NONE" ["ROW", "CHANNEL", “NONE”]

}

The following attributes are inherited by X3DTexture2DNode: metadata,
repeatS, repeatT, textureProperties and url.

The numberOfSlices attribute indicates the number of slices or the
dimension in Z axis. SlicesOverX attribute indicates the number of slices
along X axis or columns, while the SlicesOverY indicates the number of
slices along Y axis or number of rows in the matrix configuration. This
values must be provided by the user, since they cannot be deduced by the
input image.

The channels field defines in which color channel of the texture volume
data is being stored. The default behavior is to store the volume data in
the Red color channel by specifying the "R" value. When, a larger volume is
required to be converted into an ImageTextureAtlas additional channels can
be specified with "R", "G", "B", "A" characters. For instance, to store up
to three times more data. A "RGBA" value in the channel field will indicate
that all color channels of the texture are being used to store the data.

Once multiple color channels are defined, the order in which 2D slices are
tiled into the atlas must be set in the sortOrder field. The default
behavior is to tile the 2D slices that represent the Z axis of the volume
data in "ROW” (row-major) order. For each slice of the ImageTextureAtlas
the next slice in the Z axis is the contiguous slice in the row of the
matrix of slices. When the sortOrder is set to "CHANNEL" the next slice of
the atlas in the Z axis direction is stored in the the contiguous color
channel.

A 2D image representing an ImageTextureAtlas of the MRI ventricles (Volvis)
dataset (color inverted and contrast enhanced) as a set of 2D slices tiled
into a matrix configuration

Not only to represent raw volume data, the ImageTextureAtlas can also be
used for gradient volume data and segmented volume data.

Surface normals are specified to be provided as an additional 3D texture.
Using the same approach as before, we can create an ImageTextureAtlas using
the color channels of the 2D texture to store the vector information of
gradient data. We encode the gradient vector for each pixel in the atlas in
the RGB color channels {R: X, G: Y, B: Z}.

Listing below shows an example declaration of the gradient data.

<ImageTextureAtlas containerField="surfaceNormals" url="gradient.png"
slicesOverX="8" slicesOverY="8" channels=”RGB” sortOrger=”NONE”>
</ImageTextureAtlas>

The containerField and URL attributes are the only modifications required
for the ImageTextureAtlas declaration. The containerField attribute is used
to target the volume data voxels field in a X3DVolumeDataNode or gradient
data surfaceNormals field in a X3DVolumeStyleNode. This also requires, for
the X3DVolumeStyleNode type nodes with a surfaceNormals attribute, to
accept a X3DTexture2DNode as an input argument.

The following Listing definition shows how X3DVolumeData derived nodes,
such as the VolumeData, should allow a 2D texture input to accept the
ImageTextureAtlas as a parameter.

VolumeData : X3DVolumeDataNode {

SFVec3f [in,out] dimensions 1 1 1 (0,∞)

SFNode [in,out] metadata NULL [X3DMetadataObject]

SFNode [in,out] renderStyle NULL [X3DVolumeRenderStyleNode]

SFNode [in,out] voxels NULL [X3DTexture3DNode, X3DTexture2DNode]

SFVec3f [] bboxCenter 0 0 0 (-∞,∞)

SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1

}

The SegmentedVolumeData node allows the user to discern regions of the
volume and apply different rendering styles to each one. The segmented
region must be labeled per voxel.

Textures provide a mechanism to input data to the GPU, updating directly
the visualization. In the Web platform we can take advantage of the 2D
canvas API to create or modify images that can be copied to the GPU. This
is of special interest to reflect dynamic changes in the data and, for
instance, to perform the construction of the atlas in the browser.

Listing below shows an ImageTextureAtlas node declaration with a canvas.
The hideChildren attribute hides the atlas image visibility from the user,
but makes it available to be modified with JavaScript.

<ImageTextureAtlas containerField="voxels" url="" slicesOverX="8"
slicesOverY="8" hideChildren="true">

 <canvas id="v" style="width:2048px; height:2048px;">

 </canvas>

</ImageTextureAtlas>
1.2 MPRVolumeStyle node
The Multi-Planar Reconstruction (MPR) is a wide spread rendering technique.
Essentially, it enables the user to define arbitrary planes slicing through
the volume data. The rendering algorithm resamples the volume data to
reconstruct the desired plane. Usually, the following planes are defined:
Axial, Sagital, Coronal and Oblique.

The following Listing shows the proposed X3D MPR volume rendering style
definition.

MPRVolumeStyle : X3DVolumeRenderStyleNode  {

SFBool [in,out] enabled TRUE

SFNode [in,out] metadata NULL [X3DMetadataObject]

SFNode [in,out] transferFunction NULL [X3DTexture2DNode, X3DTexture3DNode]

SFBool [in,out] forceOpaque TRUE

MFNode [in,out] planes NULL [MPRPlane]

}

An additional interesting enhancement to MPR is the ability to add a
transfer function (TF) in order to illustrate or filter regions in the
reconstructed planes. It is used to correlate the visual output with
obtained results. The proposed MPR node defines a *transferFunction *attribute
with the same functionality as the one already defined in the
OpacityMapVolumeStyle. This node also defines a planes attribute to allow
the user to declare arbitrary planes.

The following Listing presents an arbitrary plane definition for the
MPRVolumeStyle node. The *normal *attribute defines the normal vector of
the plane, while the *position *attribute establishes the position of the
plane from the origin of the volume in the normal direction.

MPRPlane : X3DNode {

SFBool [in,out] enabled TRUE

SFNode [in,out] metadata NULL [X3DMetadataObject]

SFVec3 [in,out] normal 0 0 1

SFFloat [in,out] position 0.0 [0,1]

}
2.2 allowViewpointInside attribute for inside explorationAllowing to place
the viewers virtual camera inside the volume provides a new perspective to
analyze the data. Some regions of the volume data can be occluded even
after filtering the data with a transfer function (TF). We suggest to
define a new attribute to enable or disable this behavior for performance
purposes.

The following Listing shows the definition with the proposed attribute.

X3DVolumeDataNode : X3DChildNode, X3DBoundedObject

SFVec3f [in,out] dimensions 1 1 1 (0,∞)

SFNode [in,out] metadata NULL [X3DMetadataObject]

SFVec3f [] bboxCenter  0 0 0 (-∞,∞)

SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1

SFBool [in,out] allowViewPointInside TRUE

}

Our proposal is to declare a new boolean attribute allowViewPointInside to
all nodes inherited by the X3DVolumeDataNode. By default we suggest to
allow the inspection of the volume data and if the user does not require to
do so, it can be explicitly disabled. This feature allows to avoid
additional computation.

-- 
<http://www.vicomtech.org>
Ander Arbelaiz Aranzasti
Researcher | Investigador

aarbelaiz at vicomtech.org
+[34] 943 30 92 30Industry and Advanced Manufacturing | Industria y
Fabricación Avanzada

<https://www.linkedin.com/company/vicomtech>
<https://www.youtube.com/user/VICOMTech>  <https://twitter.com/@Vicomtech>
member of:  <http://www.graphicsmedia.net/>
Legal Notice - Privacy policy <http://www.vicomtech.org/en/proteccion-datos>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190612/8934ab8c/attachment-0001.html>