[x3d-public] New volume rendering extension proposals

Thu Jun 13 05:37:38 PDT 2019

I can imagine something half-way between MPRVolumeStyle and
allowViewpointInside:
- a way to slide an MPR -like plane parallel to camera near clip plane in
the camera axis direction
-  to get an obtuse angle section without moving the camera
- for example to move a plane to the center of the volume,
- and then EXAMINE-navigate around the volume to see a variety of obtuse
angles
(but I have no opinion on how handy/useful that would be)
-Doug Sanden

On Thu, Jun 13, 2019 at 12:59 AM Ander Arbelaiz <aarbelaiz at vicomtech.org>
wrote:

> Yes, there is some relation to the DICOM Multi-Planar Reconstruction, they
> are referring to the same thing. But the proposed style is somewhat more
> limited and simpler than the DICOM specification.
>
> Some public live examples of the new proposals:
>
> *MPRVolumeStyle* node: https://sunrise-lunge.glitch.me
> *allowViewPointInside *attribute: https://mountain-verse.glitch.me
>
> I could join at both Fridays.
>
> BR,
>
> El mié., 12 jun. 2019 a las 16:45, Brutzman, Donald (Don) (CIV) (<
> brutzman at nps.edu>) escribió:
>
>> Ander, thank you very much.
>>
>> This is now added to X3Dv4 Implementations page.  Please review the
>> process summary which appears there.
>> https://www.web3d.org/x3dv4-implementations
>>
>> If you and Nicholas can join X3D Working Group teleconference this Friday
>> (08-0930 pacific) to present and discuss, that would be great.  Following
>> Friday is also OK We are just now transitioning to Zoom to permit screen
>> sharing and chat, that link will be forthcoming.
>>
>> Wondering if there is any relation to DICOM Multi-Planar Reconstruction
>> Geometry Module?
>>
>> ftp://dicom.nema.org/MEDICAL/dicom/2016a/output/chtml/part03/sect_C.11.26.html
>>
>> Thanks for your important efforts.
>>
>> On 6/12/2019 2:18 AM, Ander Arbelaiz wrote:
>> > Dear X3D WG members,
>> >
>> > I would like to propose a series of extensions/additions to the X3D
>> Volume Rendering Component.
>> >
>> > These extensions were first presented on the Web3D2017 conference.
>> > Community-driven Extensions to the X3D Volume Rendering Component <
>> https://www.researchgate.net/publication/317122169_Community-driven_extensions_to_the_X3D_volume_rendering_component>(full
>> PDF copy available)
>> >
>> > On this paper the following additions were proposed:
>> >
>> > 1. New Proposed Nodes:
>> > 1.1 ImageTextureAtlas node
>> > 1.2 MPRVolumeStyle node
>> >
>> > 2. New Proposed Additions:
>> > 2.1 quality attribute to target device computation capabilities
>> > 2.2 allowViewpointInside attribute for inside exploration of volume data
>> > 2.3 sceneDepth attribute for hybrid rendering
>> > 2.4 cap attribute for Clipping
>> >
>> > So far, from those proposals I have already contribute in the X3DOM
>> library with the following:
>> >
>> > *1.1 ImageTextureAtlas node
>> > *
>> > *1.2 MPRVolumeStyle*
>> > *2.2 allowViewpointInside attribute for inside exploration of volume
>> data*
>> >
>> > Therefore, I think these additions (1,1, 1.2, 2.2) can be discussed for
>> inclusion in future revisions of the X3D standard.
>> >
>> > I will prepare some public live examples as soon as I can, and will
>> post them in the mailing list.
>> >
>> > I understand a more formal spec definition will be required. @Don
>> Brutzman <mailto:brutzman at nps.edu> @Nicholas Polys <mailto:npolys at vt.edu>,
>> any help and guidance on this regard will be appreciated. When required I
>> could present/discuss these extensions on any next X3D WG meeting.
>> >
>> > An initial description of these extensions is at the end of this post.
>> >
>> > BR,
>> >
>> >
>> >
>> >     1.1 ImageTextureAtlas node
>> >
>> > Volumetric data, specially those obtained from a MRI or a CT scan, can
>> be seen as a set of 2D image slices in an array. This data is usually
>> stored in a 3D Texture. Unfortunately, in the web context, WebGL 1.0 does
>> not support 3D texture data types. X3DTexture3DNode derived nodes can not
>> be implemented with this API. An alternative structure is required for
>> compatibility with the web.
>> >
>> > We propose a new ImageTextureAtlas node which allows to represent the
>> 3D volume data by composing all the 2D slices into a single 2D texture.
>> Instead of adding a Z dimension to the texture, all 2D slices are arranged
>> in a matrix configuration into one image.
>> >
>> > Listing below shows the basic X3D definition of the proposed node.
>> >
>> > ImageTextureAtlas : X3DTexture2DNode
>> >
>> > SFNode [in,out] metadata NULL [X3DMetadataObject]
>> >
>> > SFBool [in,out] repeatS TRUE
>> >
>> > SFBool [in,out] repeatT TRUE
>> >
>> > SFNode [in,out] textureProperties NULL [TextureProperties]
>> >
>> > MFString [in,out] url [] [URI]
>> >
>> > SFInt32 [in,out] numberOfSlices 0 [0,∞)
>> >
>> > SFInt32 [in,out] slicesOverX 0 [0,∞)
>> >
>> > SFInt32 [in,out] slicesOverY 0 [0,∞)
>> >
>> > SFBool [in,out] hideChildren TRUE
>> >
>> > SFString [in,out] channels "R"
>> >
>> > SFString [in,out] sortOrder "NONE" ["ROW", "CHANNEL", “NONE”]
>> >
>> > }
>> >
>> >
>> > The following attributes are inherited by X3DTexture2DNode: metadata,
>> repeatS, repeatT, textureProperties and url.
>> >
>> > The numberOfSlices attribute indicates the number of slices or the
>> dimension in Z axis. SlicesOverX attribute indicates the number of slices
>> along X axis or columns, while the SlicesOverY indicates the number of
>> slices along Y axis or number of rows in the matrix configuration. This
>> values must be provided by the user, since they cannot be deduced by the
>> input image.
>> >
>> > The channels field defines in which color channel of the texture volume
>> data is being stored. The default behavior is to store the volume data in
>> the Red color channel by specifying the "R" value. When, a larger volume is
>> required to be converted into an ImageTextureAtlas additional channels can
>> be specified with "R", "G", "B", "A" characters. For instance, to store up
>> to three times more data. A "RGBA" value in the channel field will indicate
>> that all color channels of the texture are being used to store the data.
>> >
>> > Once multiple color channels are defined, the order in which 2D slices
>> are tiled into the atlas must be set in the sortOrder field. The default
>> behavior is to tile the 2D slices that represent the Z axis of the volume
>> data in "ROW” (row-major) order. For each slice of the ImageTextureAtlas
>> the next slice in the Z axis is the contiguous slice in the row of the
>> matrix of slices. When the sortOrder is set to "CHANNEL" the next slice of
>> the atlas in the Z axis direction is stored in the the contiguous color
>> channel.
>> >
>> > A 2D image representing an ImageTextureAtlas of the MRI ventricles
>> (Volvis)dataset (color inverted and contrast enhanced) as a set of 2D
>> slices tiled into a matrix configuration
>> >
>> >
>> > Not only to represent raw volume data, the ImageTextureAtlas can also
>> be used for gradient volume data and segmented volume data.
>> >
>> > Surface normals are specified to be provided as an additional 3D
>> texture. Using the same approach as before, we can create an
>> ImageTextureAtlas using the color channels of the 2D texture to store the
>> vector information of gradient data. We encode the gradient vector for each
>> pixel in the atlas in the RGB color channels {R: X, G: Y, B: Z}.
>> >
>> > Listing below shows an example declaration of the gradient data.
>> >
>> > <ImageTextureAtlas containerField="surfaceNormals" url="gradient.png"
>> slicesOverX="8" slicesOverY="8" channels=”RGB” sortOrger=”NONE”>
>> </ImageTextureAtlas>
>> >
>> >
>> > The containerField and URL attributes are the only modifications
>> required for the ImageTextureAtlas declaration. The containerField
>> attribute is used to target the volume data voxels field in a
>> X3DVolumeDataNode or gradient data surfaceNormals field in a
>> X3DVolumeStyleNode. This also requires, for the X3DVolumeStyleNode type
>> nodes with a surfaceNormals attribute, to accept a X3DTexture2DNode as an
>> input argument.
>> >
>> > The following Listing definition shows how X3DVolumeData derived nodes,
>> such as the VolumeData, should allow a 2D texture input to accept the
>> ImageTextureAtlas as a parameter.
>> >
>> > VolumeData : X3DVolumeDataNode {
>> >
>> > SFVec3f [in,out] dimensions 1 1 1 (0,∞)
>> >
>> > SFNode [in,out] metadata NULL [X3DMetadataObject]
>> >
>> > SFNode [in,out] renderStyle NULL [X3DVolumeRenderStyleNode]
>> >
>> > SFNode [in,out] voxels NULL [X3DTexture3DNode, X3DTexture2DNode]
>> >
>> > SFVec3f [] bboxCenter 0 0 0 (-∞,∞)
>> >
>> > SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1
>> >
>> > }
>> >
>> >
>> > The SegmentedVolumeData node allows the user to discern regions of the
>> volume and apply different rendering styles to each one. The segmented
>> region must be labeled per voxel.
>> >
>> > Textures provide a mechanism to input data to the GPU, updating
>> directly the visualization. In the Web platform we can take advantage of
>> the 2D canvas API to create or modify images that can be copied to the GPU.
>> This is of special interest to reflect dynamic changes in the data and, for
>> instance, to perform the construction of the atlas in the browser.
>> >
>> > Listing below shows an ImageTextureAtlas node declaration with a
>> canvas. The hideChildren attribute hides the atlas image visibility from
>> the user, but makes it available to be modified with JavaScript.
>> >
>> > <ImageTextureAtlas containerField="voxels" url="" slicesOverX="8"
>> slicesOverY="8" hideChildren="true">
>> >
>> >   <canvas id="v" style="width:2048px; height:2048px;">
>> >
>> >   </canvas>
>> >
>> > </ImageTextureAtlas>
>> >
>> >
>> >     1.2 MPRVolumeStyle node
>> >
>> > The Multi-Planar Reconstruction (MPR) is a wide spread rendering
>> technique. Essentially, it enables the user to define arbitrary planes
>> slicing through the volume data. The rendering algorithm resamples the
>> volume data to reconstruct the desired plane. Usually, the following planes
>> are defined: Axial, Sagital, Coronal and Oblique.
>> >
>> > The following Listing shows the proposed X3D MPR volume rendering style
>> definition.
>> >
>> > MPRVolumeStyle : X3DVolumeRenderStyleNode  {
>> >
>> > SFBool [in,out] enabled TRUE
>> >
>> > SFNode [in,out] metadata NULL [X3DMetadataObject]
>> >
>> > SFNode [in,out] transferFunction NULL [X3DTexture2DNode,
>> X3DTexture3DNode]
>> >
>> > SFBool [in,out] forceOpaque TRUE
>> >
>> > MFNode [in,out] planes NULL [MPRPlane]
>> >
>> > }
>> >
>> >
>> > An additional interesting enhancement to MPR is the ability to add a
>> transfer function (TF) in order to illustrate or filter regions in the
>> reconstructed planes. It is used to correlate the visual output with
>> obtained results. The proposed MPR node defines a /transferFunction
>> /attribute with the same functionality as the one already defined in the
>> OpacityMapVolumeStyle. This node also defines a planes attribute to allow
>> the user to declare arbitrary planes.
>> >
>> > The following Listing presents an arbitrary plane definition for the
>> MPRVolumeStyle node. The /normal /attribute defines the normal vector of
>> the plane, while the /position /attribute establishes the position of the
>> plane from the origin of the volume in the normal direction.
>> >
>> > MPRPlane : X3DNode {
>> >
>> > SFBool [in,out] enabled TRUE
>> >
>> > SFNode [in,out] metadata NULL [X3DMetadataObject]
>> >
>> > SFVec3 [in,out] normal 0 0 1
>> >
>> > SFFloat [in,out] position 0.0 [0,1]
>> >
>> > }
>> >
>> >
>> >     2.2 allowViewpointInside attribute for inside exploration
>> >
>> > Allowing to place the viewers virtual camera inside the volume provides
>> a new perspective to analyze the data. Some regions of the volume data can
>> be occluded even after filtering the data with a transfer function (TF). We
>> suggest to define a new attribute to enable or disable this behavior for
>> performance purposes.
>> >
>> > The following Listing shows the definition with the proposed attribute.
>> >
>> > X3DVolumeDataNode : X3DChildNode, X3DBoundedObject
>> >
>> > SFVec3f [in,out] dimensions 1 1 1 (0,∞)
>> >
>> > SFNode [in,out] metadata NULL [X3DMetadataObject]
>> >
>> > SFVec3f [] bboxCenter  0 0 0 (-∞,∞)
>> >
>> > SFVec3f [] bboxSize -1 -1 -1 [0,∞) or -1 -1 -1
>> >
>> > SFBool [in,out] allowViewPointInside TRUE
>> >
>> > }
>> >
>> >
>> > Our proposal is to declare a new boolean attribute allowViewPointInside
>> to all nodes inherited by the X3DVolumeDataNode. By default we suggest to
>> allow the inspection of the volume data and if the user does not require to
>> do so, it can be explicitly disabled. This feature allows to avoid
>> additional computation.
>> >
>> > --
>> > <http://www.vicomtech.org>
>> > Ander Arbelaiz Aranzasti
>> > Researcher | Investigador
>> >
>> > aarbelaiz at vicomtech.org <mailto:aarbelaiz at vicomtech.org>
>> > +[34] 943 30 92 30Industry and Advanced Manufacturing | Industria y
>> Fabricación Avanzada
>> >
>> > <https://www.linkedin.com/company/vicomtech> <
>> https://www.youtube.com/user/VICOMTech> <https://twitter.com/@Vicomtech>
>> > member of: <http://www.graphicsmedia.net/>
>> > Legal Notice - Privacy policy <
>> http://www.vicomtech.org/en/proteccion-datos>
>>
>>
>> all the best, Don
>> --
>> Don Brutzman  Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>>  +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics
>> http://faculty.nps.edu/brutzman
>>
>
>
> --
> <http://www.vicomtech.org>
> Ander Arbelaiz Aranzasti
> Researcher | Investigador
>
> aarbelaiz at vicomtech.org
> +[34] 943 30 92 30Industry and Advanced Manufacturing | Industria y
> Fabricación Avanzada
>
> <https://www.linkedin.com/company/vicomtech>
> <https://www.youtube.com/user/VICOMTech>  <https://twitter.com/@Vicomtech>
> member of:  <http://www.graphicsmedia.net/>
> Legal Notice - Privacy policy
> <http://www.vicomtech.org/en/proteccion-datos>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190613/dbc22cbf/attachment-0001.html>