[x3d-public] X3D VR

[Joe D Williams]

> This leads to a new requirement for X3D to accept these raw matrixes 
> as input.

This leads to a continuing requirement for X3D to produce best 
practice encodings, examples, and standards-track documentation for 
transport forms and processing algorithms.

I think overall these are data structures aimed at native paralell 
processing. There are plenty of examples, including xflow and, what is 
that one from the gl folks, vulcanator? As is being shown in the HAnim 
bvh, certainly a rw matrix, in part, import, many manifestions for 
evolving structures of the same data. HAnim is an example where this 
is the common form of animation and for X3D basics and advanced 
documents the critical data, that any character authoring tool can 
import into their structures and get a comparable result. And the 
encodings just happens to inspire a couple of x3d runtimes to use the 
stuff in a realtime interactive simulation environment that can be 
augmented by human interactions.

So, in all this new stuff, x3d wants to know what is the data that 
cannot be processed or rearranged for use in x3d? Then x3d 
implementors show how to do it. When we show how to do it for x3d, 
then finally, if it is open and contributes, it gets documented and 
archived and everyone has fun with it.

Thanks and Best,
Joe
.

----- Original Message ----- 
From: "Andreas Plesch" <andreasplesch at gmail.com>
To: "X3D Graphics public mailing list" <x3d-public at web3d.org>
Sent: Tuesday, December 20, 2016 9:50 PM
Subject: [x3d-public] X3D VR


> Hi
>
> Working with x3dom and WebVR I am exploring what additions to X3D 
> would be
> necessary or useful to effectively use current VR hardware such as 
> the
> Rift, Vive, Gear, or Cardboard.
>
> The proposed RenderedTexture node is currently used in x3dom with 
> special
> stereo modes to generate left and right views from a single 
> viewpoint in a
> single GL context.
>
> A Layer for each left and right view could also be used with two
> coordinated but separate viewpoints. This would be Cobweb's pattern.
>
> Since the HMD and its own runtime know best the internal optical
> characteristics of the display, the preferred interface at least in 
> WebVR
> are view and projection matrixes directly usable by consumers for 3d
> graphics generation, and not position, orientation and fov. This 
> leads to a
> new requirement for X3D to accept these raw matrixes as input.
>
> Since x3dom uses RenderedTexture for VR, I added there additional 
> special
> stereo modes which directly receive these matrixes from the HMD to 
> be used
> for rendering at each frame. This in effect accomplishes a first, 
> head and
> body based, level of navigation. In my first tests (on github) it 
> works
> well and should be robust across devices. This approach does require 
> some
> standard API to the HMD such as WebVR.
>
> Another possibility is to add view and projection matrix fields 
> input
> fields to viewpoints which can be continually updated. One could 
> convolve
> the view matrix with position/orientation fields, or optionally 
> completely
> ignore them.
>
> One then could use external SAI to keep updating or perhaps 
> introduce an
> environment ProjectionSensor. It would relay these matrix events 
> from an
> HMD runtime to then be routed to the viewpoints.
>
> A second level of navigation is accomplished with handheld 
> controllers.
> Until some standard gestures evolve, it will be necessary to expect 
> custom
> per scene navigation. X3d should have a way to sense buttons and 
> axes
> velocities of such controllers so these are then available to 
> manipulate
> the view in some way. InstantReality has a general IOSensor to that 
> effect.
> On the web the GamePad API is a standard which would need to be 
> used.
>
> Summary: RenderedTexture with stereo modes, matrix fields for 
> viewpoints, a
> ProjectionSensor, and a ControllerSensor would all be candidates for 
> x3d VR
> support.
>
> Thanks for reading, any thoughts welcome,
>
> Andreas
>


--------------------------------------------------------------------------------


> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>