[x3d-public] X3D VR (doug sanden)

[Andreas Plesch]

>
> Date: Wed, 21 Dec 2016 14:36:02 +0000
> From: doug sanden <highaspirations at hotmail.com>
> To: X3D Graphics public mailing list <x3d-public at web3d.org>
> Subject: Re: [x3d-public] X3D VR
>
> Andreas
> Q. can/could/should io/stereo mishmash be in a separate config file?
> Q. can/could/should the io and stereo special events come from the
> singleton Browser object?
>

Let's focus on stereo first. InstantReality decided to follow similar ideas
with Engines and Viewareas:

http://doc.instantreality.org/tutorial/get-your-engine-right/
http://doc.instantreality.org/tutorial/multiple-windows-and-views/

The approach allows for very flexible configurations, including CAVEs and
clusters and such.

Consumer VR is more limited, and with the emerging WebVR API standard there
is a chance to have unified access to displays and their sensors. There may
be similar abstraction layers outside of the web realm or the webVR API
could be implemented outside of the web browser as well (as a thin layer on
top of each HMD runtime).

Still, having a place such as config file or an engine node on the same
level as the scene node seems attractive if scene content should be
separated from rendering control.

-Doug
> ...
> Great topic. I struggled with Android NDK x3dbrowser handling of gyro etc
> for navigation and found a way to hack and intercept the viewpoint matrix
> stack, and similar config issues with cardboard viewport distortion
> configuration -great spaghetti hacking- but need to make it more
> systematic, configurable and if possible following some standard pattern.
>

Maybe it makes sense to implement the webVR API for Android ? Chrome with
webVR is now available for Android
https://chromium.googlesource.com/experimental/chromium/src/
+/refs/wip/bajones/webvr_1/third_party/WebKit/Source/modules/vr/
https://cs.chromium.org/chromium/src/third_party/WebKit/Source/modules/vr/

Not sure if the source here is helpful.


> ...
> One annoying thing is having to put per-device configurations into the
> scene file, such as special handling of triggers, matrices etc.
> IDEAS:
> - hyperscene one level up from scene file
> -  .x3dcfg - a separate mini-scene with small set of nodes, inhaled
> separately by the program
>
> And the goal of either of those is to convert per-device stereo and input
> mishmash into some very standard things hidden in the Browser as seen by
> the scene, so scene files can still be rendered non-stereo with no change
> to the scene file. Yet still authorable.
> For example in conventional x3d, as the user works the mouse to navigate,
> you don't necessarily get any mouse events coming into the scene - no
> routing is mandatory to make navigation happen - and the viewpoint
> 'magically' moves as seen from the scene file. It might let you know where
> it moved, and allow you to do some moving, but it doesn't require the scene
> author to intervene, or know which buttons are pressed.
> Mind you for a few decades things have been fairly stable hardware-wise on
> the desktop, with a mouse and keyboard. So the formulas for converting
> mouse and keyboard to viewpoint navigation -and stereo settings- were baked
> into the browser.
> Now with hardware and stereo permutations fluid and growing in number,
> it's getting hard for Browser code to hide all that. And some scene authors
> _do_ want to intercept and play with it.
>

It would be great to offer some easy VR navigation modes and Leonard had
some ideas, I believe. Staring to simulate a 'click' or a selection comes
to mind . But there is no standard on how physical controllers are used, or
even minimum requirements in terms of buttons, degrees of freedom, haptics,
touchpads. So the only solution is to expect scene authors to do their own
navigation by giving them access to the controller output. Perhaps protos
can then be shared to make this easier for authors.
InstantReality has IOSensors with lots of available device types:
http://doc.instantreality.org/tutorial/inputoutput-streams/
http://doc.instantreality.org/device/
webVR is struggling with the same problem and are trying to come up with a
standard at least for mapping buttons to actions, for example. The browser
gamepad API is at least a standard to interface with controllers.

IDEA: Browser fields: instead of new nodes and components for getting
> io/stereo information into the scene to play, can/could/should they be
> fields/events on the x3d Browser object? That would keep them Singletons -
> only one per scene (vs separate node type which could be instanced multiple
> times - although I think we have a node or two like this already - should
> be singletons but are allowed to be instanced multiple). And analogous to
> the desktop Browser managing/hiding all that info.
>
> But try and imagine all the permutations of your basic scene file you'd
> need for every permutation of device and stereo setting if its all in the
> scene file as new nodes.
>

InstantReality has an 'auto' type for IOSensors which tries to find and use
an expected device type given the output fields which are requested.

So the urge to abstract a bit and make more flexible for scene authors.
>

It may be necessary to define a list of events such as
x/y/z/translation,yaw/pitch/roll, buttonABCD states which a standard VR
ControllerSensor is expected to output.

Pressing a designated button, controllers/wands often turn into a pointing
device. The Vive typically then shows a ray emanating from the in-scene
controller represention. Would a x3d browser be expected to automatically
recognize this use of the controller as a pointing device ? Again, I think
InstantReality has a way to configure this use (with a 'ray' navigation
mode or so).

This kind of discussion must have happened many times in the past so there
may be some consensus on a standard approach to controllers ? Or perhaps
each setup used to be unique enough that more ad hoc solutions were viable.

-Andreas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20161223/a5463334/attachment.html>