[x3d-public] X3D VR and viewing - HMD node or mode?
Andreas Plesch
andreasplesch at gmail.com
Sat Jan 28 17:28:42 PST 2017
continuing diff WebVR and X3D...
sixth, interface VREyeParameters:
https://w3c.github.io/webvr/#interface-vreyeparameters
readonly attribute Float32Array offset; could be new component of avatarSize
readonly attribute VRFieldOfView fieldOfView; Deprecated
readonly attribute unsigned long renderWidth; important for browser;
MetadataInteger for scene to adapt to resolution
readonly attribute unsigned long renderHeight;
seventh, vrstageparameters:
https://w3c.github.io/webvr/#interface-vrstageparameters
readonly attribute Float32Array sittingToStandingTransform; only useful
browser when changing from 'sitting/lookat' to 'walk' navigation mode
readonly attribute float sizeX; MetadataFloat, scene can use this
information to visualize stage (physical action area)
readonly attribute float sizeZ;
next: VRDisplayEvent and VRDisplayEventReason
...Andreas
On Tue, Jan 17, 2017 at 7:42 PM, Andreas Plesch <andreasplesch at gmail.com>
wrote:
> continuing diff WebVR and X3D...
>
> Third, Vrframedata: https://w3c.github.io/webvr/#interface-vrframedata
>
> readonly attribute DOMHighResTimeStamp
> <http://www.w3.org/TR/hr-time/#domhighrestimestamp> timestamp
> <https://w3c.github.io/webvr/#dom-vrframedata-timestamp>; X3D events have
> timestamps
>
> readonly attribute Float32Array leftProjectionMatrix
> <https://w3c.github.io/webvr/#dom-vrframedata-leftprojectionmatrix>;
> critical for correct rendering as fine tuned for HMD; used internally in
> left-vr RenderedTexture mode; could be routed to Viewfrustum node;
> MetadataMatrix4f
>
> readonly attribute Float32Array leftViewMatrix
> <https://w3c.github.io/webvr/#dom-vrframedata-leftviewmatrix>; see above;
> turns out that Viewfrustum expects the inverse
>
> readonly attribute Float32Array rightProjectionMatrix
> <https://w3c.github.io/webvr/#dom-vrframedata-rightprojectionmatrix>; see
> above
>
> readonly attribute Float32Array rightViewMatrix
> <https://w3c.github.io/webvr/#dom-vrframedata-rightviewmatrix>; see above
>
> readonly attribute VRPose <https://w3c.github.io/webvr/#vrpose> pose
> <https://w3c.github.io/webvr/#dom-vrframedata-pose>; see below
>
> Fourth, Vrpose: https://w3c.github.io/webvr/#interface-vrpose
>
> readonly attribute Float32Array? position
> <https://w3c.github.io/webvr/#dom-vrpose-position>; consistent with
> viewMatrix, relative to resetPose position, Metadata SFVec3f
>
> readonly attribute Float32Array? linearVelocity
> <https://w3c.github.io/webvr/#dom-vrpose-linearvelocity>; velocity in
> direction of orientation; Metadata SFFloat ; can be used by browser to
> predict position at predicted time of submission of completely rendered
> frame.
>
> readonly attribute Float32Array? linearAcceleration
> <https://w3c.github.io/webvr/#dom-vrpose-linearacceleration>; Metadata
> SFFloat
>
> readonly attribute Float32Array? orientation
> <https://w3c.github.io/webvr/#dom-vrpose-orientation>; Metadata
> SFRotation, needs conversion from quaternion
>
> readonly attribute Float32Array? angularVelocity
> <https://w3c.github.io/webvr/#dom-vrpose-angularvelocity>; Metadata
> SFFloat
>
> readonly attribute Float32Array? angularAcceleration
> <https://w3c.github.io/webvr/#dom-vrpose-angularacceleration>; Metadata
> SFFloat
>
> Summary
>
> Left/right/projection/viewMatrix: Since there is no MetadataMatrix4f,
> Metadata MFFloat
>
> PosePosition: at timestamp, actual position at frame submission time could
> be estimated
>
> PoseVelocity/Acceleration: potentially useful for in-scene prediction
>
> Next VrEyeParameters and VrStageParameters
>
>
> Andreas
>
> On Jan 15, 2017 1:51 PM, "Andreas Plesch" <andreasplesch at gmail.com> wrote:
>
>> Continuing with diff of functionality between WebVR API and X3D
>> scenegraph ...
>>
>> Second, VRLayer : https://w3c.github.io/webvr/#interface-vrlayer
>>
>> readonly attribute VRSource? source; VRSource is the html canvas element
>> used for display in the HMD; would correspond to the X3D element;
>>
>> readonly attribute sequence<float> leftBounds; viewport for left eye;
>> Metadata MFFloat leftBounds; at browser discretion but usually set to left
>> half of the render area (default)
>> readonly attribute sequence<float> rightBounds; viewport for right eye;
>> Metadata MFFloat rightBounds; at browser discretion but usually set to
>> right half of the render area (default)
>>
>> Third, VRDisplay capabilites : https://w3c.github.io/webvr/#i
>> nterface-vrdisplaycapabilities
>>
>> readonly attribute boolean hasPosition; position tracking; MetadataBool
>> readonly attribute boolean hasOrientation; orientation tracking;
>> MetadataBool
>> readonly attribute boolean hasExternalDisplay; false for cardboard;
>> MetadataBool
>> readonly attribute boolean canPresent; probably not useful for scene
>> readonly attribute unsigned long maxLayers; currently restricted to one
>>
>> Summary: additional Metadata for VrDisplay
>>
>> VRSourceLeftBounds
>> VRSourceRightBounds
>> hasPosition
>> hasOrientation
>> hasExternalDisplay
>>
>> next VRFrameData and VRPose
>>
>> Andreas
>> On Sat, Jan 14, 2017 at 2:07 PM, Andreas Plesch <andreasplesch at gmail.com>
>> wrote:
>>
>>> Sounds like a plan as it will be critical to interface well with the
>>> WebVR API as an important emerging standard for consumer VR hardware.
>>>
>>> First, a link to the WebVR standard:
>>>
>>> https://w3c.github.io/webvr/
>>> latest: https://rawgit.com/w3c/webvr/master/index.html (development)
>>>
>>> Going through every parameter in the webvr standard will be a lengthy
>>> process, so let's better get started.
>>>
>>> Note, that right in the introduction the focus is directed at interfaces
>>> to VR hardware.
>>>
>>> X3D though device independant has a fair amount of assumptions on
>>> hardware capabilities used to render and interact with the scenegraph. For
>>> example, pointing devices and keyboards and opengl/inventor based rendering
>>> are implied to be available.
>>>
>>> > Regarding configuration information, here is another way of
>>> considering our many existing capabilities together with new opportunities.
>>>
>>> >
>>>
>>> > a. Perform a diff of functionality between WebVR API and X3D scene
>>> graph.
>>>
>>> > - What is in WebVR needed for HMDs that isn't supported in X3D?
>>> > - probably also helpful to list everything found in common already.
>>>
>>> First: https://w3c.github.io/webvr/#interface-vrdisplay
>>>
>>> General points: x3d does not have a concept of a display, eg. the exact
>>> method of displaying is left to the x3d browser. The scene does not have a
>>> way to infer display parameters other than browser.options which a browser
>>> may provide. In addition to display parameters, vrDisplay also includes
>>> navigation parameters, the pose. x3d has a concept of navigation, so there
>>> are intersections.
>>>
>>> readonly attribute boolean isConnected: could be used by a browser
>>> implementation to indicate to the user that HMD is available. If x3d has
>>> guidelines on browser UI, it should include a list of available display
>>> devices. Probably best left to the discretion of the browser.
>>>
>>> readonly attribute boolean isPresenting: the browser should use this
>>> attribute to indicate that a device is currently in use. On a web page this
>>> could be a message on the monitor, replacing the scene rendering. Should be
>>> also available to the scene, to allow dynamic adjustment of scene
>>> content/viewpoint use when HMD is used. MetadataBoolean
>>> vrDisplayisPresenting .
>>>
>>> readonly attribute VRDisplayCapabilities capabilities: detail for each
>>> property later. Most capabilities potentially useful for a scene, eg.
>>> should be in Metadata. If a capability is not available, the browser should
>>> not attempt to substitute for the missing capability. Should be shown in a
>>> browser UI.
>>>
>>> readonly attribute VRStageParameters? stageParameters: detail for each
>>> property later. Concerns both viewing and interaction. Browser needs
>>> sittingToStandingTrafo, scene could use sizes of stage to add visual stage
>>> representation to scene, so in Metadata.
>>>
>>> VREyeParameters getEyeParameters(VREye whichEye): renderWidth/Height for
>>> browser rendering, and browser UI . fov and offset deprecated . Resolution
>>> useful for scene as Metadata, also more generally for any kind of display.
>>>
>>> boolean getFrameData(VRFrameData frameData): VRFramedata have
>>> projection/view matrices and pose. Viewpoint discussion applies. Browser
>>> uses these data to navigate by updating viewpoint on top of other
>>> navigation mode dependent navigation. Useful also for scene but expensive
>>> since Metadata would need to be updated every frame. Detail later but x3d
>>> needs to use the provided data.
>>>
>>> [NewObject] VRPose getPose(): deprecated in favour of FrameData
>>>
>>> void resetPose(): designate a key press to call resetPose when HMD
>>> isPresenting, similar to key presses to cycle viewpoints ? InputField
>>> MetadataBoolean vrDisplayResetPose to trigger reset ?
>>>
>>> attribute double depthNear; gets used to determine provided projection
>>> matrix, should be set to viewpoint's zNear by browser, so no new field
>>> required
>>>
>>> attribute double depthFar; same
>>>
>>> unsigned long requestAnimationFrame(FrameRequestCallback callback);
>>> browser implementation detail
>>>
>>> void cancelAnimationFrame(unsigned long handle); browser implementation
>>> detail
>>>
>>> Promise<void> requestPresent(sequence<VRLayerInit> layers); layer has
>>> source to be displayed, currently only 1 layer allowed, no new nodes .
>>>
>>> Promise<void> exitPresent(); used by browser to stop displaying on HMD,
>>> Scene can use isPresenting above to react.
>>>
>>> sequence<VRLayer> getLayers();
>>>
>>> void submitFrame(); browser uses this to indicate that a frame is ready
>>> to be displayed in the HMD. WebVR will then display it.
>>>
>>>
>>> Summary: VrDisplay node could have these Metadata:
>>>
>>> isPresenting
>>>
>>> capabilities.hasFeatureX
>>>
>>> stageParametersSizes
>>>
>>> left/rightRenderWidth/Height
>>>
>>> resetPose
>>>
>>>
>>> Next VRLayer and VRDisplay capabilities.
>>>
>>> -Andreas
>>>
>>> >
>>> > b. Characterize and define those parameters, divide and conquer:
>>> > - what information is specific to hardware devices for viewing,
>>> > - what information is specific to hardware devices for interaction,
>>> > - what information is specific to user anatomy: spacing between eyes,
>>> etc.
>>> > - what information might be considered scene specific, i.e. varying by
>>> scene usage?
>>> > - what information might be considered a user preference?
>>> >
>>> > c. Preparing for initial experimentation and evaluation, what are use
>>> cases for common use?
>>> > - the MAR Reference Model (update briefing in Seoul next week) has
>>> done a lot on that.
>>> >
>>> > d. Good assumption: X3D viewing navigation and interaction models are
>>> device neutral already. Rephrase: authors can create scenes for users that
>>> can be used immediately and effectively over a wide range of devices
>>> already, no further customization needed.
>>> >
>>> > e. Looking over all the WebVR/HMD diffs with existing X3D:
>>> > - likely this is mostly information that can be expressed as
>>> parameters.
>>> > - just like on this discussion thread, people will want to try
>>> different approaches with nodes/fields/scripts
>>> > - the information set of parameters could be captured in a Prototype,
>>> perhaps embedding a Script
>>> > - the information set of parameters could be captured in a MetadataSet
>>> collection
>>> > - this enables different players to implement in their own way using
>>> shared X3D content
>>> >
>>> > Hope this helps explain a potential path for further cooperative
>>> exploration. There are lots of alternatives.
>>> >
>>> > I will be requesting permission to share MAR Reference Model more
>>> widely in our consortium/community, it will likely complement WebVR nicely.
>>> >
>>> > We can discuss for a bit on today's X3D working group teleconference,
>>> if you or anyone wants. Probably should be a monthly focus topic, dialog
>>> often clarifies email ideas nicely.
>>> >
>>> > v/r Don
>>> >
>>> >
>>> > On 1/10/2017 7:42 PM, Andreas Plesch wrote:
>>>
>>> >>
>>>
>>> >> On Jan 8, 2017 5:45 PM, "Don Brutzman" <brutzman at nps.edu <mailto:
>>> brutzman at nps.edu>> wrote:
>>>
>>> >>>
>>>
>>> >>>
>>> >>> Another option: can we define HMD parameters separately from
>>> existing Viewpoint and NavigationInfo nodes, so that existing scenes (with
>>> many many viewpoints already defined) might be viewable without
>>> modification.
>>> >>
>>> >>
>>> >> Ok, while sofar the node/field proposals were made from a practical
>>> standpoint taking into account existing or feasible implementations, let's
>>> turn it around and see what should ideally be available from a scene
>>> authoring perspective.
>>> >>
>>> >> Let's imagine there is a browser and it has built-in options to
>>> display an existing scene in a selected HMD taking advantage of stereo
>>> immersion and head tracking/body tracking.
>>> >>
>>> >> Let's also assume the scene is designed for both desktop and HMD
>>> experience. This is probably harder than it sounds but scenes designed for
>>> WALK mode with a single viewpoint or careful transitions between
>>> viewpoints, on a human scale, should be a good start.
>>> >>
>>> >> The browser can figure out optimal rendering resolution, and use the
>>> HMD provided projection and view matrixes for navigation relative to the
>>> bound viewpoint.
>>> >>
>>> >> All this does not seem to need scene specific information (?). This
>>> is why we had previously discussed a browser configuration file or Engine
>>> node above/next to the scene level.
>>> >>
>>> >> Now, it turns out that currently the easiest way to add such
>>> functionality to the x3dom browser is to preprocess the scene (via
>>> 'injection') to use additional nodes (the RenderedTexture node). When
>>> getting out of HMD mode, these additional nodes can be removed again.
>>> >>
>>> >> For other browsers (such as cobweb) it may be advantageous to
>>> preprocess the scene in a different way, requiring other, specialized nodes
>>> (Viewfrustum).
>>> >>
>>> >> This is just one way to add functionality to a x3d browser and
>>> currently the most available one. It is also modular as nodes can be used
>>> for other purposes.
>>> >>
>>>
>>> >>> Perhaps an HMD node, rather than HMD mode? That is, a separate node
>>> that complements NavigationInfo?
>>>
>>> >>
>>> >>
>>>
>>> >>> The VRML and X3D specifications have done an excellent job in being
>>> device neutral (as well as quite supportive of most devices) when defining
>>> user viewing and interaction already. It will be good to keep building on
>>> that device independence, which incidentally lets you create large shared
>>> environments where users have multiple diverse devices interacting.
>>>
>>> >>>
>>> >>> WebVR might provide an excellent first pass for the parameters that
>>> might be needed for HMDs.
>>> >>
>>> >>
>>> >> WebVR inspired parameters supplied by/provided to a HMD node could be:
>>> >>
>>> >> isPresenting: SFBool, currently in use, scene may want to adapt to
>>> immersion experience in some way
>>> >>
>>> >> Stage: some HMDs have a certain, limited area in which a user can be
>>> tracked and be expected to stay inside. This could be useful information
>>> for a scene. Includes size, location.
>>> >>
>>> >> Enabled: a scene may want to strictly disable display on a HMD
>>> >>
>>> >> DeviceID: a scene may want to know a name for the device to reflect
>>> >>
>>> >> Capabilities: resolution: a scene may want to adapt (hi res textures)
>>> >>
>>> >> resetPose: make current location/orientation the reference.
>>> >>
>>>
>>> >>>
>>>
>>> >>> Presumably most of fields are metadata for devices to use, which is
>>> in keeping with scenes that can be utilized by many devices.
>>> >>
>>> >>
>>> >> Hmm, as evidenced above, I tend to think about the reverse, eg.
>>> fields which can be utilized by scenes. So I am probably missing something.
>>> >>
>>>
>>> >>> So if we can agree on parameters then a simple MetadataSet node (or
>>> prototype) for HMD might be sufficient to start performing some
>>> experimentation in developmental versions of players.
>>>
>>> >>>
>>> >>> Wondering if such metadata is oriented towards several categories at
>>> once:
>>> >>
>>> >>
>>> >> (a) device specific,
>>> >> all above
>>> >>
>>> >> (b) scene specific,
>>> >>
>>> >> I cannot think of anything, eg. imagine how a scene could modify the
>>> behaviour of a HMD. At this point HMDs only have a single mode. In the
>>> future, a HMD may switch between VR, AR, MR and a scene request an expected
>>> mode ?
>>> >>
>>> >> If we start to also consider controllers, this might be the place
>>> where a scene configures usage of controller buttons and axes.
>>> >>
>>> >> and (c) user specific (which can also be saved as preferences).
>>> >>
>>> >> lastPose: pose before exiting HMD mode
>>> >> resumePose: return to last pose upon entering
>>> >> ...
>>> >>
>>> >> I am still unsure if I grasped the idea behind using a MetadataSet
>>> node as a conduit between scene and HMD.
>>> >>
>>> >> Andreas
>>> >>
>>> >> This might help re-use and portability.
>>>
>>> >>>
>>>
>>> >>>
>>> >>>
>>> >>> On 1/8/2017 12:40 PM, Andreas Plesch wrote:
>>>
>>> >>>>
>>>
>>> >>>>
>>> >>>> Yves,
>>> >>>>
>>> >>>> thanks for your input. While seeing a Viewfrustum node as a
>>> generalization is convincing, there is a practical difference in how it
>>> would be used. Rather than specifying an initial viewpoint from which user
>>> navigation starts/is relative to, the matrices would be used directly for
>>> navigation. Not sure what the implications of this difference are.
>>> >>>>
>>> >>>> Another option is to define another navigation mode, 'HMD', which
>>> would take HMD output (the matrixes) and apply it to the current viewpoint.
>>> If the viewpoint is a stereo viewpoint left and right matrixes would be
>>> supplied. Additional navigation from controllers would still need to be
>>> possible. What device 'HMD' refers to could be defined by a new
>>> NavigationInfo field 'hmd' MFString = "''AUTO' 'VIVE' ... " in case of
>>> multiple connected devices.
>>> >>>>
>>> >>>> Andreas
>>> >>>>
>>> >>>> On Jan 8, 2017 11:38 AM, "Yves Piguet" <yves.piguet at gmail.com
>>> <mailto:yves.piguet at gmail.com> <mailto:yves.piguet at gmail.com <mailto:
>>> yves.piguet at gmail.com>>> wrote:
>>>
>>> >>>>>
>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> I'd also prefer a new node. Another reason is the existence of
>>> OrthoViewpoint: a viewpoint where you specify an arbitrary projection
>>> matrix is a generalization of both Viewpoint and OrthoViewpoint, so it
>>> should logically be a sibling which also implements the X3DViewpointNode
>>> abstract node. Now an extension for stereoscopic views, where you still
>>> have a single viewer position which defines viewer-related sensor events,
>>> LOD, Billboard etc. and a single node traversal for rendering with two
>>> projection matrices, seems fine.
>>> >>>>>
>>> >>>>> Yves
>>> >>>>>
>>> >>>>>
>>> >>>>> > On 8 Jan 2017, at 15:29, Andreas Plesch <andreasplesch at gmail.com
>>> <mailto:andreasplesch at gmail.com> <mailto:andreasplesch at gmail.com
>>> <mailto:andreasplesch at gmail.com>>> wrote:
>>> >>>>> >
>>> >>>>> > > Date: Sat, 7 Jan 2017 21:04:42 +0100
>>> >>>>> > > From: Holger Seelig <holger.seelig at yahoo.de <mailto:
>>> holger.seelig at yahoo.de> <mailto:holger.seelig at yahoo.de <mailto:
>>> holger.seelig at yahoo.de>>>
>>> >>>>> > > To:x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>>> >>>>> > > Subject: Re: [x3d-public] X3D VR
>>> >>>>> > >
>>> >>>>> > >
>>> >>>>> > > I think instead of defining a new field to the existing
>>> viewpoints, it
>>> >>>>> > > should be considered that for that purpose and new viewpoint
>>> node could
>>> >>>>> > > be appropriate. A node called something like
>>> ?MatrixViewpoint?. This
>>> >>>>> > > node could have two fields matrix and projectionMatrix. The
>>> matrix field
>>> >>>>> > > would store the transformation matrix with orientation and
>>> position
>>> >>>>> > > values. The projectionMatrix field would store the projection
>>> matrix.
>>> >>>>> > > This viewpoint would be an all purpose viewpoint. The main
>>> idea is if
>>> >>>>> > > one is able to define an ?viewMatrix? field as Andreas
>>> proposes he is
>>> >>>>> > > able to handle a node like MatrixViewpoint with its matrix
>>> fields.
>>> >>>>> > > Defining a new node has the advantages to left the existing
>>> viewpoints
>>> >>>>> > > untouched and to handle this behavior in a new node.
>>> >>>>> > >
>>> >>>>> > > Holger
>>> >>>>> >
>>> >>>>> > It turns out that x3dom and InstantReality have such a node,
>>> called Viewfrustum:
>>> >>>>> >
>>> >>>>> >https://doc.x3dom.org/author/Navigation/Viewfrustum.html <
>>> https://doc.x3dom.org/author/Navigation/Viewfrustum.html>
>>> >>>>> >
>>> >>>>> > So this is a pretty strong argument for a new node instead of
>>> new fields.
>>> >>>>> >
>>> >>>>> > New fields have the advantage that they should make it easier to
>>> adapt existing scenes. I think default values could be such that they
>>> guarantee full backward compatibility.
>>> >>>>> >
>>> >>>>> > I will try to use Viewfrustum with the x3dom classroom example
>>> to gain some experience.
>>> >>>>> >
>>> >>>>> > Andreas
>>> >>>>> >
>>> >>>>> > >
>>> >>>>> > > Am 07.01.2017 um 18:07 schrieb Andreas Plesch:
>>> >>>>> > > > Hi,
>>> >>>>> > > >
>>> >>>>> > > > back from the holiday season, here are proposals for new
>>> fields for
>>> >>>>> > > > Viewpoint.
>>> >>>>> > > >
>>> >>>>> > > > I do not have an x3dom implementation but the idea is to add
>>> two
>>> >>>>> > > > Matrix4d fields:
>>> >>>>> > > >
>>> >>>>> > > > projectionMatrix :
>>> >>>>> > > >
>>> >>>>> > > > default: NULL
>>> >>>>> > > >
>>> >>>>> > > > purpose: allow control of the view projection from 3d to 2d
>>> by such a
>>> >>>>> > > > matrix from sources such HMD drivers. The projection matrix
>>> controls fov
>>> >>>>> > > >
>>> >>>>> > > > The projectionMatrix should be used as is by the browser if
>>> not NULL. If
>>> >>>>> > > > NULL, a projection matrix shall be contructed from viewpoint
>>> fields.
>>> >>>>> > > >
>>> >>>>> > > >
>>> >>>>> > > > viewMatrix:
>>> >>>>> > > >
>>> >>>>> > > > default: unit matrix
>>> >>>>> > > >
>>> >>>>> > > > purpose: allow control of the viewer orientation and
>>> location by such a
>>> >>>>> > > > matrix from sources such as HMD drivers.
>>> >>>>> > > >
>>> >>>>> > > > The viewMatrix V shall be combined with the matrix M derived
>>> from the
>>> >>>>> > > > location/ orientation fields by M * V for rendering use.
>>> >>>>> > > >
>>> >>>>> > > > This means that if V is used in a scene exclusively the
>>> location fields
>>> >>>>> > > > needs to be reset to 0 0 0 from its default by the scene.
>>> >>>>> > > >
>>> >>>>> > > > Typical use would be as sinks of routes from (new)
>>> ProjectionSensor or
>>> >>>>> > > > VRDevice node fields.
>>> >>>>> > > >
>>> >>>>> > > > A VRDevice delivers both matrixes for each eye. A scene can
>>> then set up
>>> >>>>> > > > a viewpoint for each eye, and use Layers to show them side
>>> by side.
>>> >>>>> > > >
>>> >>>>> > > > In terms of efficiency, this would seem to require repeated
>>> traversals
>>> >>>>> > > > of the scene unless both Layers can be recognized to show
>>> the same scene
>>> >>>>> > > > content.
>>> >>>>> > > >
>>> >>>>> > > > A more convenient and integrated alternative would be to
>>> introduce a
>>> >>>>> > > > stereo SFBool field and have left and right variants of the
>>> matrix
>>> >>>>> > > > fields ( and perhaps also left and right variants of the
>>> location,
>>> >>>>> > > > orientation, fov, zfar, znear fields).
>>> >>>>> > > >
>>> >>>>> > > > stereo = false would ignore all left and right fields.
>>> >>>>> > > > stereo = true would use the left/right fields and render
>>> them side by side.
>>> >>>>> > > >
>>> >>>>> > > > Since in this case there is a guaranty that views for each
>>> eye use the
>>> >>>>> > > > same scene content, only one traversal should be needed to
>>> render both eyes.
>>> >>>>> > > >
>>> >>>>> > > > The idea would be switching mono to stereo and back only
>>> requires
>>> >>>>> > > > toggling the stereo field. Navigation may have a similar
>>> switch to and
>>> >>>>> > > > from mouse/keyboard modes ?
>>> >>>>> > > >
>>> >>>>> > > > Implementation in x3dom of viewMatrix and projectionMatrix
>>> may be rather
>>> >>>>> > > > straightforward. I do not see large obstacles (famous last
>>> words ...).
>>> >>>>> > > > Implementation in cobweb of viewMatrix and projectionMatrix
>>> may be also
>>> >>>>> > > > straightforward but I am less sure.
>>> >>>>> > > >
>>> >>>>> > > > Without layers in x3dom then one could put two scene side by
>>> side on the
>>> >>>>> > > > web page, use external js to feed the matrix fields, and
>>> then also use
>>> >>>>> > > > external js to grab the frames from both scene canvases,
>>> combine them
>>> >>>>> > > > into a single frame on another canvas, and use that finally
>>> as a source
>>> >>>>> > > > for the HMD display. (This sounds slow).
>>> >>>>> > > > In cobweb, it should not be hard to set up a scene with a
>>> layer for each
>>> >>>>> > > > eye.
>>> >>>>> > > >
>>> >>>>> > > > Implementation of stereo, left and
>>> rightView/ProjectionMatrix in x3dom
>>> >>>>> > > > is harder since it needs changes to the rendering code which
>>> is a bit
>>> >>>>> > > > unstructured. The first approach would be for stereo=true to
>>> go through
>>> >>>>> > > > the rendering twice with the rendering surfaces
>>> appropriately set somehow.
>>> >>>>> > > >
>>> >>>>> > > > -Andreas
>>> >>>>> > > >
>>> >>>>> > > > On Dec 21, 2016 6:00 AM, "Roy Walmsley" <
>>> roy.walmsley at ntlworld.com <mailto:roy.walmsley at ntlworld.com> <mailto:
>>> roy.walmsley at ntlworld.com <mailto:roy.walmsley at ntlworld.com>>
>>> >>>>> > > > <mailto:roy.walmsley at ntlworld.com <mailto:
>>> roy.walmsley at ntlworld.com> <mailto:roy.walmsley at ntlworld.com <mailto:
>>> roy.walmsley at ntlworld.com>>>> wrote:
>>> >>>>> > > >
>>> >>>>> > > > Hi Andreas,____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > That?s great to see what you have been doing here.
>>> Would you like
>>> >>>>> > > > to take your node suggestions one step further and
>>> propose the
>>> >>>>> > > > fields that might be specified for each node?____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > What we could do then is, in the new year, is to host an
>>> open
>>> >>>>> > > > discussion on WebVR and X3D. Contributions from any
>>> other readers
>>> >>>>> > > > are welcome, and could lead to a lively discussion on
>>> this topic.____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > Thank you for the suggestions,____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > All the best,____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > Roy____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > *From:*x3d-public [mailto:x3d-public-bounces at web3d.org
>>> <mailto:x3d-public-bounces at web3d.org> <mailto:x3d-public-bounces at web
>>> 3d.org <mailto:x3d-public-bounces at web3d.org>>
>>> >>>>> > > > <mailto:x3d-public-bounces at web3d.org <mailto:
>>> x3d-public-bounces at web3d.org> <mailto:x3d-public-bounces at web3d.org
>>> <mailto:x3d-public-bounces at web3d.org>>>] *On Behalf Of *Andreas Plesch
>>> >>>>> > > > *Sent:* 21 December 2016 05:51
>>> >>>>> > > > *To:* X3D Graphics public mailing list <
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>>> >>>>> > > > <mailto:x3d-public at web3d.org <mailto:
>>> x3d-public at web3d.org> <mailto:x3d-public at web3d.org <mailto:
>>> x3d-public at web3d.org>>>>
>>> >>>>> > > > *Subject:* [x3d-public] X3D VR____
>>> >>>>> > > >
>>> >>>>> > > > __ __
>>> >>>>> > > >
>>> >>>>> > > > Hi____
>>> >>>>> > > >
>>> >>>>> > > > Working with x3dom and WebVR I am exploring what
>>> additions to X3D
>>> >>>>> > > > would be necessary or useful to effectively use current
>>> VR hardware
>>> >>>>> > > > such as the Rift, Vive, Gear, or Cardboard.____
>>> >>>>> > > >
>>> >>>>> > > > The proposed RenderedTexture node is currently used in
>>> x3dom with
>>> >>>>> > > > special stereo modes to generate left and right views
>>> from a single
>>> >>>>> > > > viewpoint in a single GL context.____
>>> >>>>> > > >
>>> >>>>> > > > A Layer for each left and right view could also be used
>>> with two
>>> >>>>> > > > coordinated but separate viewpoints. This would be
>>> Cobweb's pattern.____
>>> >>>>> > > >
>>> >>>>> > > > Since the HMD and its own runtime know best the internal
>>> optical
>>> >>>>> > > > characteristics of the display, the preferred interface
>>> at least in
>>> >>>>> > > > WebVR are view and projection matrixes directly usable
>>> by consumers
>>> >>>>> > > > for 3d graphics generation, and not position,
>>> orientation and fov.
>>> >>>>> > > > This leads to a new requirement for X3D to accept these
>>> raw matrixes
>>> >>>>> > > > as input.____
>>> >>>>> > > >
>>> >>>>> > > > Since x3dom uses RenderedTexture for VR, I added there
>>> additional
>>> >>>>> > > > special stereo modes which directly receive these
>>> matrixes from the
>>> >>>>> > > > HMD to be used for rendering at each frame. This in
>>> effect
>>> >>>>> > > > accomplishes a first, head and body based, level of
>>> navigation. In
>>> >>>>> > > > my first tests (on github) it works well and should be
>>> robust across
>>> >>>>> > > > devices. This approach does require some standard API to
>>> the HMD
>>> >>>>> > > > such as WebVR.____
>>> >>>>> > > >
>>> >>>>> > > > Another possibility is to add view and projection matrix
>>> fields
>>> >>>>> > > > input fields to viewpoints which can be continually
>>> updated. One
>>> >>>>> > > > could convolve the view matrix with position/orientation
>>> fields, or
>>> >>>>> > > > optionally completely ignore them.____
>>> >>>>> > > >
>>> >>>>> > > > One then could use external SAI to keep updating or
>>> perhaps
>>> >>>>> > > > introduce an environment ProjectionSensor. It would
>>> relay these
>>> >>>>> > > > matrix events from an HMD runtime to then be routed to
>>> the
>>> >>>>> > > > viewpoints.____
>>> >>>>> > > >
>>> >>>>> > > > A second level of navigation is accomplished with
>>> handheld
>>> >>>>> > > > controllers. Until some standard gestures evolve, it
>>> will be
>>> >>>>> > > > necessary to expect custom per scene navigation. X3d
>>> should have a
>>> >>>>> > > > way to sense buttons and axes velocities of such
>>> controllers so
>>> >>>>> > > > these are then available to manipulate the view in some
>>> way.
>>> >>>>> > > > InstantReality has a general IOSensor to that effect. On
>>> the web the
>>> >>>>> > > > GamePad API is a standard which would need to be
>>> used.____
>>> >>>>> > > >
>>> >>>>> > > > Summary: RenderedTexture with stereo modes, matrix
>>> fields for
>>> >>>>> > > > viewpoints, a ProjectionSensor, and a ControllerSensor
>>> would all be
>>> >>>>> > > > candidates for x3d VR support.____
>>> >>>>> > > >
>>> >>>>> > > > Thanks for reading, any thoughts welcome,____
>>> >>>>> > > >
>>> >>>>> > > > Andreas ____
>>> >>>>> > > >
>>> >>>>> > > >
>>> >>>>> > > >
>>> >>>>> > > >
>>> >>>>> > > > _______________________________________________
>>> >>>>> > > > x3d-public mailing list
>>> >>>>> > > >x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>>> >>>>> > > >http://web3d.org/mailman/listinfo/x3d-public_web3d.org <
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org>
>>> >>>>> > > >
>>> >>>>> > >
>>> >>>>> > >
>>> >>>>> > > --
>>> >>>>> > > Holger Seelig
>>> >>>>> > > Mediengestalter Digital ? Digital Media Designer
>>> >>>>> > >
>>> >>>>> > > Scheffelstra?e 31a
>>> >>>>> > > 04277 Leipzig
>>> >>>>> > > Germany
>>> >>>>> > >
>>> >>>>> > > Cellular:+49 1577 147 26 11 <tel:+49%201577%201472611>
>>> >>>>> > > E-Mail: holger.seelig at create3000.de <mailto:
>>> holger.seelig at create3000.de> <mailto:holger.seelig at create3000.de
>>> <mailto:holger.seelig at create3000.de>>
>>> >>>>> > > Web: http://titania.create3000.de <
>>> http://titania.create3000.de>
>>> >>>>> > >
>>> >>>>> > > Future to the fantasy ? ?
>>> >>>>> > >
>>> >>>>> > >
>>> >>>>> > >
>>> >>>>> > > ------------------------------
>>> >>>>> > >
>>> >>>>> > > Subject: Digest Footer
>>> >>>>> > >
>>> >>>>> > > _______________________________________________
>>> >>>>> > > x3d-public mailing list
>>> >>>>> > >x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>>> >>>>> > >http://web3d.org/mailman/listinfo/x3d-public_web3d.org <
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org>
>>> >>>>> > >
>>> >>>>> > >
>>> >>>>> > > ------------------------------
>>> >>>>> > >
>>> >>>>> > > End of x3d-public Digest, Vol 94, Issue 17
>>> >>>>> > > ******************************************
>>> >>>>> >
>>> >>>>> > _______________________________________________
>>> >>>>> > x3d-public mailing list
>>> >>>>> >x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>>> >>>>> >http://web3d.org/mailman/listinfo/x3d-public_web3d.org <
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org>
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> _______________________________________________
>>> >>>> x3d-public mailing list
>>> >>>> x3d-public at web3d.org <x3d-public at web3d.org> <mailto:
>>> x3d-public at web3d.org>
>>> >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>> <http://web3d.org/mailman/listinfo/x3d-public_web3d.org> <
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org>
>>> >>>>
>>> >>>
>>> >>>
>>> >>> all the best, Don
>>> >>> --
>>> >>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu <mailto:brutzman at nps.edu>
>>> >>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149 <tel:(831)%20656-2149>
>>> >>> X3D graphics, virtual worlds, navy roboticshttp://faculty.nps.edu
>>> /brutzman <http://faculty.nps.edu/brutzman>
>>> >>
>>> >>
>>> >
>>> >
>>> > all the best, Don
>>> > --
>>> > Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>> > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>> > X3D graphics, virtual worlds, navy robotics
>>> http://faculty.nps.edu/brutzman
>>>
>>>
>>> --
>>> Andreas Plesch
>>> 39 Barbara Rd.
>>> Waltham, MA 02453
>>>
>>
>>
>>
>> --
>> Andreas Plesch
>> 39 Barbara Rd.
>> Waltham, MA 02453
>>
>
--
Andreas Plesch
39 Barbara Rd.
Waltham, MA 02453
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20170128/91a85e4b/attachment-0001.html>
More information about the x3d-public
mailing list