[x3d-public] ScreenSensor, was: LineSensor

Thu Apr 4 06:43:00 PDT 2019

I am working on a spec. comment on screen aligned PlaneSensor
functionality (aka PointSensor in freeWrl or SpaceSensor in Instant)
and developed an example, a description and suggested spec. language
here:

https://bl.ocks.org/andreasplesch/f196e98c86bc9dc9686a7e5b4acede7d
https://gist.github.com/andreasplesch/f196e98c86bc9dc9686a7e5b4acede7d

This is an important but currently not available feature which cannot
be implemented with a Proto, at least I cannot think of a way to do.

Any feedback will be welcome. Actually, I have a couple comments
myself in a follow-up message.

-Andreas

On Fri, Mar 29, 2019 at 7:39 AM Andreas Plesch <andreasplesch at gmail.com> wrote:
>
> Doug, a
>
> since there is no way to add or edit a spec. comment, I thought it
> would be wise to invest a little more time and effort. Also, while
> working through the issues, it makes sense to be a bit more thorough
> to not have to rediscover solutions later.
>
> I had thought about a more comprehensive comment,  addressing both
> line sensor and screen sensor modes but then decided to divide and
> conquer since these enhancements are separable and since line sensor
> is potentially easier to refine since it already is part of the spec.
>
> That being said, I thought more about what trackPoint_changed should
> provide in the line sensor case. Two ideas:
>
> The trackPoint_changed event could explicitly become undefined, for
> line sensor, and declared not portable between between browsers. After
> all, generally in 3d two lines (the bearing and the sensed line) do
> not intersect.
>
> An alternative resolution may be to redefine the trackPoint_changed
> event specifically for the line sensor case. Instead of considering a
> point on a plane to track, a point on the sensed line is tracked. The
> trackPoint position is then defined as a point of the sensed line by
> adding an adjusted relative translation vector to the initial
> intersection point which is on the line by definition. For this
> purpose the relative translation vector is considered without the
> added offset and is clamped in the fixed dimension and unclamped in
> the variable dimension. Also, the relative translation vector's z
> component (in sensor local coordinates) is zeroed. These operations
> have the overall effect of placing the track point on the sensed line
> near the indicated pointer location, independant of the browser's
> choice of tracking plane orientation.
>
> This is consistent with the plane sensor track point since this track
> point also can be seen as the sum of the initial intersection point
> and the unclamped relative translation vector which here will also
> have zero z component. The only difference is the clamping in the
> fixed dimension for line sensor, where min and maxPosition are equal.
>
> It may be difficult to find concise but sufficient language for the
> spec. I am not sure if it can be much shorter as in the explanation
> above.
>
> Browser currently do not do this, only x_ite is doing what is probably
> effectively equivalent, using the closest point on the line to the
> pointer in screen space. So such a definition would make current
> browsers less compliant.
>
> Here is an example where a modified x3dom is using such a trackPoint:
>
> https://bl.ocks.org/andreasplesch/15ca27439fde2a6a721fbc36d667ea15
>
> I have added this discussion to
> https://github.com/andreasplesch/x3dom/projects/5
>
> As always, any comment or feedback is much appreciated,
>
> -Andreas
>
> > Date: Thu, 28 Mar 2019 08:05:13 -0600
> > From: GPU Group <gpugroup at gmail.com>
> > To: X3D Graphics public mailing list <x3d-public at web3d.org>
> > Subject: Re: [x3d-public] LineSensor
> >
> > Andreas,
> > Looks good.
> > You're taking more care than I would at the spec comment stage. I think
> > there's another process after spec comment stage, that gathers / aggregates
> > and analyzes comments, and seeks agreement on the need for and design of
> > web3d- coherent solution. So your extra care at spec comments would mean
> > less work in the later stages.
> > For me personally, if I was working at the later stage and thinking of
> > web3d as an ongoing evolving spec, I would appreciate also seeing the
> > related point / screen- aligned / viewpoint- aligned sensor case, which
> > would help show what some of us see as stress on the current pointing
> > device sensor component definition, and show value in going beyond
> > minimalistic proprietary workarounds for an awkward sensor definition to
> > something more explicit that may include more configurations, and be more
> > coherent for scene authors.
> > I haven't thought too deeply on wands and 3D mice, or how they may be
> > related, and I think it would help to gather comments on that in case it
> > relates.
> > -Doug
> >
> >
> > On Wed, Mar 27, 2019 at 9:07 PM Andreas Plesch <andreasplesch at gmail.com>
> > wrote:
> >
> > > In preparation for a well presented spec. comment, I developed a
> > > minimal but discerning example scene illustrating the problematic on
> > > edge view for 1D PlaneSensors:
> > >
> > >
> > > https://gist.github.com/andreasplesch/33016fd0918178f5dfcf4632d477f4e4#file-planesensor1donedge-x3d
> > >
> > > For loading straight into browsers:
> > >
> > >
> > > https://gist.githubusercontent.com/andreasplesch/33016fd0918178f5dfcf4632d477f4e4/raw/7672b81e34ff30855eaad53519e0266d25de876b/PlaneSensor1DOnEdge.x3d
> > >
> > > Here in action with x3dom:
> > >
> > > https://bl.ocks.org/andreasplesch/33016fd0918178f5dfcf4632d477f4e4
> > >
> > > Dragging the red cone in strictly conforming X3D browsers should be
> > > ill defined in the on edge views. Testing shows that in these browsers
> > > the cone jumps around or does not move at all as the spec. currently
> > > requires:
> > > - BS contact
> > > - Instant
> > > - x3dom
> > > - freeWrl
> > > In contrast, other browser ignore the spec. in favour of providing the
> > > expected dragging behaviour even in on edge views:
> > > - Octaga
> > > - x-ite
> > > - view3dscene
> > > - H3Dviewer
> > > These browser must have internally a special line sensor mode but it
> > > is unclear what this mode then does. For example, is
> > > trackPoint_changed returned and where should it be since there is no
> > > intersection with the tracking plane ?
> > >
> > > If think I could see that x_ite picks the point on the line closest to
> > > the pointer in screen space and unprojects that point then to local
> > > sensor coordinates.
> > >
> > > How does view3dscene behave ? How does it calculate translation and
> > > track point ?
> > >
> > > The diverging  browser behaviour reinforces the clear need for spec.
> > > refinement addressing this situation.
> > >
> > > Here is my github project page where I put together the spec. comment:
> > >
> > > https://github.com/andreasplesch/x3dom/projects/5
> > >
> > > With regards to trackPoint_changed, I would favor leaving its value up
> > > to the browser, eg. implementation dependent, in the line sensor case
> > > but perhaps there is a simple way to define the value in a way which
> > > is consistent with regular 2d plane sensor use. If trackPoint_changed
> > > is undefined in the line sensor case, then there would be already 4
> > > browsers which comply with a refined PlaneSensor without any updates.
> > >
> > > Any feedback welcome,
> > >
> > > -Andreas
> > >
> > > On Sun, Mar 24, 2019 at 4:05 PM doug sanden <highaspirations at hotmail.com>
> > > wrote:
> > > >
> > > > Sounds great for v4 proposal! Great work Andreas.
> > > > -Doug
> > > >
> > > > ________________________________________
> > > >
> > > > I looked at the freewrl code for PointSensor and agree that it seems to
> > > clamp the translation in local sensor space. Not sure how the 3d clamping
> > > works with the 2d fields, probably the z is just left untouched. If so,
> > > translations clamped in xy but not z would end up off the tracking plane, I
> > > believe.
> > > >
> > > > I do not really think that this is useful for PointSensor. If clamping
> > > in the xy plane of the local sensor space is needed, a regular PlaneSensor
> > > seems like the better solution.
> > > >
> > > > So I am rather satisfied with the proposed way to extend PlaneSensor to
> > > also act as line sensor or point sensor. The line sensor role only needs a
> > > little bit extra prose, and the point sensor aka screen sensor role one
> > > additional field which I think is unavoidable. The slightly modified
> > > meaning of min.maxPosition will take an additional two sentences. There are
> > > two implementations which demonstrate the same functionality albeit in
> > > different ways.
> > > >
> > > > How does that sound for a v4 proposal ? Perhaps there is one already ?
> > > >
> > > > Andreas
> > > >
> > > > ---on the phone---
> > > >
> > > > On Fri, Mar 22, 2019, 7:39 PM GPU Group <gpugroup at gmail.com<mailto:
> > > gpugroup at gmail.com> wrote:
> > > > Andreas - tried your x3dom sample - screen mode works great! Great work!
> > > > For freewrl looks like I was clamping to min, max in 3D - ie if your max
> > > < min, then it ignors, otherwise it clamps in 3D space and I think that's
> > > in Sensor node space. Whether that's useful I don't know.Probably I just
> > > copied and pasted from PlaneSensor without testing or thinking too deep.
> > > > -Doug
> > > >
> > > https://sourceforge.net/p/freewrl/git/ci/develop/tree/freex3d/src/lib/input/SensInterps.c
> > > >  - line 1500 do_PointSensor
> > > >
> > > >
> > > >
> > > > On Fri, Mar 22, 2019 at 2:46 PM Andreas Plesch <andreasplesch at gmail.com
> > > <mailto:andreasplesch at gmail.com>> wrote:
> > > > Hi Doug,I took the plunge and added a planeOrientation='screen' field
> > > > in this exploration:
> > > >
> > > > https://jsfiddle.net/andreasplesch/mzg9dqyc/158/
> > > >
> > > > The idea seems to work as advertised. In addition to dragging the
> > > > colored axis arrows, it is now possible to drag the grey box itself in
> > > > the screen plane whatever the current view may be.
> > > >
> > > > One thing I noticed is that the min/maxPosition field would need to
> > > > have a slightly different meaning since the X and Y axis of the local
> > > > coordinate system are not useful in this case.
> > > >
> > > > Did you have a min/maxPosition field for PointSensor ?
> > > >
> > > > One possible meaning would be to limit translation in the screen plane
> > > > along the horizontal axis and the vertical axis of the screen. I
> > > > thought a bit about that and I think it may be very doable but could
> > > > not quite finish [The problem I am a bit stuck on is how to project
> > > > the intersection point from local coordinates to the new coordinate
> > > > system. Perhaps back to world, then world to camera space using the
> > > > view matrix, then just setting z=0. Presumably this could work for the
> > > > translation directly. Then clamp the translation, and project back:
> > > > inverse view matrix and local transform.]
> > > >
> > > > Another option is to just ignore the field in this case.
> > > >
> > > > Cheers,
> > > >
> > > > -Andreas
> > > >
> > > >
> > > > On Fri, Mar 22, 2019 at 11:25 AM GPU Group <gpugroup at gmail.com<mailto:
> > > gpugroup at gmail.com>> wrote:
> > > > >
> > > > > Great ideas and PlaneOrientation "screen" sounds intuitive, I like it.
> > > And saves a node definition, and you Andreas having solved the linesensor
> > > via planesensor, it makes sense to keep going that way, packing more in the
> > > PlaneSensor node. Keep up the great work.
> > > > > -Doug
> > > > >
> > > > > On Fri, Mar 22, 2019 at 9:12 AM Andreas Plesch <
> > > andreasplesch at gmail.com<mailto:andreasplesch at gmail.com>> wrote:
> > > > >>
> > > > >> Hi Doug,
> > > > >>
> > > > >> I think a PointSensor is a good idea if there is currently no other
> > > > >> way to drag a point around in a plane parallel to the screen. Since
> > > > >> dragging is still restricted to a plane, perhaps ScreenSensor would be
> > > > >> a better name ? This is a common and useful dragging method,
> > > > >> especially if you have well defined viewpoints, like a perfect map
> > > > >> view.
> > > > >>
> > > > >> To be more specific, the tracking plane is defined by the point on the
> > > > >> sibling geometry first indicated when dragging starts and by the
> > > > >> avatar viewing direction as the plane normal. The avatar viewing
> > > > >> direction is the current viewpoint orientation, or the orientation of
> > > > >> a ray to the center of the screen.
> > > > >>
> > > > >> Autooffset, I  think, works the same by just accumulating offsets for
> > > > >> each drag action.
> > > > >>
> > > > >> And translation_changed and trackpoint_changed, I think, would still
> > > > >> be reported in the local coordinate system of the sensor.
> > > > >>
> > > > >> Would it make sense to have a min. and maxPosition for it ? I think
> > > so.
> > > > >>
> > > > >> So I think it would be pretty straightforward to implement based on
> > > > >> the existing PlaneSensor. Perhaps it should become a "screen" mode for
> > > > >> PlaneSensor since it would have pretty much the same fields ?
> > > > >>
> > > > >> Then it may suffice to add only little prose to the PlaneSensor spec.:
> > > > >>
> > > > >> "If the mode field is set to "screen", the tracking plane is not the
> > > > >> XY plane of the local sensor coordinate system. Instead, the tracking
> > > > >> plane is the plane parallel to screen anchored by the initial
> > > > >> intersection point."
> > > > >>
> > > > >> On the other hand, such mode fields are generally not used in any X3D
> > > > >> nodes, probably for good reason. Perhaps planeOrientation would be a
> > > > >> better name, with values "XY" (default), "screen", "XZ", "YZ".
> > > > >>
> > > > >> Although there is no screen in VR, the vertical plane perfectly ahead
> > > > >> in the viewing direction is still an important plane and would be
> > > > >> useful to drag around things in.
> > > > >>
> > > > >> More ideas,
> > > > >>
> > > > >> -Andreas
> > > > >>
> > > > >> On Fri, Mar 22, 2019 at 9:34 AM GPU Group <gpugroup at gmail.com<mailto:
> > > gpugroup at gmail.com>> wrote:
> > > > >> >
> > > > >> > Freewrl also does a PointSensor. I think I submitted that to spec
> > > comments, or I should have/meant to, for v4 consideration. Internally
> > > PointSensor is a planesensor aligned to the screen. So if you EXAMINE
> > > around something that's a PointSensor, you can drag it in a plane parallel
> > > to the screen from the current avatar viewing angle, and it feels like you
> > > can drag it any direction you want by changing the avatar pose.That's
> > > unlike a PlaneSensor that's not following/moving with the avatar.
> > > > >> > And confirming what Andreas said -and in freewrl code for drag
> > > sensors it mentions Max Limper complaining about x3dom PlaneSensor the same
> > > thing about it failing when viewed on-edge.
> > > > >> > -Doug Sanden
> > > > >> >
> > > > >> >
> > > > >> > On Thu, Mar 21, 2019 at 9:46 PM Andreas Plesch <
> > > andreasplesch at gmail.com<mailto:andreasplesch at gmail.com>> wrote:
> > > > >> >>
> > > > >> >> I rethought the LineSensor idea and instead opted to add an
> > > internal
> > > > >> >> line sensor mode to PlaneSensor. The idea is that if the x or y
> > > > >> >> components of both minPosition and maxPosition are 0, PlaneSensor
> > > acts
> > > > >> >> as a LineSensor and is free to use a better intersection plane than
> > > > >> >> the default XY plane which can become completely unusable in the
> > > case
> > > > >> >> when it is viewed on edge.
> > > > >> >>
> > > > >> >> In the implementation used in the fiddle below, the intersection
> > > plane
> > > > >> >> normal in line sensor mode is computed as the cross product of the
> > > > >> >> line axis (x or y axis) with the cross product of the line axis and
> > > > >> >> the view direction when dragging starts:
> > > > >> >>
> > > > >> >> https://jsfiddle.net/andreasplesch/mzg9dqyc/111/
> > > > >> >>
> > > > >> >> It seems to work well and allows programming of a translation
> > > widget
> > > > >> >> in X3D without scripts as shown. On edge view of the PlaneSensor XY
> > > > >> >> plane still works because the XY plane is not actually used.
> > > > >> >>
> > > > >> >> This approach is more spec friendly as it does not require a new
> > > node
> > > > >> >> and just allows the defacto use as LineSensor to become robust.
> > > > >> >> However, since it requires use of another plane than the XY plane,
> > > it
> > > > >> >> may be necessary to add some language. Perhaps:
> > > > >> >>
> > > > >> >> "This technique provides a way to implement a line sensor that maps
> > > > >> >> dragging motion into a translation in one dimension. In this case,
> > > the
> > > > >> >> tracking plane can be any plane that contains the one unconstrained
> > > > >> >> axis (X or Y). A browser is therefore free to choose a tracking
> > > plane
> > > > >> >> suitable to robustly determine intersections, in particular when
> > > the
> > > > >> >> XY plane is unusable. In this case, the generated
> > > trackpoint_changed
> > > > >> >> value becomes implementation dependent."
> > > > >> >>
> > > > >> >> I would prefer this over a LineSensor node, and it was pretty
> > > > >> >> straightforward to add to the x3dom PlaneSensor implementation.
> > > > >> >>
> > > > >> >> I think it would be possible to check for other, equal values than
> > > 0
> > > > >> >> and then use the same approach but I am not sure if it would be
> > > > >> >> useful.
> > > > >> >>
> > > > >> >> If there is feedback, especially on the suggested prose, I will
> > > glad
> > > > >> >> it to add it to a potential standard comment.
> > > > >> >>
> > > > >> >> For example, one could tighten the spec. by giving a formula for
> > > the
> > > > >> >> orientation of a suitable tracking plane in the one dimensional
> > > case
> > > > >> >> but that may be overreach.
> > > > >> >>
> > > > >> >> Cheers, -Andreas
> > > > >> >>
> > > > >> >> On Thu, Mar 21, 2019 at 6:06 PM Andreas Plesch <
> > > andreasplesch at gmail.com<mailto:andreasplesch at gmail.com>> wrote:
> > > > >> >> >
> > > > >> >> > I now understand the need for LineSensor and may give it a try
> > > for
> > > > >> >> > x3dom. I know freewrl has it. It may make sense to replicate it.
> > > > >> >> >
> > > > >> >> > My initial idea would be to use the x axis of the local
> > > coordinate
> > > > >> >> > system as the Line orientation. Or maybe the y axis since X3D
> > > > >> >> > geometries tend to align along y (cylinder, extrusion).
> > > > >> >> >
> > > > >> >> > Then one can construct an intersection plane which includes the
> > > x axis
> > > > >> >> > and another direction at a high angle with the viewing direction
> > > to
> > > > >> >> > get a clean intersection for the inital point and subsequent
> > > points
> > > > >> >> > during dragging.
> > > > >> >> >
> > > > >> >> > Is that how freewrl does it ? I think I saw that it may use an
> > > > >> >> > additional field for the line orientation but it may be best to
> > > avoid
> > > > >> >> > it.
> > > > >> >> >
> > > > >> >> > Andreas
> > > > >> >> >
> > > > >> >> > --
> > > > >> >> > Andreas Plesch
> > > > >> >> > Waltham, MA 02453
> > > > >> >>
> > > > >> >>
> > > > >> >>
> > > > >> >> --
> > > > >> >> Andreas Plesch
> > > > >> >> Waltham, MA 02453
> > > > >> >>
> > > > >> >> _______________________________________________
> > > > >> >> x3d-public mailing list
> > > > >> >> x3d-public at web3d.org<mailto:x3d-public at web3d.org>
> > > > >> >> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Andreas Plesch
> > > > >> Waltham, MA 02453
> > > >
> > > >
> > > >
> > > > --
> > > > Andreas Plesch
> > > > Waltham, MA 02453
> > >
> > >
> > >
> > > --
> > > Andreas Plesch
> > > Waltham, MA 02453
> > >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190328/e06b54d4/attachment.html>
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > x3d-public mailing list
> > x3d-public at web3d.org
> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> >
> >
> > ------------------------------
> >
> > End of x3d-public Digest, Vol 120, Issue 119
> > ********************************************
>
>
>
> --
> Andreas Plesch
> Waltham, MA 02453

-- 
Andreas Plesch
Waltham, MA 02453