[x3d-public] Web3DUX working group minutes 8 April 2020

Fri Apr 10 17:53:34 PDT 2020

MIXING REAL AND VIRTUAL WORLDS

Imagine having AI and machine vision on the output end construct a
representative world and send it to the near end, and keep it updated as
the real world changes. And inputs from the near end steered into the near
scene would be output by the remote end, after translating into real-world
outputs.
That requires just 2 functions:
realworld2virtualworld()
virtualworld2realworld()
-Doug

On Fri, Apr 10, 2020 at 5:42 PM GPU Group <gpugroup at gmail.com> wrote:

> updating virtual world in real time from real world on other end
>
> 1) a way to stream skeletal movements and face-texture-patches / face
> gesture movements
>
> https://sites.google.com/site/commutar/home/telepresence
>
> - a way to track a face, scrape and send just the facial texture patch,
> and head gestures
>
> - a way to patch a skin texture for incoming facial gestures
>
> - and pose head / body
>
> 2) a way to steam 'oriented textures' to compose/compile/patch/update a
> panorama in real-time as a remote camera pans around
>
> - for situational awareness, and late-arriving image frames are in correct
> 3D direction/pose
>
> 3) a way to control things remotely ie camera panning, remote telerobots
>
> - cluster of nodes relating to AI, machine vision, operator pose-sensing,
> and communication of pose, and feedback information
>
> -Doug Sanden
>
>
>
> On Fri, Apr 10, 2020 at 5:37 PM GPU Group <gpugroup at gmail.com> wrote:
>
>> VR as intermediary to other devices > audio receiving devices >
>> smarthomeassistants > steering audio
>> - when we speak into a VR system, it may be necessary to steer the output
>> audio in what I'll call a VR output device
>> - currently v4 specs are looking at what I think is steerable stereo
>> audio output on the input device end
>> - that would need to be aimable on the output end, for example so you can
>> tell a specific smarthomeassistant or other audio-activated device to do
>> something, in such a way it knows its the one being ordered.
>>
>> -Doug
>>
>>
>>
>> On Fri, Apr 10, 2020 at 4:02 PM Feng Liu <LIU_F at mercer.edu> wrote:
>>
>>> All brilliant ideas! There are definitely UX involves in the design and
>>> testing of those apps/AI robots.
>>> Feng
>>> ------------------------------
>>> *From:* GPU Group <gpugroup at gmail.com>
>>> *Sent:* Friday, April 10, 2020 4:50 PM
>>> *To:* Nicholas Polys <npolys at vt.edu>
>>> *Cc:* Feng Liu <LIU_F at mercer.edu>; Sadagic, Amela (CIV) <
>>> asadagic at nps.edu>; x3d-public at web3D.org <x3d-public at web3d.org>
>>> *Subject:* Re: [x3d-public] Web3DUX working group minutes 8 April 2020
>>>
>>> LOW POWER OUTPUTS
>>> Instead of human-size robot arms to do remotely what humans want, many
>>> devices now have their own low-power self-contained motors right-sized for
>>> each job.
>>> Imagining stapling 2 pages together at the office, done from home.
>>> a) high power way: steer an industrial robot to hammer a mechanical
>>> stapler
>>> b) low power way: emit a bluetooth command to an electric stapler, which
>>> has a tiny motor just powerful enough to drive a staple.
>>> Similarly for many tasks with equipment - just emit bluetooth commands.
>>> Perhaps where VR can come in handy: by providing a simpler intuitive
>>> interface to a plethora of self-powered wireless devices.
>>> For example, in VR you would use a virtual pointer to point at the
>>> stapler, as if you were going to pound a mechanical stapler. That would be
>>> translated into bluetooth commands the electric stapler can understand to
>>> drive a staple.
>>>
>>> Similarly for pushing wheelchairs in a hospital to substitute for
>>> infected personnel:
>>> a) high power way: command a large walking humanoid robot to push the
>>> wheel chair
>>> b) low power way - emit blutooth commands to an electric wheelchair
>>> which has tiny motors on each wheel.
>>> And VR could simplify the interface, by clicking on the wheel chair,
>>> then a chain of points on the floor -a path- and the wheelchair would drive
>>> itself.
>>> -Doug
>>>
>>> On Fri, Apr 10, 2020 at 12:24 PM GPU Group <gpugroup at gmail.com> wrote:
>>>
>>> Wow - you've done some great things Nicholas, so interesting.
>>> -Doug
>>>
>>> Q. what's missing from VR - what's holding it back from solving all the
>>> world's problems?
>>> H: weak on outputs:
>>> - great for input devices and building a virtual world and entering the
>>> virtual world and perhaps even solving problems in the virtual world.
>>> x but weak on getting the virtual world results back out to do work in
>>> the real world
>>> If so then perhaps this is the next things to work on.
>>> And I can think of a few problems:
>>> 1) the real world has a way of updating itself, so keeping virtual world
>>> version synced is one challenge
>>> 2) output devices - robot arms tend to be expensive, and the more power
>>> they have, the more dangerous they are, so safety precautions
>>> I can think of some nodes for #1, but not used to thinking about #2, I
>>> draw a blank
>>>
>>>
>>> On Fri, Apr 10, 2020 at 11:41 AM Nicholas Polys <npolys at vt.edu> wrote:
>>>
>>> The User Body is an abstraction we developed with IGD as part of
>>>
>>> *Polys, Nicholas* and Brutzman, Don and Steed, Anthony and Behr,
>>> Johannes. (2008). “Future Standards for Immersive VR:  Report on the IEEE
>>> VR 2007 Workshop*”.* *IEEE Computers Graphics & Applications *Vol. 28,
>>> Number 2, IEEE Computer Society, 2008.
>>>
>>> It is described in detail in the
>>> Interaction section of  the Instant Player Tutorials site
>>> http://doc.instantreality.org/tutorial/
>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.instantreality.org%2Ftutorial%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=mj7MpnDD2lnRoHV9E26%2F%2BvgMWkZyfNB0MsZbv4QmNH0%3D&reserved=0>
>>>
>>>
>>>  We use it to run X3D in our CAVE system and use our 6DOF Wand  and
>>> picking
>>>
>>> with  best regards,
>>> _n_polys
>>>
>>>
>>> On Fri, Apr 10, 2020 at 1:26 PM GPU Group <gpugroup at gmail.com> wrote:
>>>
>>> Yes looks good. Web3d covid-19 response: create some new nodes for it.
>>> -Doug
>>> related thoughts:
>>>
>>>
>>> UserBody extension
>>>
>>> - interesting and reminded me a bit of the 3Dmouse/spacemouse
>>>
>>>  https://www.3dconnexion.com/products/spacemouse.html
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.3dconnexion.com%2Fproducts%2Fspacemouse.html&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=GVWNMe7%2BfjR9ainWKY4IztYDiVc%2BbF1pVeJSKoDVmK8%3D&reserved=0>
>>>
>>>
>>> issue: if it's 3rd person, it can be hard to follow, for example
>>>
>>> https://www.youtube.com/watch?v=UC6suYVm60s
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DUC6suYVm60s&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=1CUyGpGYqfcdt0oCpXrf9GyVelLxeHtak1tL0ATrE9s%3D&reserved=0>
>>>
>>> - the soccer players are wearing HMDs. The HMDs show the soccer field
>>> from above - that's all they see. That's like a 3rd person shooter.
>>> As they look down on the field from 3rd person, they have to figure out
>>> how to move from first-person.
>>>
>>>
>>> To avoid that 3rd-person problem while decoupling the viewpiont from
>>> UserBody ray-casting and collision,  UserBody can still be coupled to the
>>> avatar travel/pose -like your arms are attached to your body- so move with
>>> the avatar / relative to the avatar. That might not be a requirement of any
>>> UserBody nodes, but might be how they are easiest to use.
>>>
>>>
>>> Maybe a first step is to develop some emulators / simulators for various
>>> new types of input devices to control UserBody.
>>>
>>>
>>> more...
>>> Multitouch x3d
>>> In freewrl I have WM_TOUCH windows 7-10 style independent (non-gesture)
>>> touch events being handled independently / non-gesture-wise in windows
>>> version, and in such a way you can drag multiple drag sensors
>>> simultaneously - a touchID is associated with each drag during its life
>>> (and touchIDs can be recycled).
>>>
>>> Multitouch Emulation for X3D
>>>  Freewrl -all platforms- has a built-in multi-touch emulator (current
>>> develop branch commandline --touchtype 1 or options panel),
>>> - to use, with mouse, RMB-right-mouse-button-click to create a new
>>> touch/drag LMB to drag it, and another RMB to delete it. Can create several
>>> and have them in various drag states using regular mouse.
>>>
>>> General Multitouch emulation for windows 10
>>> for win10 I developed a touchServer and touchSender for emulating a
>>> multi-touch device on a 2nd computer and the server via tcp injects the
>>> touches into the desktop for any application
>>>
>>> Multi-touch application
>>> I had developed a play game for kids for a (now obsolete) kids
>>> mutlitouch table
>>>
>>> https://support.smarttech.com/hardware/other-hardware/tables
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupport.smarttech.com%2Fhardware%2Fother-hardware%2Ftables&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=D53czbwljxKcC7Fj7Rzj1vy8R3a5KR8V86DCafuHywQ%3D&reserved=0>
>>>
>>>
>>> for organic chemistry - assembling carbon atom chains - not in x3d, they
>>> had their own API - and could have up to 16 simaltaneous touches/drags.
>>>
>>>
>>>
>>> On Fri, Apr 10, 2020 at 9:38 AM Nicholas Polys <npolys at vt.edu> wrote:
>>>
>>>
>>>
>>> wondering if this is appropriate for X3D 4.0
>>>
>>> https://dl.acm.org/doi/abs/10.1145/1394209.1394218
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fabs%2F10.1145%2F1394209.1394218&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406710405&sdata=gUjMCIlKlzej5aqWxOhxjX98KpfMpL2gvM8uVRmJ%2FiE%3D&reserved=0>
>>>
>>>
>>>
>>> On Fri, Apr 10, 2020 at 9:04 AM GPU Group <gpugroup at gmail.com> wrote:
>>>
>>> COVID-19 > social distancing > teleworking > tools for teleworking >
>>> telepresence / teletravel / remote work
>>> Economies are taking big hits from the social distancing measures - in
>>> the trillions globally.
>>> So if there was some ways to work better from a distance, that would
>>> have benefits:
>>> - reduce spread rate of transmissible diseases that kill people
>>> - reduce the cost to society of transportation
>>> - keep economy going during strict social distancing 'lockdowns'
>>> -- and reduce the cost to governments of social programs to support
>>> unemployed workers
>>> - keep key essential workers employed while waiting for test results
>>> - reduce GHG emissions / co-benefits of meeting Paris and net-zero-2050
>>> early
>>> But what precisely, and how relate to web3d?
>>> -Doug Sanden
>>> some of my old thoughts:
>>> https://sites.google.com/site/commutar/home/telerobotics
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelerobotics&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=an5ZQfN2MglYrdM0nHCH6d%2F0xzcDyVUPHyocPyOyCHM%3D&reserved=0>
>>>
>>> https://sites.google.com/site/commutar/home/telepresence
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelepresence&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=7Jn0fw2%2FaDqtiuuMYlrrw0U3F6yp8urBuPxkF7e3Slc%3D&reserved=0>
>>>
>>>
>>> On Thu, Apr 9, 2020 at 10:48 PM Feng Liu <LIU_F at mercer.edu> wrote:
>>>
>>>
>>> Web3DUX Group Meeting minutes
>>> Date: April 8, 2020
>>> Attendees: Amela Sadagic, Don Brutzman, Richard F. Puk and Feng Liu
>>> Regrets: Nicholas Polys;
>>>
>>> Items discussed:
>>> 1. COVID-19 and Current Web3D / X3D contributions discussion:
>>> - Everyone attended the meeting are doing well.
>>> - NIH 3D print exchange site – enable X3D view in
>>> a browser: https://3dprint.nih.gov/discover/3dpx/013429/x3d
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2F3dprint.nih.gov%2Fdiscover%2F3dpx%2F013429%2Fx3d&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=IUKhA7fYk71r5nSA5XVqaCShoC8nyROOkpbxzpjTbGo%3D&reserved=0>
>>> - America made provide a platform publishing 3D design of the fighting
>>> COVID-19 supplies: https://www.americamakes.us/
>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.americamakes.us%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=3Tg3elUbPA%2F3GcWoGlSAJ7NYl5vC7HTUlkOd55%2FvGJQ%3D&reserved=0>
>>>
>>> - Amela’s team is working on optimizing the 3D model with a goal of
>>> improving usability. There are many factors (adoption procedure and policy,
>>> etc.) involved before a mask/a design got approval
>>> - Feng expressed her interest in inviting her summer students to
>>> participate in the effort.
>>> - Don reminded us to align the working group's long term goals with the
>>> current effort of the involvement of fighting COVID-19. To identify Web3D +
>>> UX research opportunities with the involvements.
>>> - Feng is going to looking for COVID-19 research
>>> funding opportunities regarding Web3D and UX areas.
>>>
>>> 2. Update the report from the last meeting. No progress on the following
>>> items so far
>>> - Due to COVID-19 impact. The following item still needs to be done.
>>> - Review Publications on Three types of interactions
>>> -- Create a folder in one drive – share with you all to collect
>>> Literature Review Publications on Three types of interactions – Feng (Done)
>>> -- Develop a spreadsheet to categorize the subjects with the papers we
>>> review  - Amela.
>>> -- List an “other” category for other UX related topics
>>> -- Collecting papers and put them into different categories – Amela and
>>> Feng
>>> -- Collect data/currents publications/papers and authors to invite to
>>> the group for later
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> x3d-public mailing list
>>> x3d-public at web3d.org
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>>
>>> _______________________________________________
>>> x3d-public mailing list
>>> x3d-public at web3d.org
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>>
>>>
>>>
>>> --
>>> Nicholas F. Polys, Ph.D.
>>>
>>> Director of Visual Computing
>>> Virginia Tech Research Computing
>>>
>>> Affiliate Professor
>>> Virginia Tech Department of Computer Science
>>>
>>>
>>>
>>> --
>>> Nicholas F. Polys, Ph.D.
>>>
>>> Director of Visual Computing
>>> Virginia Tech Research Computing
>>>
>>> Affiliate Professor
>>> Virginia Tech Department of Computer Science
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200410/215e5634/attachment-0001.html>