[x3d-public] Web3DUX working group minutes 8 April 2020

GPU Group gpugroup at gmail.com
Sun Jun 21 17:45:35 PDT 2020


ALEXA / smart-home interface 2 ways:
- ALEXA to VR: "Alexa, walk"
- VR to ALEXA to bluetooth device; click on lightswitch in scene, to tell
Alexa to tell the (bluethooth receiving) lightswitch to toggle
-Doug.

On Fri, Apr 10, 2020 at 6:53 PM GPU Group <gpugroup at gmail.com> wrote:

> MIXING REAL AND VIRTUAL WORLDS
>
> Imagine having AI and machine vision on the output end construct a
> representative world and send it to the near end, and keep it updated as
> the real world changes. And inputs from the near end steered into the near
> scene would be output by the remote end, after translating into real-world
> outputs.
> That requires just 2 functions:
> realworld2virtualworld()
> virtualworld2realworld()
> -Doug
>
> On Fri, Apr 10, 2020 at 5:42 PM GPU Group <gpugroup at gmail.com> wrote:
>
>> updating virtual world in real time from real world on other end
>>
>> 1) a way to stream skeletal movements and face-texture-patches / face
>> gesture movements
>>
>> https://sites.google.com/site/commutar/home/telepresence
>>
>> - a way to track a face, scrape and send just the facial texture patch,
>> and head gestures
>>
>> - a way to patch a skin texture for incoming facial gestures
>>
>> - and pose head / body
>>
>> 2) a way to steam 'oriented textures' to compose/compile/patch/update a
>> panorama in real-time as a remote camera pans around
>>
>> - for situational awareness, and late-arriving image frames are in
>> correct 3D direction/pose
>>
>> 3) a way to control things remotely ie camera panning, remote telerobots
>>
>> - cluster of nodes relating to AI, machine vision, operator pose-sensing,
>> and communication of pose, and feedback information
>>
>> -Doug Sanden
>>
>>
>>
>> On Fri, Apr 10, 2020 at 5:37 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>>> VR as intermediary to other devices > audio receiving devices >
>>> smarthomeassistants > steering audio
>>> - when we speak into a VR system, it may be necessary to steer the
>>> output audio in what I'll call a VR output device
>>> - currently v4 specs are looking at what I think is steerable stereo
>>> audio output on the input device end
>>> - that would need to be aimable on the output end, for example so you
>>> can tell a specific smarthomeassistant or other audio-activated device to
>>> do something, in such a way it knows its the one being ordered.
>>>
>>> -Doug
>>>
>>>
>>>
>>> On Fri, Apr 10, 2020 at 4:02 PM Feng Liu <LIU_F at mercer.edu> wrote:
>>>
>>>> All brilliant ideas! There are definitely UX involves in the design and
>>>> testing of those apps/AI robots.
>>>> Feng
>>>> ------------------------------
>>>> *From:* GPU Group <gpugroup at gmail.com>
>>>> *Sent:* Friday, April 10, 2020 4:50 PM
>>>> *To:* Nicholas Polys <npolys at vt.edu>
>>>> *Cc:* Feng Liu <LIU_F at mercer.edu>; Sadagic, Amela (CIV) <
>>>> asadagic at nps.edu>; x3d-public at web3D.org <x3d-public at web3d.org>
>>>> *Subject:* Re: [x3d-public] Web3DUX working group minutes 8 April 2020
>>>>
>>>> LOW POWER OUTPUTS
>>>> Instead of human-size robot arms to do remotely what humans want, many
>>>> devices now have their own low-power self-contained motors right-sized for
>>>> each job.
>>>> Imagining stapling 2 pages together at the office, done from home.
>>>> a) high power way: steer an industrial robot to hammer a mechanical
>>>> stapler
>>>> b) low power way: emit a bluetooth command to an electric stapler,
>>>> which has a tiny motor just powerful enough to drive a staple.
>>>> Similarly for many tasks with equipment - just emit bluetooth commands.
>>>> Perhaps where VR can come in handy: by providing a simpler intuitive
>>>> interface to a plethora of self-powered wireless devices.
>>>> For example, in VR you would use a virtual pointer to point at the
>>>> stapler, as if you were going to pound a mechanical stapler. That would be
>>>> translated into bluetooth commands the electric stapler can understand to
>>>> drive a staple.
>>>>
>>>> Similarly for pushing wheelchairs in a hospital to substitute for
>>>> infected personnel:
>>>> a) high power way: command a large walking humanoid robot to push the
>>>> wheel chair
>>>> b) low power way - emit blutooth commands to an electric wheelchair
>>>> which has tiny motors on each wheel.
>>>> And VR could simplify the interface, by clicking on the wheel chair,
>>>> then a chain of points on the floor -a path- and the wheelchair would drive
>>>> itself.
>>>> -Doug
>>>>
>>>> On Fri, Apr 10, 2020 at 12:24 PM GPU Group <gpugroup at gmail.com> wrote:
>>>>
>>>> Wow - you've done some great things Nicholas, so interesting.
>>>> -Doug
>>>>
>>>> Q. what's missing from VR - what's holding it back from solving all the
>>>> world's problems?
>>>> H: weak on outputs:
>>>> - great for input devices and building a virtual world and entering the
>>>> virtual world and perhaps even solving problems in the virtual world.
>>>> x but weak on getting the virtual world results back out to do work in
>>>> the real world
>>>> If so then perhaps this is the next things to work on.
>>>> And I can think of a few problems:
>>>> 1) the real world has a way of updating itself, so keeping virtual
>>>> world version synced is one challenge
>>>> 2) output devices - robot arms tend to be expensive, and the more power
>>>> they have, the more dangerous they are, so safety precautions
>>>> I can think of some nodes for #1, but not used to thinking about #2, I
>>>> draw a blank
>>>>
>>>>
>>>> On Fri, Apr 10, 2020 at 11:41 AM Nicholas Polys <npolys at vt.edu> wrote:
>>>>
>>>> The User Body is an abstraction we developed with IGD as part of
>>>>
>>>> *Polys, Nicholas* and Brutzman, Don and Steed, Anthony and Behr,
>>>> Johannes. (2008). “Future Standards for Immersive VR:  Report on the IEEE
>>>> VR 2007 Workshop*”.* *IEEE Computers Graphics & Applications *Vol. 28,
>>>> Number 2, IEEE Computer Society, 2008.
>>>>
>>>> It is described in detail in the
>>>> Interaction section of  the Instant Player Tutorials site
>>>> http://doc.instantreality.org/tutorial/
>>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.instantreality.org%2Ftutorial%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=mj7MpnDD2lnRoHV9E26%2F%2BvgMWkZyfNB0MsZbv4QmNH0%3D&reserved=0>
>>>>
>>>>
>>>>  We use it to run X3D in our CAVE system and use our 6DOF Wand  and
>>>> picking
>>>>
>>>> with  best regards,
>>>> _n_polys
>>>>
>>>>
>>>> On Fri, Apr 10, 2020 at 1:26 PM GPU Group <gpugroup at gmail.com> wrote:
>>>>
>>>> Yes looks good. Web3d covid-19 response: create some new nodes for it.
>>>> -Doug
>>>> related thoughts:
>>>>
>>>>
>>>> UserBody extension
>>>>
>>>> - interesting and reminded me a bit of the 3Dmouse/spacemouse
>>>>
>>>>  https://www.3dconnexion.com/products/spacemouse.html
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.3dconnexion.com%2Fproducts%2Fspacemouse.html&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=GVWNMe7%2BfjR9ainWKY4IztYDiVc%2BbF1pVeJSKoDVmK8%3D&reserved=0>
>>>>
>>>>
>>>> issue: if it's 3rd person, it can be hard to follow, for example
>>>>
>>>> https://www.youtube.com/watch?v=UC6suYVm60s
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DUC6suYVm60s&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=1CUyGpGYqfcdt0oCpXrf9GyVelLxeHtak1tL0ATrE9s%3D&reserved=0>
>>>>
>>>> - the soccer players are wearing HMDs. The HMDs show the soccer field
>>>> from above - that's all they see. That's like a 3rd person shooter.
>>>> As they look down on the field from 3rd person, they have to figure out
>>>> how to move from first-person.
>>>>
>>>>
>>>> To avoid that 3rd-person problem while decoupling the viewpiont from
>>>> UserBody ray-casting and collision,  UserBody can still be coupled to the
>>>> avatar travel/pose -like your arms are attached to your body- so move with
>>>> the avatar / relative to the avatar. That might not be a requirement of any
>>>> UserBody nodes, but might be how they are easiest to use.
>>>>
>>>>
>>>> Maybe a first step is to develop some emulators / simulators for
>>>> various new types of input devices to control UserBody.
>>>>
>>>>
>>>> more...
>>>> Multitouch x3d
>>>> In freewrl I have WM_TOUCH windows 7-10 style independent (non-gesture)
>>>> touch events being handled independently / non-gesture-wise in windows
>>>> version, and in such a way you can drag multiple drag sensors
>>>> simultaneously - a touchID is associated with each drag during its life
>>>> (and touchIDs can be recycled).
>>>>
>>>> Multitouch Emulation for X3D
>>>>  Freewrl -all platforms- has a built-in multi-touch emulator (current
>>>> develop branch commandline --touchtype 1 or options panel),
>>>> - to use, with mouse, RMB-right-mouse-button-click to create a new
>>>> touch/drag LMB to drag it, and another RMB to delete it. Can create several
>>>> and have them in various drag states using regular mouse.
>>>>
>>>> General Multitouch emulation for windows 10
>>>> for win10 I developed a touchServer and touchSender for emulating a
>>>> multi-touch device on a 2nd computer and the server via tcp injects the
>>>> touches into the desktop for any application
>>>>
>>>> Multi-touch application
>>>> I had developed a play game for kids for a (now obsolete) kids
>>>> mutlitouch table
>>>>
>>>> https://support.smarttech.com/hardware/other-hardware/tables
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupport.smarttech.com%2Fhardware%2Fother-hardware%2Ftables&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=D53czbwljxKcC7Fj7Rzj1vy8R3a5KR8V86DCafuHywQ%3D&reserved=0>
>>>>
>>>>
>>>> for organic chemistry - assembling carbon atom chains - not in x3d,
>>>> they had their own API - and could have up to 16 simaltaneous touches/drags.
>>>>
>>>>
>>>>
>>>> On Fri, Apr 10, 2020 at 9:38 AM Nicholas Polys <npolys at vt.edu> wrote:
>>>>
>>>>
>>>>
>>>> wondering if this is appropriate for X3D 4.0
>>>>
>>>> https://dl.acm.org/doi/abs/10.1145/1394209.1394218
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fabs%2F10.1145%2F1394209.1394218&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406710405&sdata=gUjMCIlKlzej5aqWxOhxjX98KpfMpL2gvM8uVRmJ%2FiE%3D&reserved=0>
>>>>
>>>>
>>>>
>>>> On Fri, Apr 10, 2020 at 9:04 AM GPU Group <gpugroup at gmail.com> wrote:
>>>>
>>>> COVID-19 > social distancing > teleworking > tools for teleworking >
>>>> telepresence / teletravel / remote work
>>>> Economies are taking big hits from the social distancing measures - in
>>>> the trillions globally.
>>>> So if there was some ways to work better from a distance, that would
>>>> have benefits:
>>>> - reduce spread rate of transmissible diseases that kill people
>>>> - reduce the cost to society of transportation
>>>> - keep economy going during strict social distancing 'lockdowns'
>>>> -- and reduce the cost to governments of social programs to support
>>>> unemployed workers
>>>> - keep key essential workers employed while waiting for test results
>>>> - reduce GHG emissions / co-benefits of meeting Paris and net-zero-2050
>>>> early
>>>> But what precisely, and how relate to web3d?
>>>> -Doug Sanden
>>>> some of my old thoughts:
>>>> https://sites.google.com/site/commutar/home/telerobotics
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelerobotics&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=an5ZQfN2MglYrdM0nHCH6d%2F0xzcDyVUPHyocPyOyCHM%3D&reserved=0>
>>>>
>>>> https://sites.google.com/site/commutar/home/telepresence
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelepresence&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=7Jn0fw2%2FaDqtiuuMYlrrw0U3F6yp8urBuPxkF7e3Slc%3D&reserved=0>
>>>>
>>>>
>>>> On Thu, Apr 9, 2020 at 10:48 PM Feng Liu <LIU_F at mercer.edu> wrote:
>>>>
>>>>
>>>> Web3DUX Group Meeting minutes
>>>> Date: April 8, 2020
>>>> Attendees: Amela Sadagic, Don Brutzman, Richard F. Puk and Feng Liu
>>>> Regrets: Nicholas Polys;
>>>>
>>>> Items discussed:
>>>> 1. COVID-19 and Current Web3D / X3D contributions discussion:
>>>> - Everyone attended the meeting are doing well.
>>>> - NIH 3D print exchange site – enable X3D view in
>>>> a browser: https://3dprint.nih.gov/discover/3dpx/013429/x3d
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2F3dprint.nih.gov%2Fdiscover%2F3dpx%2F013429%2Fx3d&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=IUKhA7fYk71r5nSA5XVqaCShoC8nyROOkpbxzpjTbGo%3D&reserved=0>
>>>> - America made provide a platform publishing 3D design of the fighting
>>>> COVID-19 supplies: https://www.americamakes.us/
>>>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.americamakes.us%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=3Tg3elUbPA%2F3GcWoGlSAJ7NYl5vC7HTUlkOd55%2FvGJQ%3D&reserved=0>
>>>>
>>>> - Amela’s team is working on optimizing the 3D model with a goal of
>>>> improving usability. There are many factors (adoption procedure and policy,
>>>> etc.) involved before a mask/a design got approval
>>>> - Feng expressed her interest in inviting her summer students to
>>>> participate in the effort.
>>>> - Don reminded us to align the working group's long term goals with the
>>>> current effort of the involvement of fighting COVID-19. To identify Web3D +
>>>> UX research opportunities with the involvements.
>>>> - Feng is going to looking for COVID-19 research
>>>> funding opportunities regarding Web3D and UX areas.
>>>>
>>>> 2. Update the report from the last meeting. No progress on the
>>>> following items so far
>>>> - Due to COVID-19 impact. The following item still needs to be done.
>>>> - Review Publications on Three types of interactions
>>>> -- Create a folder in one drive – share with you all to collect
>>>> Literature Review Publications on Three types of interactions – Feng (Done)
>>>> -- Develop a spreadsheet to categorize the subjects with the papers we
>>>> review  - Amela.
>>>> -- List an “other” category for other UX related topics
>>>> -- Collecting papers and put them into different categories – Amela and
>>>> Feng
>>>> -- Collect data/currents publications/papers and authors to invite to
>>>> the group for later
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> x3d-public mailing list
>>>> x3d-public at web3d.org
>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>>>
>>>> _______________________________________________
>>>> x3d-public mailing list
>>>> x3d-public at web3d.org
>>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>>>
>>>>
>>>>
>>>> --
>>>> Nicholas F. Polys, Ph.D.
>>>>
>>>> Director of Visual Computing
>>>> Virginia Tech Research Computing
>>>>
>>>> Affiliate Professor
>>>> Virginia Tech Department of Computer Science
>>>>
>>>>
>>>>
>>>> --
>>>> Nicholas F. Polys, Ph.D.
>>>>
>>>> Director of Visual Computing
>>>> Virginia Tech Research Computing
>>>>
>>>> Affiliate Professor
>>>> Virginia Tech Department of Computer Science
>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200621/f6c41057/attachment-0001.html>


More information about the x3d-public mailing list