[x3d-public] Web3DUX working group minutes 8 April 2020

Fri Apr 10 16:42:00 PDT 2020

updating virtual world in real time from real world on other end

1) a way to stream skeletal movements and face-texture-patches / face
gesture movements

https://sites.google.com/site/commutar/home/telepresence

- a way to track a face, scrape and send just the facial texture patch, and
head gestures

- a way to patch a skin texture for incoming facial gestures

- and pose head / body

2) a way to steam 'oriented textures' to compose/compile/patch/update a
panorama in real-time as a remote camera pans around

- for situational awareness, and late-arriving image frames are in correct
3D direction/pose

3) a way to control things remotely ie camera panning, remote telerobots

- cluster of nodes relating to AI, machine vision, operator pose-sensing,
and communication of pose, and feedback information

-Doug Sanden

On Fri, Apr 10, 2020 at 5:37 PM GPU Group <gpugroup at gmail.com> wrote:

> VR as intermediary to other devices > audio receiving devices >
> smarthomeassistants > steering audio
> - when we speak into a VR system, it may be necessary to steer the output
> audio in what I'll call a VR output device
> - currently v4 specs are looking at what I think is steerable stereo audio
> output on the input device end
> - that would need to be aimable on the output end, for example so you can
> tell a specific smarthomeassistant or other audio-activated device to do
> something, in such a way it knows its the one being ordered.
>
> -Doug
>
>
>
> On Fri, Apr 10, 2020 at 4:02 PM Feng Liu <LIU_F at mercer.edu> wrote:
>
>> All brilliant ideas! There are definitely UX involves in the design and
>> testing of those apps/AI robots.
>> Feng
>> ------------------------------
>> *From:* GPU Group <gpugroup at gmail.com>
>> *Sent:* Friday, April 10, 2020 4:50 PM
>> *To:* Nicholas Polys <npolys at vt.edu>
>> *Cc:* Feng Liu <LIU_F at mercer.edu>; Sadagic, Amela (CIV) <asadagic at nps.edu>;
>> x3d-public at web3D.org <x3d-public at web3d.org>
>> *Subject:* Re: [x3d-public] Web3DUX working group minutes 8 April 2020
>>
>> LOW POWER OUTPUTS
>> Instead of human-size robot arms to do remotely what humans want, many
>> devices now have their own low-power self-contained motors right-sized for
>> each job.
>> Imagining stapling 2 pages together at the office, done from home.
>> a) high power way: steer an industrial robot to hammer a mechanical
>> stapler
>> b) low power way: emit a bluetooth command to an electric stapler, which
>> has a tiny motor just powerful enough to drive a staple.
>> Similarly for many tasks with equipment - just emit bluetooth commands.
>> Perhaps where VR can come in handy: by providing a simpler intuitive
>> interface to a plethora of self-powered wireless devices.
>> For example, in VR you would use a virtual pointer to point at the
>> stapler, as if you were going to pound a mechanical stapler. That would be
>> translated into bluetooth commands the electric stapler can understand to
>> drive a staple.
>>
>> Similarly for pushing wheelchairs in a hospital to substitute for
>> infected personnel:
>> a) high power way: command a large walking humanoid robot to push the
>> wheel chair
>> b) low power way - emit blutooth commands to an electric wheelchair which
>> has tiny motors on each wheel.
>> And VR could simplify the interface, by clicking on the wheel chair, then
>> a chain of points on the floor -a path- and the wheelchair would drive
>> itself.
>> -Doug
>>
>> On Fri, Apr 10, 2020 at 12:24 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>> Wow - you've done some great things Nicholas, so interesting.
>> -Doug
>>
>> Q. what's missing from VR - what's holding it back from solving all the
>> world's problems?
>> H: weak on outputs:
>> - great for input devices and building a virtual world and entering the
>> virtual world and perhaps even solving problems in the virtual world.
>> x but weak on getting the virtual world results back out to do work in
>> the real world
>> If so then perhaps this is the next things to work on.
>> And I can think of a few problems:
>> 1) the real world has a way of updating itself, so keeping virtual world
>> version synced is one challenge
>> 2) output devices - robot arms tend to be expensive, and the more power
>> they have, the more dangerous they are, so safety precautions
>> I can think of some nodes for #1, but not used to thinking about #2, I
>> draw a blank
>>
>>
>> On Fri, Apr 10, 2020 at 11:41 AM Nicholas Polys <npolys at vt.edu> wrote:
>>
>> The User Body is an abstraction we developed with IGD as part of
>>
>> *Polys, Nicholas* and Brutzman, Don and Steed, Anthony and Behr,
>> Johannes. (2008). “Future Standards for Immersive VR:  Report on the IEEE
>> VR 2007 Workshop*”.* *IEEE Computers Graphics & Applications *Vol. 28,
>> Number 2, IEEE Computer Society, 2008.
>>
>> It is described in detail in the
>> Interaction section of  the Instant Player Tutorials site
>> http://doc.instantreality.org/tutorial/
>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdoc.instantreality.org%2Ftutorial%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=mj7MpnDD2lnRoHV9E26%2F%2BvgMWkZyfNB0MsZbv4QmNH0%3D&reserved=0>
>>
>>
>>  We use it to run X3D in our CAVE system and use our 6DOF Wand  and
>> picking
>>
>> with  best regards,
>> _n_polys
>>
>>
>> On Fri, Apr 10, 2020 at 1:26 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>> Yes looks good. Web3d covid-19 response: create some new nodes for it.
>> -Doug
>> related thoughts:
>>
>>
>> UserBody extension
>>
>> - interesting and reminded me a bit of the 3Dmouse/spacemouse
>>
>>  https://www.3dconnexion.com/products/spacemouse.html
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.3dconnexion.com%2Fproducts%2Fspacemouse.html&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406690418&sdata=GVWNMe7%2BfjR9ainWKY4IztYDiVc%2BbF1pVeJSKoDVmK8%3D&reserved=0>
>>
>>
>> issue: if it's 3rd person, it can be hard to follow, for example
>>
>> https://www.youtube.com/watch?v=UC6suYVm60s
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DUC6suYVm60s&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=1CUyGpGYqfcdt0oCpXrf9GyVelLxeHtak1tL0ATrE9s%3D&reserved=0>
>>
>> - the soccer players are wearing HMDs. The HMDs show the soccer field
>> from above - that's all they see. That's like a 3rd person shooter.
>> As they look down on the field from 3rd person, they have to figure out
>> how to move from first-person.
>>
>>
>> To avoid that 3rd-person problem while decoupling the viewpiont from
>> UserBody ray-casting and collision,  UserBody can still be coupled to the
>> avatar travel/pose -like your arms are attached to your body- so move with
>> the avatar / relative to the avatar. That might not be a requirement of any
>> UserBody nodes, but might be how they are easiest to use.
>>
>>
>> Maybe a first step is to develop some emulators / simulators for various
>> new types of input devices to control UserBody.
>>
>>
>> more...
>> Multitouch x3d
>> In freewrl I have WM_TOUCH windows 7-10 style independent (non-gesture)
>> touch events being handled independently / non-gesture-wise in windows
>> version, and in such a way you can drag multiple drag sensors
>> simultaneously - a touchID is associated with each drag during its life
>> (and touchIDs can be recycled).
>>
>> Multitouch Emulation for X3D
>>  Freewrl -all platforms- has a built-in multi-touch emulator (current
>> develop branch commandline --touchtype 1 or options panel),
>> - to use, with mouse, RMB-right-mouse-button-click to create a new
>> touch/drag LMB to drag it, and another RMB to delete it. Can create several
>> and have them in various drag states using regular mouse.
>>
>> General Multitouch emulation for windows 10
>> for win10 I developed a touchServer and touchSender for emulating a
>> multi-touch device on a 2nd computer and the server via tcp injects the
>> touches into the desktop for any application
>>
>> Multi-touch application
>> I had developed a play game for kids for a (now obsolete) kids
>> mutlitouch table
>>
>> https://support.smarttech.com/hardware/other-hardware/tables
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsupport.smarttech.com%2Fhardware%2Fother-hardware%2Ftables&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406700411&sdata=D53czbwljxKcC7Fj7Rzj1vy8R3a5KR8V86DCafuHywQ%3D&reserved=0>
>>
>>
>> for organic chemistry - assembling carbon atom chains - not in x3d, they
>> had their own API - and could have up to 16 simaltaneous touches/drags.
>>
>>
>>
>> On Fri, Apr 10, 2020 at 9:38 AM Nicholas Polys <npolys at vt.edu> wrote:
>>
>>
>>
>> wondering if this is appropriate for X3D 4.0
>>
>> https://dl.acm.org/doi/abs/10.1145/1394209.1394218
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdl.acm.org%2Fdoi%2Fabs%2F10.1145%2F1394209.1394218&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406710405&sdata=gUjMCIlKlzej5aqWxOhxjX98KpfMpL2gvM8uVRmJ%2FiE%3D&reserved=0>
>>
>>
>>
>> On Fri, Apr 10, 2020 at 9:04 AM GPU Group <gpugroup at gmail.com> wrote:
>>
>> COVID-19 > social distancing > teleworking > tools for teleworking >
>> telepresence / teletravel / remote work
>> Economies are taking big hits from the social distancing measures - in
>> the trillions globally.
>> So if there was some ways to work better from a distance, that would have
>> benefits:
>> - reduce spread rate of transmissible diseases that kill people
>> - reduce the cost to society of transportation
>> - keep economy going during strict social distancing 'lockdowns'
>> -- and reduce the cost to governments of social programs to support
>> unemployed workers
>> - keep key essential workers employed while waiting for test results
>> - reduce GHG emissions / co-benefits of meeting Paris and net-zero-2050
>> early
>> But what precisely, and how relate to web3d?
>> -Doug Sanden
>> some of my old thoughts:
>> https://sites.google.com/site/commutar/home/telerobotics
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelerobotics&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=an5ZQfN2MglYrdM0nHCH6d%2F0xzcDyVUPHyocPyOyCHM%3D&reserved=0>
>>
>> https://sites.google.com/site/commutar/home/telepresence
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsites.google.com%2Fsite%2Fcommutar%2Fhome%2Ftelepresence&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406720403&sdata=7Jn0fw2%2FaDqtiuuMYlrrw0U3F6yp8urBuPxkF7e3Slc%3D&reserved=0>
>>
>>
>> On Thu, Apr 9, 2020 at 10:48 PM Feng Liu <LIU_F at mercer.edu> wrote:
>>
>>
>> Web3DUX Group Meeting minutes
>> Date: April 8, 2020
>> Attendees: Amela Sadagic, Don Brutzman, Richard F. Puk and Feng Liu
>> Regrets: Nicholas Polys;
>>
>> Items discussed:
>> 1. COVID-19 and Current Web3D / X3D contributions discussion:
>> - Everyone attended the meeting are doing well.
>> - NIH 3D print exchange site – enable X3D view in
>> a browser: https://3dprint.nih.gov/discover/3dpx/013429/x3d
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2F3dprint.nih.gov%2Fdiscover%2F3dpx%2F013429%2Fx3d&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=IUKhA7fYk71r5nSA5XVqaCShoC8nyROOkpbxzpjTbGo%3D&reserved=0>
>> - America made provide a platform publishing 3D design of the fighting
>> COVID-19 supplies: https://www.americamakes.us/
>> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.americamakes.us%2F&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406730396&sdata=3Tg3elUbPA%2F3GcWoGlSAJ7NYl5vC7HTUlkOd55%2FvGJQ%3D&reserved=0>
>>
>> - Amela’s team is working on optimizing the 3D model with a goal of
>> improving usability. There are many factors (adoption procedure and policy,
>> etc.) involved before a mask/a design got approval
>> - Feng expressed her interest in inviting her summer students to
>> participate in the effort.
>> - Don reminded us to align the working group's long term goals with the
>> current effort of the involvement of fighting COVID-19. To identify Web3D +
>> UX research opportunities with the involvements.
>> - Feng is going to looking for COVID-19 research
>> funding opportunities regarding Web3D and UX areas.
>>
>> 2. Update the report from the last meeting. No progress on the following
>> items so far
>> - Due to COVID-19 impact. The following item still needs to be done.
>> - Review Publications on Three types of interactions
>> -- Create a folder in one drive – share with you all to collect
>> Literature Review Publications on Three types of interactions – Feng (Done)
>> -- Develop a spreadsheet to categorize the subjects with the papers we
>> review  - Amela.
>> -- List an “other” category for other UX related topics
>> -- Collecting papers and put them into different categories – Amela and
>> Feng
>> -- Collect data/currents publications/papers and authors to invite to the
>> group for later
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> <https://nam04.safelinks.protection.outlook.com/?url=http%3A%2F%2Fweb3d.org%2Fmailman%2Flistinfo%2Fx3d-public_web3d.org&data=02%7C01%7CLIU_F%40mercer.edu%7Ca4b6f1e45aec4ba84ba108d7dd90d329%7C4fb34d2889b247109bcc30824d17fc30%7C0%7C0%7C637221486406740392&sdata=cSJlFwpGQug5cTOAW0cVcJgQ42hYdy7vS6%2FdcIuK6NY%3D&reserved=0>
>>
>>
>>
>> --
>> Nicholas F. Polys, Ph.D.
>>
>> Director of Visual Computing
>> Virginia Tech Research Computing
>>
>> Affiliate Professor
>> Virginia Tech Department of Computer Science
>>
>>
>>
>> --
>> Nicholas F. Polys, Ph.D.
>>
>> Director of Visual Computing
>> Virginia Tech Research Computing
>>
>> Affiliate Professor
>> Virginia Tech Department of Computer Science
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200410/beeb6127/attachment-0001.html>