[x3d-public] Sound v4 > ListenerPointSource QUESTIONS

Wed Mar 1 22:31:11 PST 2023

Thanks for asking Doug.  References:

*	X3D4 Architecture, Sound Component, 16.4.13 ListenerPointSource
*	https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-DIS/Part01/components/sound.html#ListenerPointSource
*	X3D Tooltips, ListenerPointSource
*	https://www.web3d.org/x3d/content/X3dTooltips.html#ListenerPointSource

“ListenerPointSource represents the position and orientation of a person listening to virtual sound in the audio scene, and provides single or multiple sound channels as output. Multiple ListenerPointSource nodes can be active for sound processing.”

Suggested analogy: a virtual microphone in the virtual environment.  The essential idea is that sound can be recorded as if a listener were at that point.  Thus position and orientation can be placed at a given location (such as a seat in a virtual auditorium) or even be animated to rehearse an audio soundtrack through a virtual environment.

Further option: can be user driven at run time.

*	trackCurrentView field is TRUE then position and orientation matches the user's current view.

The ListenerPointSource can be an input to other parts of an audio graph, and feeding outputs to external speakers, or whatever.  Note similarity to other sources such as MicrophoneSource and StreamAudioSource - they provide outputs for processing by other audio nodes, and don’t have inputs.

Our design intent was to take full advantage of a sound spatialization in a virtual environment so that any point (static or moving or navigating) might be a potential sample point.  Seems pretty powerful, especially as our aural designs become further sophisticated to match or augment our geometric designs.

Similarly, just as *Source nodes provide audio signals, the *Destination consume audio signals.  In other words (electrical engineering parlance) they are signal sources and signal sinks.  All variation occurs between those.

We followed the example (and insightful design) of Web Audio API Recommendation and stayed relatively silent about monaural, binaural, etc.  If a signal source has one or two channels, for example, the audio graph mostly looks the same.  ChannelMerger ChannelSelector and ChannelSplitter lets an author work on individual channels if they want.

No doubt you noticed in the Support Levels we did not place any bounds on the number of nodes supported at one time, expecting that implementations and hardware support will vary quite a bit and further improve over time.

Hopefully the spec prose is a bit clearer now – we will all learn a lot as we keep building more sophisticated audio graphs and sound environments.

All the best, Don

From: x3d-public <x3d-public-bounces at web3d.org> On Behalf Of GPU Group
Sent: Wednesday, March 1, 2023 11:47 AM
To: X3D Graphics public mailing list <x3d-public at web3d.org>
Subject: [x3d-public] Sound v4 > ListenerPointSource QUESTIONS

I'm struggling to understand how the LPS ListenerPointSource should work. It seems like a great idea with plenty of possibilities.

Guess 1:

LP ListenerPoint - would have just the pose of the listener.

- that pose would be used in any Panner nodes instead of viewpoint

- LP would be a singleton and apply to all contexts

- would work under AD AudioDestination

Guess 2:

Multiple LP - like LP, except each LP could refer to / apply its pose to a specific audio context using keys / IDs.

- each AD would have a key/ID (SFString or SFInt32), (ID would be simple number like 0 or 1 or 2, not UUID of mediaStream)

- each LP would have a keys/IDs field, (MFString, MFInt32), and could apply its pose to panner nodes under the ADs with a key in its key list, or if key list empty, to all contexts.

Guess 3:

LPS works with SAD StreamAudioSource, with each SAD converting its output to a bufferable stream like HTML MediaStream. The program would hold a list of Streams with key/ID: list (Stream,key)

Each LPS would have a keys/IDs field, and merge all the streams its interested in.

The LPS pose would not affect Panner nodes under SADs.

Rather it would take each stream as positionless and add its own pose with an internal panner node.

And have a built in AD to deliver its output to the scene 

(when I implemented Sound and SpatialSound, I included an AD node if and only if it had no audio parent)

Guess 4:

Like 3, except LPS pose affects Panner nodes under its SAD list. 

x But multiple LPS would have conflicting pose claims, the Panner node under SAD wouldn't know which one to use)

I've lost the link to a paper that explained it. 

Q. how is it supposed to work?

Thanks, Doug Sanden

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230302/b1a77874/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5464 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230302/b1a77874/attachment-0001.p7s>