[x3d-public] X3D Specification Editors: Audio and Sound, 9 July 2020

Don Brutzman brutzman at nps.edu
Thu Jul 9 11:10:05 PDT 2020


Minutes for 2-hour meeting today.  Attendees Efi Lakka, Thanos Malamos, Dick Puk, Don Brutzman.

No member-only information is included in these minutes.

Warmup thoughts, design question: is there a way to slave ListenerPoint (a) to current view, or (b) each provided Viewpoint?  Also, how do you return to current view if a separate ListenterPoint was bound?

Details follow.  No new attachments for this week's meeting, see last week's notes.

On 7/8/2020 7:31 PM, Don Brutzman wrote:
> 1. continuing...
> 
> On 7/8/2020 10:14 AM, Richard F. Puk wrote:
>> [...]
>> I have the following comments on the minutes (Shown in red):
>>
>> In my opinion, the new X3DSoundSourceNodes should not have the work “Source” in the names of the concrete nodes as the node names are clear without it, it is redundant, and also is different from the existing nodes.
> 
> As discussed, that is one possible solution, but other considerations also pertain.
> 
> Relevant node names of interest are:
> 
> +- X3DTimeDependentNode -+- TimeSensor
> | |
> | +- X3DSoundSourceNode -+- AudioClip         (~Web Audio API: MediaElementAudioSourceNode)
> |                        +- MovieTexture
> |                        +- AudioBufferSource (~Web Audio API: AudioBuffer + AudioBufferSourceNode)
> |                        +- OscillatorSource  (~Web Audio API: OscillatorNode)
> |                        +- StreamAudioSource (~Web Audio API: MediaStreamAudioSourceNode)
> |                        +- MicrophoneSource
> |
> | +- X3DSoundDestinationNode -+- AudioDestination (~Web Audio API: AudioDestinationNode)

As elaboration: specifically AudioDestination is a hardware device ID, maintained by operating system (OS).

FWIW, without loss of functionality, a synonym for AudioDestination might be HardwareAudioDestination (in contrast to StreamAudioDestination).

> |                             +- StreamAudioDestination (~Web Audio API:MediaStreamAudioDestinationNode)

nowadays typically WebRTC, which is stable and approved (and usable!!) specification.  8)

> + +- X3DSoundProcessingNode -+- BiquadFilter
> |                            +- Convolver
> |                            +- Delay
> |                            +- DynamicsCompressor
> |                            +- Gain
> |                            +- WaveShaper
> |                            +- PeriodicWave
> |
> | +- X3DSoundAnalysisNode -+- Analyser
> |
> | +- X3DSoundChannelNode -+- ChannelSplitter
> |                         +- ChannelMerger
> 
> AudioClip and MovieTexture are well understood legacy nodes and their names will not change.
> 
> Want to voice caution here.  Conceptually, the base names "AudioBuffer", "Oscillator" and "StreamAudio" by themselves might be referring to sources of audio (i.e. outputs from streams) or sinks for audio (i.e. inputs to streams).  When choosing node names (i.e. the words in the X3D language) we strive for clarity and want to avoid ambiguity.  So it may make sense to emphasize purpose by keeping the suffix "Source" for these nodes.
> 
> Am confident that when we start putting Web Audio graphs together using this new set of nodes, implementation/evaluation results will either make good sense (like the Web Audio javascript) or else confusing gaps will become more evident.  Example usage is important to consider.
> 
> We'll discuss and reach initial consensus on good names during Thursday morning's teleconference, 0900-1030 pacific)

We discussed this topic in exhilarating detail.

Naming heuristic: "when the going gets tough, the tough get... verbose."

Suggestion: for the near term we go for longer names for clarity, but once we have examples in hand, we reconsider whether shorter names can work.

> 2. Dick and I made excellent progress updating the Concepts 4.4.2.3 Interface Hierarchy to match the latest meeting notes.
> 
> We are ready to move acoustic fields from Material to AcousticProperties node in Shape component when Michalis finishes his merge of glTF PBR Pull Request 8.

confirmed

> Several editorial issues are pending, including:
> 
>> The concrete node derivations need to replace “AudioNode” with the appropriate abstract node type.
>>
>> nDick
> 
> Our agenda for Thursday is to finalize node list and interfaces, then start dropping in prose from Efi's detailed documentation report.
a. Agreed on abstract types:

     16.3 Abstract types
         16.3.1 X3DAudioListenerNode
         16.3.2 X3DSoundAnalysisNode
         16.3.3 X3DSoundChannelNode
         16.3.4 X3DSoundDestinationNode
         16.3.5 X3DSoundProcessingNode
         16.3.6 X3DSoundNode
         16.3.7 X3DSoundSourceNode

Triage follows for nodes, in order to edit document further for draft release:

     16.4 Node reference

b. *Accepted*

         16.4.1 AudioClip
         16.4.2 Sound
         16.4.3 SpatialSound
         16.4.4 ListenerPoint
         16.4.4 AudioBufferSource
         16.4.3 AudioBufferSource
         16.4.2 OscillatorSource
         16.4.7 BiquadFilter
         16.4.8 Convolver
         16.4.9 Delay
         16.4.10 DynamicsCompressor
         16.4.12 WaveShaper
         16.4.13 PeriodicWave
         16.4.17 Analyser
         16.4.** StreamAudioSource

         16.4.14 AudioDestination
         16.4.xx StreamAudioDestination
	
         16.4.yy MicrophoneSource (physical hardware device)

c. *Not Included*

         16.4.5 MediaElementAudioSource (same as AudioClip)
         16.4.6 MediaStreamAudioSource  (same as StreamAudioSource)

         16.4.15 MediaStreamAudioDestination (same as StreamAudioDestination)
         16.4.16 MediaStreamTrack
         16.4.1  AudioParam (merged functionality? not used)
         16.4.21 Panner     (merged functionality in SpatialSound)

d. *Pending major issues, decision TBD*

         16.4.5 BinauralListenerPoint (will attempt to merge with ListenerPoint)
               (SurroundSound might also be a variation on ListenerPoint)

         16.4.18 ChannelSplitter (likely, need channel design discussion, next session)
         16.4.19 ChannelMerger   (likely, need channel design discussion, next session)

         16.4.1 AudioContext (perhaps integrated within interface hierarchy, more discussion)

         16.4.11 Gain (included as a field in many nodes, or within interface hierarchy)

         Virtual Microphone Sensor (perhaps same as ListenerPoint or media stream?)

	How to simply pan or rebalance left/right?

	SFNode/MFNode fields defining parent-child node relationships, allowing tree-like construction of a W3C Audio API graph using X3D nodes.

==================

3. Next steps:

a. Request: Efi please confirm all field definitions are correct for the Accepted nodes - so far so great.

b. TODO Don and Dick will update draft specification in github, specifically nodes fields and prose, using Efi's two provided analysis/report documents.

c. Objective: shift sharpest focus from our draft notes to actual github spec as our outcome document.

d. What are our example audio graphs to represent in X3D?Looking at examples will help us confirm that the right fields and parent-child node relationships are described well.  Efi has a detailed example comparing earlier/evolved examples that will help.  She can (and will soon) release this.  As they are each reviewed and considered mature, we will place them online in version control as part of X3D Examples Archive.

e. It is clear that a good amount of work remains, but at least 80% is well defined and sensible.  We need to ship a draft X3D4 Sound Component that reflects this progress, noting more work to follow.  This will allow review and assessment (and possibly engagement) by many others.

f. It is not completely certain yet, but we are very close hope to resolve this sufficiently to support a Web3D 2020 Conference paper.  This is important and primarily a restructuring of Efi's excellent documents updates in paper form. We can certainly support a Web3D 2020 Conference tutorial.

* Statistics joke: "The first 90% of the work takes 90% of the time.  The last 10% of the work takes the other 90% of the time."  (perhaps similar to Zeno's Paradox?!)

* Zeno's Paradoxes
   https://en.wikipedia.org/wiki/Zeno%27s_paradoxes

Summary of our near-term objective: produce a sufficiently advanced X3D4 Sound Component that enables meaningful understanding, implementation, evaluation and finalization.

> We are getting pretty close to releasing the Sound component for inclusion in draft X3D4 Specification.
> 
> Due diligence continues.  All feedback remains helpful and welcome.

Next meeting of group: back to Wednesday 15 July, same time.

"What say?"  Have fun with X3D Audio and Sound!  8)

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman



More information about the x3d-public mailing list