[x3d-public] X3D Specification Editors: Audio and Sound, 1 July 2020 (corrected)

Don Brutzman brutzman at nps.edu
Wed Jul 1 11:28:36 PDT 2020

[corrected copy follows]

Attendees: Efi Lakka, Thanos Malamos, Dick Puk, Don Brutzman

Confirming: no member-only Web3D Consortium material is included here.


1. *Plans*

We are working towards editing the draft X3D4 component.  Weekly document updates attached.

We plan to submit both paper and tutorial to Web3D 2020 Conference, to be held virtually in November 2020.  Deadlines are now announced:

* Web3D 2020, Important Dates

- Paper and poster submission: July 27, 2020
- Paper/poster acceptance: September 7, 2020
- Tutorial and workshop submission: September 12, 2020
- Industrial use cases submission: September 19, 2020
- Standards session submission: September 19, 2020
- Camera-ready paper/poster: September 21, 2020
- Demonstration submission: September 26, 2020
- Tutorial/workshop/demonstration/industrial use cases acceptance: October 12, 2020

Because of tight schedules and late decision to support a virtual conference, no deadline extensions are expected.

Certainly high-fidelity spatial sound directly supports the conference theme "3D for a Hyperconnected World!"

- Efi will begin refactoring the documentation papers as a conference paper.
- Dick and Don will continue refactoring as specification prose for X3D4.
- All of us continue resolving the short set of remaining design issues.
- After paper submission, we produce tutorial proposal.


2. *Reference material*

> Dear all, 
> please find attached the documents for today's meeting (01/07/2020).
> Best regards,
> Efi Lakka
updates to follow.

3. *Questions and Answers*

a. We are wondering about representation of AudioContext, not seeing that in your document.  Is it an X3D node?

In the initial structure, we have used the AudioContext as new node in X3D (in order to match with Web Audio API structure). At the moment, we decide to extent X3D only with the important and necessary nodes from Web Audio API. Hence, we don’t use it as X3D node.

Given that approach, how do we describe correspondences between X3D4 and Web Audio specifications?
- Table of correspondences (see document),
- Seems like we we need a specification EXAMPLE showing alignment of a thorough example


b. Terminology.
-	In general, should "Audio" refer to aural inputs and "Sound" refer to sonic outputs?
-	If so, does "X3DSoundSourceNode" become more appropriately "X3DAudioSourceNode"?
-	If so, are there any backwards compatibility issues?  (primarily in implementations and concepts, not in any example X3D content)

Audio is anything audible that has been produced, recorded or processed by something electronic or digital.

Sound is anything audible, from any format or instrument(music, bird singing…). Sound can be broken down into Wavelength, Frequency, Amplitude, Pressure, Intensity, Speed of Sound and Direction.

While audio is a sound, not all sounds are audio. Audio means the reproduction of sound.

So...  we will leave this business alone.


c. One small problem with your attachment caused by an error in the X3D Interface Hierarchy: ViewpointGroup is not derived for X3DViewpointNode. Instead, it is derived directly from X3DChildNode. This is because ViewpointGroup is a catalog of viewpoints rather than a viewpoint itself. We have created a Mantis issue to fix the spec.

* Mantis 1317: ViewpointGroup listed in incorrect location within X3D Interface hierarchy



4. *Discussion*

a. Abstract node types.

We should be careful about abstract node types and not add any that don't offer common functionality.  Their purpose is to
- define common fields,
- simplify validation of parent-child node relationships,
- simplify object-oriented software implementations, and
- (optionally) facilitate future extensions for adding potential new nodes.

Let's try to avoid adding any excess abstract node types, if possible.

Note that SoundProcessingGroup is _not_ a node type. Rather it is a way to establish parent-child node relationships and potentially facilitating creation of audio graphs, which may have multiple levels in a tree. (Similarly for SoundAnalysisGroup and SoundChannelGroup, can we avoid them?)

Discussion: appropriate rename X3DSoundProcessingNode, X3DSoundAnalysisNode, X3DSoundChannelNode. (Updated document attached)

For example, perhaps these names actually suggest field names for X3DSoundProcessingNode concrete nodes, such as
a. [in out] MFNode effects   [BiquadFilter,Convolver,Delay,DynamicsCompressor,Gain,WaveShaper,PeriodicWave]
b. [in out] MFNode analysers [Analyser]
c. [in out] MFNode channels  [ChannelSplitter,ChannelMerger]  # is there a Channel?

Since there are many "effects" a plausible combination abstraction is

a'.[in out] MFNode effects   [X3DSoundProcessingNode]

... but perhaps avoid X3DSoundAnalysisNode, X3DSoundChannelNode since they are so specialized.  Looking at details of node definitions and example graphs will give us much better confidence whether we need an abstract node type for future extensions.

Whenever we can define SFNode/MFNode child fields with well-defined relationships, that eliminates need for any grouping.  This is terser and easier for authors (programmers) to model as well.

==== last week, to be rechecked =====

b. Remove "SoundProcessingGroup", this will likely be an field in each parent node rather than an explicit node.  Goal to achieve: if we have that field within the processing nodes themselves, then we do not need to group them or use ROUTE connections.  We want to construct an audio graph with equal expressive power as Web Audio API (and other sound approaches).

* [in out] MFNode soundProcessing [X3DSoundProcessingNode]

c. Gain might also be a field, rather than a node.  Units in decibels seems preferred, though amplification is possible too.

* [in out] SFFloat gain 0.0

Web Audio API uses amplification factors, but sound equipment uses decibels.  Possible UNIT addition for sound?  Must be careful because amplification factors are linear and decibels are logarithmic. Perhaps

* [in out] SFString gainFactorType "AMPLIFICATION" ["AMPLIFICATION" "DECIBEL"]

[1] Wikipedia: Decibel

Of note is that decibel amplification is a logarithmic power ratio to a reference level.  This is typical of audio hardware, and usually (but not always) consistent. So what would our reference level be?

i.   avoid decibels
ii.  use reference in the enumeration value, e.g. "DECIBEL-XYZ" which seems extremely confusing and error prone.
iii. just pick most common reference for "DECIBEL" e.g. + (or -) 3dB equals amplification factor of sqrt(2) (or 0.5) - unambiguous.

There is some disagreement on this topic.  For example, allowing sound engineers or mixing decks to be modeled in these audio graphs is an important use case.  Decibels to appear in some of the other nodes... Worth a close look.

d. (confirmed) We agreed to stay consistent with Analyser, not Analyzer.

e. Generality - we continue to consider whether this is mappable to other engines, e.g. MPEG4 or Dolby or whatever.  Also whether it is mappable to audio engineers putting together audio equipment, mixer board, room (or auditorium or arena) setups, etc.

- (confirmed) identify field signature questions that need resolution, for example MicrophoneSource.  Suggestions welcome.
- (TODO) SpatialSound url field to load a Javascript Web Audio API (perhaps with optional support).  Similar to X3D Script node, either external or embedded source with ecmascript: prefix.
- (not needed) Hoping to explain how we apply the terms "audio" and "sounds" so that the distinction is sensible to readers.  Already pretty clear: "acoustics" for reflection absorption etc.

Our goals for next week are to follow this path forward:

- look at multiple examples of audio graph(s) and define parent-child field relationships,
- (hopefully) eliminate need for any grouping,
- minimize (but allow if appropriate) abstract types, e.g. X3DSoundProcessingNode
- update X3D4 specification draft to include (currently) well-defined nodes,
- check and recheck examples to make sure that they match specification hierarchy and prose.
- recheck, is AudioContext needed?
- We expect many more questions can then be evaluated (for example, representing hardware input-output hookups corresponding to software-component audio graphs).

Next week: look for "80% solution" audio graph that let's us check most relationships in the Sound component.

4. Step by step: continuing convergence.  Time to press ahead:  "better is the enemy of good enough" !

Next week's meeting: Thursday, same time.

Have fun with X3D Audio and Sound!  8)

all the best, Don
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
all the best, Don
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Report01_07_2020updated.pdf
Type: application/pdf
Size: 200824 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200701/1e832da9/attachment-0002.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: AnalysisNodes01_07_2020.pdf
Type: application/pdf
Size: 388112 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200701/1e832da9/attachment-0003.pdf>

More information about the x3d-public mailing list