[x3d-public] Audio and Sound meeting 10 August 2020: conference tutorial, W3C Audio working group, defining/connecting audio graphs

[Don Brutzman]

Attendees Efi Lakka, Thanos, Dick, Don

---

1. Web3D 2020 tutorial.

We discussed and agreed to propose/prepare/present a tutorial at Web3D 2020.  This will be a great complement to our submitted draft paper.

Tutorial proposal 12 SEP, preparations online beforehand, performed 9-13 November.

[1] https://2020.web3dconference.org/tutorials/

Tutorial Outline.

a. Thanos. Fundamentals of Sound and Acoustics.
b. Don.  X3D3 Sound
c. Efi.  W3C Web Audio API

90 minutes, 30-minute break.

d. Efi, all.  X3D4 Sound model and examples.
e. Participants. Questions, discussion, future work.

60 minutes, done!  2.5 hours net, 3 hours total.

Videos, presentations, examples posted on Web3D 2020 website. Questions/responses can continue online.

Here is a slideset with examples that includes coverage of Sound in X3D3

[2] X3D for Web Authors
     http://x3dgraphics.com/slidesets/X3dForWebAuthors/Chapter12EnvironmentSensorSound.pdf

[3] X3D Example Archives: X3D for Web Authors, Chapter 12 Environment Sensor Sound
     http://x3dgraphics.com/examples/X3dForWebAuthors/Chapter12EnvironmentSensorSound/

---

2. W3C engagement.

a. Immersive Web Working Group is working on 6 (!) drafts related to WebXR.

[4] W3C Immersive Web Working Group
     https://www.w3.org/immersive-web/

Curiously none seem to have any mention of audio, though a relationship is mentioned in the charter.  Perhaps there are examples.

Thanos comments: "I think it is worth a second look in webxr and see if there is a possibility to cooperate with and use X3D for geometry and functionality. Also we may start a channel of communication with audioapi WG Although we are focusing on an application development environment and audioapi WG is mainly working for the audio in browsers perhaps the work we have done is interesting for them too."

b. The W3C Audio working group is active with a long and excellent history.

[5] W3C Audio Working Group repository
     https://github.com/WebAudio

[6] W3C Audio Working Group home page
     https://www.w3.org/groups/wg/audio
     "The mission of the Audio Working Group is to add advanced sound and music synthesis capabilities to the Open Web Platform."

[7] W3C Audio Working Group Charter
     https://www.w3.org/2011/audio/charter/audio-2019.html

We reviewed the charter.  Thanos, Efi and Don are requesting to join the group.  We will see what their latest work is and consider how to best present our X3D4-related work during the coming months.

---

3. Next steps for specification.

[8] X3D4 Sound Component
     https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-WD2/Part01/Architecture.html

[9] Web Audio API
     https://www.w3.org/TR/webaudio

Am hoping that we can look at examples and then consider best way to connect an audio graph together... hopefully with parent-child SFNode/MFNode field relationships.  As an alternative we can also consider ROUTE connections.

Specifically
- Web Audio API lists many objects that can connect to each other as an "audio graph" and is master reference for functionality.
- We have ~16 new X3D nodes that directly correspond to each of those Web Audio API objects.
- We want to define correspondences that duplicate the connections between nodes as that same Web Audio API audio graph.

Efi showed general pattern:

* Input > Effects > Destination

She notes that we might add following field to AudioClip, AudioBufferSource, OscillatorSource, StreamAudioSource, MicrophoneSource.

* MFNode [in out[ audioGraph [] [X3DChildNode]

Looking at paired examples, listed in W3C Audio API Javascript and X3D XML, will help us match the graph connections equivalently (without getting confused by terminology overlaps).

Referring to Efi's table of corresponding names will help us be precise.  Looking at her each of arrows in her diagram, with each flowing from a source on left to a destination on right, will likely correspond to (X3D node child -> field relationship -> X3D node parent).

Comments and action items:
a. We added "gain" as an SFFloat field throughout other nodes in the component, rather than having a Gain node.
b. Dick notes that term "audio graph" is referred to many times, but is not crisply defined in [9].  Specifically, not included in "Terms defined by this specification" section, online at
    https://www.w3.org/TR/webaudio/#index-defined-here
c. We will provide that feedback to W3C Audio Working Group.
d. We should similarly define "audio graph" in X3D specification.

We discussed how making paired X3D diagrams by copying/adapting Efi's existing API diagrams.  She will post latest version and continue work.

---

We meet again next week on Wednesday, same time.  Having fun with X3D4 audio!

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman