[x3d-public] X3D4 sound and audio minutes 4 November: conference preparations, specification polishing
Don Brutzman
brutzman at nps.edu
Wed Nov 4 14:45:01 PST 2020
Attendees: Efi, Thanos, Dick, Don
1. Web3D 2020 Paper and Tutorial review.
Paper online with video (brava Efi), we discussed our respective contributions to tutorial.
[1.1] Web3D 2020 paper: Extending X3D Realism with Audio Graphs, Acoustic Properties and 3D Spatial Sound
Papers Session 4: November 12 @ 12:00 AM - 1:30 AM UTC
https://web3d.siggraph.org/event/papers-session-4
https://web3d.siggraph.org/extending-x3d-realism-audio-graphs-acoustic-properties-3d-spatial-sound/
[1.1] Web3D 2020 Tutorial #5, X3D4 Sound and Audio
https://web3d.siggraph.org/event/tutorial-5/
Given that videos for each will be online in advance, for the live sessions we will give short synopses and then offer more time for discussion.
---
2. X3D4 Specification review:
a. Delete? Yes, pretty obvious, change committed.
"TODO continued design, implementation and evaluation work for this component is needed to ensure that full coverage of W3C Audio API capabilities is achieved."
---
b. Defer alphabetizing field names until after Web3D 2020 Conference to avoid potential errors from rushing.
---
c. Agreed to allow have original 16.4.7 Sound with no gain field, and OK to add /children/ field.
We may get feedback from implements that they don't want /children/ field here, that's OK we can remove it if consensus so says.
---
d. Table 16.2 — Sound component support levels
List abstract nodes as "n/a" in level 1.
List all nodes individually in Level 2.
Changes applied and checked into Web3D github.
---
e. CHANGELOG.txt lists changes needed both in Efi's examples and also in their X3DOM source.
She has already accomplished some of the changes in preparation for Web3D 2020 Conference - hooray! No problems yet noted, will advise of any mismatches.
She expects to release updated examples and X3DOM adapted source sometime before the end this month, following Web3D 2020 Conference.
New Year's Resolution: have all examples updated and running in production X3DOM and X_ITE, give another tutorial going over all examples! 8)
---
e. Doppler
Agreement that Doppler effects are directly calculatable by an X3D browser.
A boolean field to allow author expression of whether doppler is appropriate seems necessary to avoid undue computational expense or unwanted doppler effects. Such effects require real-time responsiveness.
* SFBool [in out] dopperEnabled FALSE ??
better naming: there are fields like enableHRTF so better name is
* SFBool [in out] enabledDoppler FALSE
Doppler capabilities provde a physically based way to handle velocity-related sound considerations in a general, robust fashion. Although computational updates must be applied in real time, relative velocity computations only needed to be performed for these two nodes and any sound sources (i.e. a very small set of nodes).
This field has been added to both SpatialSound and ListenerPointSource, also listed as Level 3 in table of support levels for this component.
Doppler can add a lot of realism to audio content in a virtual scene.
---
f. We reviewed Efi's excellent improvements to Figure 16.4, some refinements will be made.
As sound perfectionists (sometimes more politely referred to as purists) we noted the following potential improvements.
- ensure direction vectors are perfectly centered,
- change "source direction" and "listener direction" so that words are on either side of arrow,
- add coneInnerAngle/coneOuterAngle labels to right-hand side diagram,
- change "panner" label above blue figures to "sound producer" or "sound source"
Further under 16.4.18 SpatialSound, delete the undefined term "panner" (without loss of meaning) from following sentence:
"This node provides full spatialization of *panner* capabilities defined by W3C Web Audio API [W3C-WebAudio] within an X3D scene."
If possible, hoping you can provide an updated figure on Friday for inclusion in tutorial materials and public refresh of X3D4 WD2 draft specification.
---
g. Efi/Thanos please check, I don't think we need the following field.
16.4.20 StreamAudioSource
StreamAudioSource : X3DSoundSourceNode {
MFFloat [in,out] mediaStream NULL [−1,1] # Undesirable to retain large stream datasets in memory
If we are getting information from a stream, we don't need to save all that for the X3D author.
Note this is different than the following node, which does have a buffer: 16.4.5 BufferAudioSource
Wondering, don't we want a url option for 16.4.5 BufferAudioSource? Also need to check Web Audio API.
---
h. Recommending additional formats for sound: .mp3 .ogg .flac AAC
Seems like a good idea.
Recommending formats for video: mp4, others? Perhaps.
We should consider matching recommendations for HTML5... summary of support at
* Wikipedia: HTML5 audio
https://en.wikipedia.org/wiki/HTML5_audio
Thanos will think about what formats ought to be added.
Note that we are not talking about an X3D requirement, just augmenting our current recommended support.
* 16.4.2 AudioClip
"The url field specifies the URL from which the sound is loaded. Browsers shall support at least the wavefile format in uncompressed PCM format (see [WAV]). It is recommended that browsers also support the MIDI file type 1 sound format (see 2.[MIDI]) and the MP3 compressed format (see 2.[I11172-1]). MIDI files are presumed to use the General MIDI patch set. 9.2.1 URLs contains details on the url field."
Dick and I will be moving this support paragraph up into Concepts section, similarly for prose found for MovieTexture.
---
i. We reviewed addition of url field to BufferAudioSource as recommended 1 NOV. Sounded good.
Absent objections, I will apply these changes over the next day or two prior to refreshing X3D4 Working Draft 2 online.
===============================================================
6. BufferAudioSource url field needed
Also looking at your BufferAudioSource url field above, it appears as if this is
an omission in our draft. If I'm following logic in W3C Web Audio API, the list
of dependencies is:
- 1.9. The AudioBufferSourceNode Interface
https://www.w3.org/TR/webaudio/#AudioBufferSourceNode
- 1.9.2. Attributes
buffer, of type AudioBuffer, nullable
Represents the audio asset to be played.
- 1.4. The AudioBuffer Interface
https://www.w3.org/TR/webaudio/#AudioBuffer
- copyFromChannel(destination, channelNumber, bufferOffset)
The copyFromChannel() method copies the samples from the specified channel
of the AudioBuffer to the destination array.
Our interface starts with:
16.4.5 BufferAudioSource
BufferAudioSource : X3DSoundSourceNode {
MFFloat [in,out] buffer [] [−1,1]
SFString [in,out] description ""
etc.
Seems reasonable that we would want to be able to load an audio file into an
audio graph, and as you have shown, that this is the place to do it. The only
other node in Sound component which has a url field is the legacy AudioClip node.
Suggest we add X3DUrlObject and first-draft prose:
16.4.5 BufferAudioSource
BufferAudioSource : X3DSoundSourceNode, X3DUrlObject {
MFFloat [in,out] buffer [] [−1,1]
MFString [in,out] url [] [URI]
SFString [in,out] description ""
etc.
"If a /url/ field is provided, then /buffer/ and related fields are initialized
with the contents of the loaded file."
and, copied from AudioClip:
"The url field specifies the URL from which the sound is loaded.
Browsers shall support at least the wavefile format in uncompressed PCM format
(see [WAV]). It is recommended that browsers also support the
MIDI file type 1 sound format (see 2.[MIDI]) and the
MP3 compressed format (see 2.[I11172-1]). MIDI files are presumed to use the
General MIDI patch set. 9.2.1 URLs contains details on the url field."
Pretty straightforward. Author has option to define data /buffer/ directly in
an X3D scene, or else simply load audio file via url (the usual case).
===============================================================
---
3. TODO by spec editors:
a. BufferAudioSource addition of X3DUrlObject and url field, noting in prose how they relate to the existing /buffer/ field.
b. Consider rename X3DTimeDependentNode to X3DTimeDependentObject for consistency. No change in interface hierarchy or node signatures, desirable (and likely necessary) for consistent application in X3DUOM, Java and Python language bindings.
c. Remove StreamAudioSource MFFloat mediaStream array as nonperformant and unnecessary. No need to retain a (perhaps endless) streaming source in memory for X3D scene animation.
I hope to accomplish these Thursday in order to support a full round of updates Friday.
---
Thanks for all review. Have fun with X3D Audio and doppler! 8)
* https://en.wikipedia.org/wiki/Doppler_effect
and something you don't want to hear IRL:
* https://en.wikipedia.org/wiki/File:Speeding-car-horn_doppler_effect_sample.ogg
* https://upload.wikimedia.org/wikipedia/commons/9/90/Speeding-car-horn_doppler_effect_sample.ogg
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list