[x3d-public] X3D4 sound and audio minutes 4 November: conference preparations, specification polishing
Don Brutzman
brutzman at nps.edu
Fri Nov 6 10:59:11 PST 2020
Spec editor review, Dick and Don.
Mostly agreed and confirmed changes below were applied.
Additional changes:
1. If we can get a better name for ListenerPointSource, that is worth considering. Noted in node description that it is a virtual source.
2. We will move media formats list from AudioClip/MovieTexture to new section
16.2.6 Audio encoding formats
We should probably create a section for Video encoding formats. Maybe even in Clause 4... I think a lot of prose like that appears under "url" descriptions... we will think about this further.
3. Add url to BufferAudioSource, also refers to 16.2.6.
4. Rename X3DTimeDependentNode to X3DTimeDependentObject ? Yes this is OK.
Deleted duplicative entry of X3DTimeDependentNode from table.
5. Accept the recommended formats/containers, work on phrasing, goes in 16.2.6.
Don will apply these changes, finalize X3DUOM and implementation tools, test, deploy.
On 11/4/2020 2:45 PM, Don Brutzman wrote:
> Attendees: Efi, Thanos, Dick, Don
>
> 1. Web3D 2020 Paper and Tutorial review.
>
> Paper online with video (brava Efi), we discussed our respective contributions to tutorial.
>
> [1.1] Web3D 2020 paper: Extending X3D Realism with Audio Graphs, Acoustic Properties and 3D Spatial Sound
> Papers Session 4: November 12 @ 12:00 AM - 1:30 AM UTC
> https://web3d.siggraph.org/event/papers-session-4
> https://web3d.siggraph.org/extending-x3d-realism-audio-graphs-acoustic-properties-3d-spatial-sound/
>
> [1.1] Web3D 2020 Tutorial #5, X3D4 Sound and Audio
> https://web3d.siggraph.org/event/tutorial-5/
>
> Given that videos for each will be online in advance, for the live sessions we will give short synopses and then offer more time for discussion.
>
> ---
>
> 2. X3D4 Specification review:
>
> a. Delete? Yes, pretty obvious, change committed.
>
> "TODO continued design, implementation and evaluation work for this component is needed to ensure that full coverage of W3C Audio API capabilities is achieved."
>
> ---
>
> b. Defer alphabetizing field names until after Web3D 2020 Conference to avoid potential errors from rushing.
>
> ---
>
> c. Agreed to allow have original 16.4.7 Sound with no gain field, and OK to add /children/ field.
>
> We may get feedback from implements that they don't want /children/ field here, that's OK we can remove it if consensus so says.
>
> ---
>
> d. Table 16.2 — Sound component support levels
>
> List abstract nodes as "n/a" in level 1.
>
> List all nodes individually in Level 2.
>
> Changes applied and checked into Web3D github.
>
> ---
>
> e. CHANGELOG.txt lists changes needed both in Efi's examples and also in their X3DOM source.
>
> She has already accomplished some of the changes in preparation for Web3D 2020 Conference - hooray! No problems yet noted, will advise of any mismatches.
>
> She expects to release updated examples and X3DOM adapted source sometime before the end this month, following Web3D 2020 Conference.
>
> New Year's Resolution: have all examples updated and running in production X3DOM and X_ITE, give another tutorial going over all examples! 8)
>
> ---
>
> e. Doppler
>
> Agreement that Doppler effects are directly calculatable by an X3D browser.
>
> A boolean field to allow author expression of whether doppler is appropriate seems necessary to avoid undue computational expense or unwanted doppler effects. Such effects require real-time responsiveness.
>
> * SFBool [in out] dopperEnabled FALSE ??
>
> better naming: there are fields like enableHRTF so better name is
>
> * SFBool [in out] enabledDoppler FALSE
>
> Doppler capabilities provde a physically based way to handle velocity-related sound considerations in a general, robust fashion. Although computational updates must be applied in real time, relative velocity computations only needed to be performed for these two nodes and any sound sources (i.e. a very small set of nodes).
>
> This field has been added to both SpatialSound and ListenerPointSource, also listed as Level 3 in table of support levels for this component.
>
> Doppler can add a lot of realism to audio content in a virtual scene.
>
> ---
>
> f. We reviewed Efi's excellent improvements to Figure 16.4, some refinements will be made.
>
> As sound perfectionists (sometimes more politely referred to as purists) we noted the following potential improvements.
>
> - ensure direction vectors are perfectly centered,
> - change "source direction" and "listener direction" so that words are on either side of arrow,
> - add coneInnerAngle/coneOuterAngle labels to right-hand side diagram,
> - change "panner" label above blue figures to "sound producer" or "sound source"
>
> Further under 16.4.18 SpatialSound, delete the undefined term "panner" (without loss of meaning) from following sentence:
>
> "This node provides full spatialization of *panner* capabilities defined by W3C Web Audio API [W3C-WebAudio] within an X3D scene."
>
> If possible, hoping you can provide an updated figure on Friday for inclusion in tutorial materials and public refresh of X3D4 WD2 draft specification.
>
> ---
>
> g. Efi/Thanos please check, I don't think we need the following field.
>
> 16.4.20 StreamAudioSource
> StreamAudioSource : X3DSoundSourceNode {
> MFFloat [in,out] mediaStream NULL [−1,1] # Undesirable to retain large stream datasets in memory
>
> If we are getting information from a stream, we don't need to save all that for the X3D author.
>
> Note this is different than the following node, which does have a buffer: 16.4.5 BufferAudioSource
>
> Wondering, don't we want a url option for 16.4.5 BufferAudioSource? Also need to check Web Audio API.
>
> ---
>
> h. Recommending additional formats for sound: .mp3 .ogg .flac AAC
>
> Seems like a good idea.
>
> Recommending formats for video: mp4, others? Perhaps.
>
> We should consider matching recommendations for HTML5... summary of support at
>
> * Wikipedia: HTML5 audio
> https://en.wikipedia.org/wiki/HTML5_audio
>
> Thanos will think about what formats ought to be added.
>
> Note that we are not talking about an X3D requirement, just augmenting our current recommended support.
>
> * 16.4.2 AudioClip
>
> "The url field specifies the URL from which the sound is loaded. Browsers shall support at least the wavefile format in uncompressed PCM format (see [WAV]). It is recommended that browsers also support the MIDI file type 1 sound format (see 2.[MIDI]) and the MP3 compressed format (see 2.[I11172-1]). MIDI files are presumed to use the General MIDI patch set. 9.2.1 URLs contains details on the url field."
>
> Dick and I will be moving this support paragraph up into Concepts section, similarly for prose found for MovieTexture.
>
> ---
>
> i. We reviewed addition of url field to BufferAudioSource as recommended 1 NOV. Sounded good.
>
> Absent objections, I will apply these changes over the next day or two prior to refreshing X3D4 Working Draft 2 online.
>
> ===============================================================
> 6. BufferAudioSource url field needed
>
> Also looking at your BufferAudioSource url field above, it appears as if this is
> an omission in our draft. If I'm following logic in W3C Web Audio API, the list
> of dependencies is:
>
> - 1.9. The AudioBufferSourceNode Interface
> https://www.w3.org/TR/webaudio/#AudioBufferSourceNode
>
> - 1.9.2. Attributes
> buffer, of type AudioBuffer, nullable
> Represents the audio asset to be played.
>
> - 1.4. The AudioBuffer Interface
> https://www.w3.org/TR/webaudio/#AudioBuffer
>
> - copyFromChannel(destination, channelNumber, bufferOffset)
> The copyFromChannel() method copies the samples from the specified channel
> of the AudioBuffer to the destination array.
>
> Our interface starts with:
>
> 16.4.5 BufferAudioSource
> BufferAudioSource : X3DSoundSourceNode {
> MFFloat [in,out] buffer [] [−1,1]
> SFString [in,out] description ""
> etc.
>
> Seems reasonable that we would want to be able to load an audio file into an
> audio graph, and as you have shown, that this is the place to do it. The only
> other node in Sound component which has a url field is the legacy AudioClip node.
>
> Suggest we add X3DUrlObject and first-draft prose:
>
> 16.4.5 BufferAudioSource
> BufferAudioSource : X3DSoundSourceNode, X3DUrlObject {
> MFFloat [in,out] buffer [] [−1,1]
> MFString [in,out] url [] [URI]
> SFString [in,out] description ""
> etc.
>
> "If a /url/ field is provided, then /buffer/ and related fields are initialized
> with the contents of the loaded file."
>
> and, copied from AudioClip:
>
> "The url field specifies the URL from which the sound is loaded.
> Browsers shall support at least the wavefile format in uncompressed PCM format
> (see [WAV]). It is recommended that browsers also support the
> MIDI file type 1 sound format (see 2.[MIDI]) and the
> MP3 compressed format (see 2.[I11172-1]). MIDI files are presumed to use the
> General MIDI patch set. 9.2.1 URLs contains details on the url field."
>
> Pretty straightforward. Author has option to define data /buffer/ directly in
> an X3D scene, or else simply load audio file via url (the usual case).
> ===============================================================
>
> ---
>
> 3. TODO by spec editors:
>
> a. BufferAudioSource addition of X3DUrlObject and url field, noting in prose how they relate to the existing /buffer/ field.
>
> b. Consider rename X3DTimeDependentNode to X3DTimeDependentObject for consistency. No change in interface hierarchy or node signatures, desirable (and likely necessary) for consistent application in X3DUOM, Java and Python language bindings.
>
> c. Remove StreamAudioSource MFFloat mediaStream array as nonperformant and unnecessary. No need to retain a (perhaps endless) streaming source in memory for X3D scene animation.
>
> I hope to accomplish these Thursday in order to support a full round of updates Friday.
>
> ---
>
> Thanks for all review. Have fun with X3D Audio and doppler! 8)
>
> * https://en.wikipedia.org/wiki/Doppler_effect
>
> and something you don't want to hear IRL:
>
> * https://en.wikipedia.org/wiki/File:Speeding-car-horn_doppler_effect_sample.ogg
>
> * https://upload.wikimedia.org/wikipedia/commons/9/90/Speeding-car-horn_doppler_effect_sample.ogg
>
> all the best, Don
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list