[x3d-public] Sound v4 > Convolver, BufferAudioNode RECOMMENDATIONS

[GPU Group]

RECOMMENDATIONS

1. Convolver: change MFFloat buffer to SFNode buffer [AudioBuffer]

2. Add AudioBuffer node type

3. Refactor BufferAudioSource to delegate loading to AudioBuffer

4. browser developers: beware implementing BufferAudioSource via web audio
AudioBufferSource

 -Doug Sanden


Explanation:

Convolver is incomplete because it doesn't have a field to specifiy the
number of channels in the buffer. Without that, there's no way to know how
many channel chunks to split the MFFloat PCM32 data into. Yet the
documentation claims it can have any number of channels.

3 options to fix:

a) in text, say single channel only

b) add a bufferChannels field

c) delegate loading to AudIoBuffer node

I recommend c) to gain the additional benefits of URL loading. Most web
audio Convolver examples load impulse response data from .wav files.

https://middleearmedia.com/web-audio-api-convolver-node/

In web audio, both Convolver and AudioBufferSource have a .buffer field
that takes an AudioBuffer node.

https://developer.mozilla.org/en-US/docs/Web/API/ConvolverNode

- example code shows loading AudioBuffer from .wav

https://developer.mozilla.org/en-US/docs/Web/API/AudioBufferSourceNode

- example code shows loading PCM data in AudioBuffer algorithmically at run
time

https://developer.mozilla.org/en-US/docs/Web/API/AudioBuffer

- explains 2 ways to load data, from .wav or from raw data (PCM)



Having both BufferAudioSource and Convolver delegate loading to AudioBuffer
will reduce code needed, match web audio, while fixing Convolver
shortcoming and adding url loading to Convolver through AudioBuffer

i have the recommendations working in freewrl with these scene files:

https://freewrl.sourceforge.io/tests/16_Sound/convolver_control.wrl

https://freewrl.sourceforge.io/tests/16_Sound/convolver_control_PCM.wrl

https://freewrl.sourceforge.io/tests/16_Sound/BufferAudioSource.x3d



Implementing BufferAudioSource

https://developer.mozilla.org/en-US/docs/Web/API/AudioBufferSourceNode

"It will automatically be garbage-collected" sometime after it quits
playing.

The unusual nature of automatic garbage collection in procedural code
doesn't apply well to declarative web3d syntax where nodes perisist for the
life of the scene run. Labsound native library for web audio has no
AudioBufferSourceNode - the one node its missing - due to the garbage
collection behavior, and they've had no requests for it. So I can't test
implementing BufferAudioSource via web audio AudioBufferSource and can't
recommend either way.

In native code via Labsound I'm able to implement BufferAudioSource the
same way as AudioClip --Labsound SampledAudioSource which has detune and
playbackRate, stop,start,loop-- and have that working in FreeWRL.


ORIGINAL BUFFERAUDIOSOURCE

BufferAudioSource : X3DSoundSourceNode, X3DUrlObject {

  SFTime   [in,out] autoRefresh           0.0        [0,∞)

  SFTime   [in,out] autoRefreshTimeLimit  3600.0     [0,∞)

  MFFloat  [in,out] buffer                []         [−1,1]

  SFTime   [in,out] bufferDuration        0          [0,∞)

  SFString [in,out] channelCountMode      "max"      ["max", "clamped-max",
"explicit"]

  SFString [in,out] channelInterpretation "speakers" ["speakers",
"discrete"]

  SFString [in,out] description           ""

  SFFloat  [in,out] detune                0          [0,∞)

  SFBool   [in,out] enabled               TRUE

  SFFloat  [in,out] gain                  1          (-∞,∞)

  SFBool   [in,out] load                  TRUE

  SFBool   [in,out] loop                  FALSE

  SFTime   [in,out] loopEnd               0          [0,∞)

  SFTime   [in,out] loopStart             0          [0,∞)

  SFNode   [in,out] metadata              NULL       [X3DMetadataObject]

  SFInt32  [in,out] numberOfChannels      0          [0,∞)

  SFTime   [in,out] pauseTime             0          (-∞,∞)

  SFFloat  [in,out] playbackRate          1          (-∞,∞)

  SFTime   [in,out] resumeTime            0          (-∞,∞)

  SFFloat  [in,out] sampleRate            0          [0,∞)

  SFTime   [in,out] startTime             0          (-∞,∞)

  SFTime   [in,out] stopTime              0          (-∞,∞)

  MFString [in,out] url                   []         [URI]

  SFInt32  [out]    bufferLength          0          [0,∞)

  SFInt32  [out]    channelCount                     [0,∞)

  SFTime   [out]    elapsedTime

  SFBool   [out]    isActive

  SFBool   [out]    isPaused

}

/ORIGINAL BUFFERAUDIOSOURCE

REFACTORED BUFFERAUDIOSOURCE

AudioBufferSource : X3DSoundSourceNode {

  SFNode   [in,out] buffer                NULL       [AudioBuffer]

  SFString [in,out] channelCountMode      "max"      ["max", "clamped-max",
"explicit"]

  SFString [in,out] channelInterpretation "speakers" ["speakers",
"discrete"]

  SFString [in,out] description           ""

  SFFloat  [in,out] detune                0          [0,∞)

  SFBool   [in,out] enabled               TRUE

  SFBool   [in,out] loop                  FALSE

  SFTime   [in,out] loopEnd               0          [0,∞)

  SFTime   [in,out] loopStart             0          [0,∞)

  SFNode   [in,out] metadata              NULL       [X3DMetadataObject]

  SFTime   [in,out] pauseTime             0          (-∞,∞)

  SFFloat  [in,out] playbackRate          1          (-∞,∞)

  SFTime   [in,out] resumeTime            0          (-∞,∞)

  SFTime   [in,out] startTime             0          (-∞,∞)

  SFTime   [in,out] stopTime              0          (-∞,∞)

  SFTime   [out]    elapsedTime

  SFBool   [out]    isActive

  SFBool   [out]    isPaused

}

/REFACTORED BUFFERAUDIOSOURCE

AUDIOBUFFER

BufferAudioSource : X3DUrlObject {

  SFTime   [in,out] autoRefresh           0.0        [0,∞)

  SFTime   [in,out] autoRefreshTimeLimit  3600.0     [0,∞)

  MFFloat  [in,out] bufferData            []         [−1,1]

  SFInt32  [in,out] bufferChannels        1          [0,∞)

  SFString [in,out] channelCountMode      "max"      ["max", "clamped-max",
"explicit"]

  SFString [in,out] channelInterpretation "speakers" ["speakers",
"discrete"]

  SFString [in,out] description           ""

  SFBool   [in,out] load                  TRUE

  SFNode   [in,out] metadata              NULL       [X3DMetadataObject]

  SFFloat  [in,out] sampleRate            0          [0,∞)

  MFString [in,out] url                   []         [URI]

  SFInt32  [out]    bufferLength          0          [0,∞)

  SFInt32  [out]    channelCount                     [0,∞)

}



Note on field names bufferData and bufferChannels

- web audio uses numberOfChannels as does current BufferAudioNode

- but its not clear numberOfChannels only applies the the raw PCM32 data --
not to the URL loaded .wav file which will get its channel count from the
.wav file.

- to make the association clearer I recommend the names for buffer data and
channel count have the same prefix, so will be sequential in specs
alphabetically listed fields

/AUDIOBUFFER
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230223/24338b00/attachment-0001.html>