[x3d-public] Sound v4 > channel nodes > node design comparison v4 vs Doug proposed; ChannelMerger
GPU Group
gpugroup at gmail.com
Mon Mar 6 06:39:53 PST 2023
Proposed and v4 implementation: Selections/sourceChannels are also
semi-baked connections, and I currently have no tested code for
unconnecting if a runtime SF route changed a
ChannelSelector.selectedChannel or MF route to indexSource
-Doug
On Mon, Mar 6, 2023 at 6:05 AM GPU Group <gpugroup at gmail.com> wrote:
> I take that back, proposed also semi-bakes destinations into connections
> and I don't have a tested code for un-connecting at this time.
> -Doug
>
> On Mon, Mar 6, 2023 at 5:00 AM GPU Group <gpugroup at gmail.com> wrote:
>
>> Mind you v4 design only half-passes the SF routing threshold:
>> Selections/sources. Destinations are baked in scenegraph.
>> If the routing threshold was changed to MF, then proposed would have full
>> runtime control over sources and destinations.
>> One way to save proposed, is to promise utility nodes for SF to MF in
>> v4.1.
>> -Doug
>>
>> On Sun, Mar 5, 2023 at 3:11 PM GPU Group <gpugroup at gmail.com> wrote:
>>
>>> I don;t have a good answer for changing individual elements of an
>>> MFInt32 field.
>>> I guess that use case kills the proposed method.
>>> -Doug
>>> more.. there was another browser I used about 15 years ago, and it had
>>> some utility nodes where you route an MF field to the input, and it would
>>> have 10 SF outputs, and another one going the other way - 10 SF inputs, and
>>> routed and MF.
>>> Seemed limited use with just 10. But would work for a 6 channel case.
>>> Something more general like the IntegerSequencer, which takes 2
>>> routings, one for the index (next, previous, or index) and one for the
>>> value, but for this MFInt32 case the input value would be SInt32. Then
>>> routing from the MFInt32 field in Merger to the utility function, and
>>> routing back when changed.
>>> But that would take some prose to explain the use case even if there was
>>> a utility node like that.
>>>
>>>
>>>
>>> On Sun, Mar 5, 2023 at 2:13 PM GPU Group <gpugroup at gmail.com> wrote:
>>>
>>>> Don,
>>>> Thanks for feedback.
>>>> Tuples needed. And web3d doesn't have tuple fields.
>>>> If web3d had MFInt32_3tuple field type that would be great in this
>>>> use-case.
>>>> In this case the tuple needed is (destinationChannel, sourceChannel,
>>>> audioStream)
>>>> Next best I can think of 3 MFInt32 fields --and some prose to explain
>>>> they should have the same number of entries.
>>>> It's not in w3c because its another mechanism to convert procedural
>>>> code to declarative.
>>>> -Doug
>>>> more..
>>>> I showed the v4 design works under challenging scenarios/use-cases, but
>>>> is awkward in some scenarios --requiring prose to explain. I couldn't
>>>> understand how to use the nodes as the prose was written. Maybe more
>>>> detailed prose would help.
>>>> "The Merger.children field represents a list of destinationChannels ..."
>>>> That's when I tried the 2-tuple. The prose for it would look like
>>>> "The Merger.children field represents a list of sourceChannels"
>>>> The 2-tuple didn't work with scenarios d) and e).
>>>> The 3-tuple prose would look like this:
>>>> "The Merger.children field represents a list of audio streams"
>>>> Decoupling the children field from both destinationChannel and
>>>> sourceChannel had benefits:
>>>> - no need to DEF/USE the same Splitter (or non-splitter) audio stream
>>>> to fill channel slots, if you have one splitter and/or audio stream, you
>>>> can repeat the indexStream index
>>>> - can do all the other permutations of merging and swapping channels
>>>> with indexSource and indexDestination
>>>>
>>>> PROSE COMPARISON TEST?
>>>> Perhaps a good test of design is which is easier to explain for all use
>>>> cases
>>>>
>>>>
>>>> On Sun, Mar 5, 2023 at 12:57 PM Brutzman, Donald (Don) (CIV) <
>>>> brutzman at nps.edu> wrote:
>>>>
>>>>> Thanks for continued scrutiny and experimentation efforts Doug.
>>>>>
>>>>>
>>>>>
>>>>> Primary point from last Friday is that we are trying to align with Web
>>>>> Audio API parameters closely and correctly.
>>>>>
>>>>>
>>>>>
>>>>> Design point: MFInt32/MFFloat arrays with distinct elements in the
>>>>> array are quite difficult for authors to manipulate without scripting, and
>>>>> even then tend to be quite error prone. If actually needed, it is much
>>>>> preferred to define each parameter as an independent field. Looking over
>>>>> what you’ve shared here, for example:
>>>>>
>>>>>
>>>>>
>>>>> - SFInt32 [in out] indexStream 0
>>>>> - SFInt32 [in out] indexSource 0
>>>>> - SFInt32 [in out] indexDestination 0
>>>>> - SFInt32 [in out] channelCount 0
>>>>>
>>>>>
>>>>>
>>>>> Reference
>>>>>
>>>>> -
>>>>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-DIS/Part01/components/sound.html#ChannelMerger
>>>>>
>>>>>
>>>>>
>>>>> However am not seeing a match between what you’re suggesting here and
>>>>> what W3C Audio API says, please advise.
>>>>>
>>>>>
>>>>>
>>>>> - https://www.w3.org/TR/webaudio/#channelmergernode
>>>>>
>>>>>
>>>>>
>>>>> *1.14. The **ChannelMergerNode
>>>>> <https://www.w3.org/TR/webaudio/#channelmergernode>** Interface*
>>>>>
>>>>> The ChannelMergerNode
>>>>> <https://www.w3.org/TR/webaudio/#channelmergernode> is for use in
>>>>> more advanced applications and would often be used in conjunction with
>>>>> ChannelSplitterNode
>>>>> <https://www.w3.org/TR/webaudio/#channelsplitternode>.
>>>>>
>>>>> *Property *
>>>>>
>>>>> *Value *
>>>>>
>>>>> *Notes *
>>>>>
>>>>> numberOfInputs
>>>>> <https://www.w3.org/TR/webaudio/#dom-audionode-numberofinputs>
>>>>>
>>>>> see notes
>>>>>
>>>>> Defaults to 6, but is determined by ChannelMergerOptions
>>>>> <https://www.w3.org/TR/webaudio/#dictdef-channelmergeroptions>,
>>>>> numberOfInputs
>>>>> <https://www.w3.org/TR/webaudio/#dom-channelmergeroptions-numberofinputs>
>>>>> or the value specified by createChannelMerger
>>>>> <https://www.w3.org/TR/webaudio/#dom-baseaudiocontext-createchannelmerger>.
>>>>>
>>>>>
>>>>> numberOfOutputs
>>>>> <https://www.w3.org/TR/webaudio/#dom-audionode-numberofoutputs>
>>>>>
>>>>> 1
>>>>>
>>>>> channelCount
>>>>> <https://www.w3.org/TR/webaudio/#dom-audionode-channelcount>
>>>>>
>>>>> 1
>>>>>
>>>>> Has channelCount constraints
>>>>> <https://www.w3.org/TR/webaudio/#audionode-channelcount-constraints>
>>>>>
>>>>> channelCountMode
>>>>> <https://www.w3.org/TR/webaudio/#dom-audionode-channelcountmode>
>>>>>
>>>>> "explicit
>>>>> <https://www.w3.org/TR/webaudio/#dom-channelcountmode-explicit>"
>>>>>
>>>>> Has channelCountMode constraints
>>>>> <https://www.w3.org/TR/webaudio/#audionode-channelcountmode-constraints>
>>>>>
>>>>> channelInterpretation
>>>>> <https://www.w3.org/TR/webaudio/#dom-audionode-channelinterpretation>
>>>>>
>>>>> "speakers
>>>>> <https://www.w3.org/TR/webaudio/#dom-channelinterpretation-speakers>"
>>>>>
>>>>> tail-time <https://www.w3.org/TR/webaudio/#tail-time>
>>>>>
>>>>> No
>>>>>
>>>>> This interface represents an AudioNode
>>>>> <https://www.w3.org/TR/webaudio/#audionode> for combining channels
>>>>> from multiple audio streams into a single audio stream. It has a variable
>>>>> number of inputs (defaulting to 6), but not all of them need be connected.
>>>>> There is a single output whose audio stream has a number of channels equal
>>>>> to the number of inputs when any of the inputs is actively processing
>>>>> <https://www.w3.org/TR/webaudio/#actively-processing>. If none of the
>>>>> inputs are actively processing
>>>>> <https://www.w3.org/TR/webaudio/#actively-processing>, then output is
>>>>> a single channel of silence. […]
>>>>>
>>>>>
>>>>>
>>>>> v/r Don
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> *From:* x3d-public <x3d-public-bounces at web3d.org> *On Behalf Of *GPU
>>>>> Group
>>>>> *Sent:* Sunday, March 5, 2023 5:51 AM
>>>>> *To:* X3D Graphics public mailing list <x3d-public at web3d.org>
>>>>> *Subject:* Re: [x3d-public] Sound v4 > channel nodes > node design
>>>>> comparison v4 vs Doug proposed
>>>>>
>>>>>
>>>>>
>>>>> correction to b)
>>>>>
>>>>> proposed
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indexStream='0,1'
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Sun, Mar 5, 2023 at 6:44 AM GPU Group <gpugroup at gmail.com> wrote:
>>>>>
>>>>> Here's an even better proposed design using a 3-tuple, and showing
>>>>> more tested scenarios
>>>>>
>>>>> CHANNEL NODE DESIGN COMPARISON
>>>>>
>>>>> v4: channel nodes as written in v4 Spec Draft Sound Component
>>>>>
>>>>> - merger.children is a sequential list of output/destination channels
>>>>>
>>>>> proposed: add 3 fields to ChannelMerger
>>>>>
>>>>> - MFInt32 indexStream, indexSource, indexDestination
>>>>>
>>>>> - and eliminate ChannelSelector node.
>>>>>
>>>>> - Merger.children is a list of audio streams.
>>>>>
>>>>> I have both v4 and proposed implemented in freewrl: if indexStream,
>>>>> Source or Destination is empty, thunks to v4
>>>>>
>>>>> I tested both methods on 5 scenarios and they both work as follows:
>>>>>
>>>>> a) swapping 2 channels
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> proposed
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indexStream='0,0'
>>>>>
>>>>> indexSource='0,1'
>>>>>
>>>>> indexDestination='1,0'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> b) merging 2 channels into 1
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <Gain>
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> proposed
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indexStream='0,0'
>>>>>
>>>>> indexSource='0,1'
>>>>>
>>>>> indexDestination='0,0'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> c) merging 2 channels into 2nd merger channel (skipping first output
>>>>> channel)
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <Gain/>
>>>>>
>>>>> <Gain>
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> proposed:
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indexStream='0,0'
>>>>>
>>>>> indexSource='0,1'
>>>>>
>>>>> indexDestination='1,1'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>>
>>>>>
>>>>> d) assign different audio streams to first and 2nd channnels (play
>>>>> beat in left, cnote in right)
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='2' >
>>>>>
>>>>> <Gain gain='.5'><AudioClip
>>>>> url='"sound/beat.mp3"'/></Gain>
>>>>>
>>>>> <Gain gain='.5'><AudioClip
>>>>> url='"sound/cnote.wav"'/></Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> proposed:
>>>>>
>>>>> <ChannelMerger channelCount='2'
>>>>>
>>>>> indexSteam='0,0'
>>>>>
>>>>> indexSourc='-1,-1'
>>>>>
>>>>> indexDestination='0,1'
>>>>>
>>>>> >
>>>>>
>>>>> <Gain gain='.5'><AudioClip
>>>>> url='"sound/beat.mp3"'/></Gain>
>>>>>
>>>>> <Gain gain='.5'><AudioClip
>>>>> url='"sound/cnote.wav"'/></Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> e) assign different audio streams to first and 2nd channnels via
>>>>> splitter take first channel (play beat in left, cnote in right)
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='2' >
>>>>>
>>>>> <ChannelSelector channelChoice='0'>
>>>>>
>>>>> <ChannelSplitter channelCount='2'><AudioClip
>>>>> url='"sound/beat.mp3"'/></ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelChoice='0'>
>>>>>
>>>>> <ChannelSplitter channelCount='2'><AudioClip
>>>>> url='"sound/cnote.wav"'/></ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> proposed:
>>>>>
>>>>> <ChannelMerger channelCount='2'
>>>>>
>>>>> indexStream='0,1'
>>>>>
>>>>> indexSourc='0,0'
>>>>>
>>>>> indexDestination='0,1'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter channelCount='2'><AudioClip
>>>>> url='"sound/beat.mp3"'/></ChannelSplitter>
>>>>>
>>>>> <ChannelSplitter channelCount='2'><AudioClip
>>>>> url='"sound/cnote.wav"'/></ChannelSplitter>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> /CHANNEL NODE DESIGN COMPARISON
>>>>>
>>>>>
>>>>>
>>>>> On Sat, Mar 4, 2023 at 10:20 AM GPU Group <gpugroup at gmail.com> wrote:
>>>>>
>>>>> CHANNEL NODE DESIGN COMPARISON
>>>>>
>>>>> v4: channel nodes as written in v4 Spec Draft Sound Component
>>>>>
>>>>> Doug proposed: add 2 fields to ChannelMerger MFInt32 indxDst and
>>>>> indxSrc and eliminate ChannelSelector node
>>>>>
>>>>> I have both v4 and Doug proposed implemented in freewrl: if indxDst or
>>>>> indxSrc is empty, thunks to v4
>>>>>
>>>>> I tested both methods on 3 scenarios and they both work as follows:
>>>>>
>>>>> a) swapping 2 channels
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> Doug proposed
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indxSrc='0,1'
>>>>>
>>>>> indxDst='1,0'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> b) merging 2 channels into 1
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <Gain>
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> Doug proposed
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indxSrc='0,1'
>>>>>
>>>>> indxDst='0,0'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> c) merging 2 channels into 2nd merger channel (skipping first output
>>>>> channel)
>>>>>
>>>>> v4
>>>>>
>>>>> <ChannelMerger channelCount='6' >
>>>>>
>>>>> <Gain/>
>>>>>
>>>>> <Gain>
>>>>>
>>>>> <ChannelSelector channelSelection='1'>
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter'
>>>>> channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> <ChannelSelector channelSelection='0'>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelSelector>
>>>>>
>>>>> </Gain>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>> Doug proposed:
>>>>>
>>>>> <ChannelMerger channelCount='6'
>>>>>
>>>>> indxSrc='0,1'
>>>>>
>>>>> indxDst='1,1'
>>>>>
>>>>> >
>>>>>
>>>>> <ChannelSplitter DEF='Channel_Splitter' channelCount='6'>
>>>>>
>>>>> <AudioClip url='"sound/6_Channel_ID.wav"'/>
>>>>>
>>>>> </ChannelSplitter>
>>>>>
>>>>> <ChannelSplitter USE='Channel_Splitter'/>
>>>>>
>>>>> </ChannelMerger>
>>>>>
>>>>>
>>>>>
>>>>> /CHANNEL NODE DESIGN COMPARISON
>>>>>
>>>>> -Doug
>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230306/0b4e04d4/attachment-0001.html>
More information about the x3d-public
mailing list