[x3d-public] X3D Specification Editors: Audio and Sound, 15 July 2020
Don Brutzman
brutzman at nps.edu
Wed Jul 15 11:00:46 PDT 2020
Attendees Efi Lakka, Thanos Malamos, Dick Puk, Don Brutzman.
---
1. Updated X3D4 draft Shape component to show Acoustic Properties details.
=====================================
12.4.1 AcousticProperties
AcousticProperties : X3DAppearanceChildNode {
SFFloat [in,out] absorption 0 [0,1]
SFFloat [in,out] diffuse 0 [0,1]
SFNode [in,out] metadata NULL [X3DMetadataObject]
SFFloat [in,out] refraction 0 [0,1]
SFFloat [in,out] specular 0 [0,1]
}
The AcousticProperties node determines acoustic effects including surface reflection, physical phenomena such as absorption, specular, diffuse and refraction coefficients of materials.
The absorption field specifies the sound absorption coefficient of a surface which is the ratio of the sound intensity absorbed or otherwise not reflected by a specific surface that of the initial sound intensity. This characteristic depends on the nature and thickness of the material. Particularly, the sound is absorbed when it encounters fibrous or porous materials, panels that have some flexibility, volumes of air that resonate, openings in the room boundaries (e.g. doorways). Moreover, the absorption of sound by a particular material/panel depends on the frequency and angle of incidence of the sound wave.
The diffuse field describes the sound specular coefficient, which is one of the physical phenomena of sound that occurs when a sound wave strikes a plane surface, and a part of the sound energy is reflected back into space but the angle of reflection is equal to the angle of incidence.
The refraction field determines the sound diffusion coefficient, which aims to measure the degree of scattering produced on reflection. Specifically, it is produced in the same way as the specular reflection, but in this case, the sound wavelength is comparable with the corrugation dimensions of an irregular reflection surface and the incident sound wave will be scattered in all directions. In other words, it is a measure of the surface’s ability to uniformly scatter in all directions.
The specular field describes the sound specular coefficient, which is one of the physical phenomena of sound that occurs when a sound wave strikes a plane surface, and a part of the sound energy is reflected back into space but the angle of reflection is equal to the angle of incidence.
=====================================
Several questions:
* Editors note: what is correct name, refraction or reflection? Aren't diffuse and specular coefficients for reflection? Typically refraction refers to wave bending as defined by Snell's Law.
We believe that refraction is for sound passing through the material. Good description:
[1] Wikipedia: Refraction
https://en.wikipedia.org/wiki/Refraction
* Editors note: aren't some equations needed here?
TODO:
- Efi share document illustrating refraction
- Improved definition for refraction in paragraph above
- Great diagram attached, we will probably redraw for specification
- What diagrams and prose for a new section, "16.2.x Sound Model"
Sound component 16.2.2 Sound attenuation and spatialization
might be renamed as 16.2.2 Sound propagation, attenuation and spatialization
Alternatively a separate section "16.2.3 Sound propagation" can be written.
Suggested path forward: review Efi's document again, then integrate prose and diagrams in *16.2.3 Sound propagation* as available and discussed.
Having simple sound equations adds clarity to our coefficient terms, similar to the X3D lighting model sections. We are indeed fortunate that we are taking advantage of centuries of knowledge, not trying to re-invent how sound works! Distilling this immense body of knowledge for X3D4 clarity is a simple straightforward task.
Likely future discussion points: what do we mean by refraction "through" a material? For example, a Box is provided with a refraction value in it's corresponding AcousticProperties.
- Do we expect an aural render to compute how much of the Box is occluding a sound source and adjust the received signal?
- Do we expect such computation of propagation intensity to occur external to objects or also internally?
- Can Collision proxy geometry be used for simplified geometric analysis?
- How do we indicate whether solid inside? (Use of Box Cone Cylinder Sphere geoemetry primitives defines the "solid" field already).
- Perhaps we should add "solid" field to Shape?
Suggest that our near-term goal is to capture the issues in the soon-to-be-released X3D4 draft specification so that wide-ranging consideration can continue.
----
2. Accepted nodes.
We confirmed consensus to add the "accepted nodes" from last week's minutes (listed below) to X3D4 draft. This includes nodes, fields and descriptions in Efi's documentation.
----
3. Channels
We discussed the notion of channels, particularly whether there is a one waveform (for example, one superpositioned-sinusoid audio signal) per channel, or multiple signals (such as stereo) per channel.
[2] Wikipedia: Audio signal
https://en.wikipedia.org/wiki/Audio_signal
[3] Wikipedia: Audio signal flow
https://en.wikipedia.org/wiki/Audio_signal_flow
[4] Wikipedia: Multiplexing
https://en.wikipedia.org/wiki/Multiplexing
Great news! Update published:
[4] Web Audio API
W3C Candidate Recommendation, 11 June 2020
Latest published version: https://www.w3.org/TR/webaudio/
Editor's Draft version: https://webaudio.github.io/web-audio-api/
see
* 1.14 The ChannelMergerNode Interface
* 1.15 The ChannelSplitterNode Interface
* 1.30 The StereoPannerNode Interface
[5] Web Audio API 1.0 Implementation Report
Chris Lilley, 6 September 2018
https://webaudio.github.io/web-audio-api/implementation-report.html
This report shows status of Safari, Chrome/Blink, Firefox, Edge for 8426 WebAudio API tests.
Key issue: are channels monophonic, or can they be stereo. Answer: monophonic.
The Web Audio API (appears to) defines channels as corresponding to a single source.
We are aligning with that. We are now working to best align Web Audio API with X3D architecture.
We are also looking to ensure that we are not designing the X3D alignment in a way that blocks other audio API approaches. To our expected advantage, the W3C Audio API design has already composed and harmonized requirements from multiple existing implementations. That expectation is based on (a) harmonization is the nature of W3C specification process, and (b) multiple implementations already exist, including sound libraries beyond the four listed above.
Of interest, and as expected, we intend to become active in the W3C Audio Group (as Web3D liaisons) and any differences or lessons learned from X3D4 use of this API will no doubt hold interest there.
We do need prose describing channels and concepts as a new section in Sound component. We'll start with Efi's documents for that. We will add following to draft for review:
16.4.18 ChannelSplitter (likely, need channel design discussion, next session)
16.4.19 ChannelMerger
----
Thanks for more great progress. Have fun with X3D4 Sound and W3C Audio!
v/r Don
On 7/9/2020 11:10 AM, Don Brutzman wrote:
> Minutes for 2-hour meeting today. Attendees Efi Lakka, Thanos Malamos, Dick Puk, Don Brutzman.
>
> No member-only information is included in these minutes.
>
> Warmup thoughts, design question: is there a way to slave ListenerPoint (a) to current view, or (b) each provided Viewpoint? Also, how do you return to current view if a separate ListenterPoint was bound?
>
> Details follow. No new attachments for this week's meeting, see last week's notes.
>
> On 7/8/2020 7:31 PM, Don Brutzman wrote:
>> 1. continuing...
>>
>> On 7/8/2020 10:14 AM, Richard F. Puk wrote:
>>> [...]
>>> I have the following comments on the minutes (Shown in red):
>>>
>>> In my opinion, the new X3DSoundSourceNodes should not have the work “Source” in the names of the concrete nodes as the node names are clear without it, it is redundant, and also is different from the existing nodes.
>>
>> As discussed, that is one possible solution, but other considerations also pertain.
>>
>> Relevant node names of interest are:
>>
>> +- X3DTimeDependentNode -+- TimeSensor
>> | |
>> | +- X3DSoundSourceNode -+- AudioClip (~Web Audio API: MediaElementAudioSourceNode)
>> | +- MovieTexture
>> | +- AudioBufferSource (~Web Audio API: AudioBuffer + AudioBufferSourceNode)
>> | +- OscillatorSource (~Web Audio API: OscillatorNode)
>> | +- StreamAudioSource (~Web Audio API: MediaStreamAudioSourceNode)
>> | +- MicrophoneSource
>> |
>> | +- X3DSoundDestinationNode -+- AudioDestination (~Web Audio API: AudioDestinationNode)
>
> As elaboration: specifically AudioDestination is a hardware device ID, maintained by operating system (OS).
>
> FWIW, without loss of functionality, a synonym for AudioDestination might be HardwareAudioDestination (in contrast to StreamAudioDestination).
>
>> | +- StreamAudioDestination (~Web Audio API:MediaStreamAudioDestinationNode)
>
> nowadays typically WebRTC, which is stable and approved (and usable!!) specification. 8)
>
>> + +- X3DSoundProcessingNode -+- BiquadFilter
>> | +- Convolver
>> | +- Delay
>> | +- DynamicsCompressor
>> | +- Gain
>> | +- WaveShaper
>> | +- PeriodicWave
>> |
>> | +- X3DSoundAnalysisNode -+- Analyser
>> |
>> | +- X3DSoundChannelNode -+- ChannelSplitter
>> | +- ChannelMerger
>>
>> AudioClip and MovieTexture are well understood legacy nodes and their names will not change.
>>
>> Want to voice caution here. Conceptually, the base names "AudioBuffer", "Oscillator" and "StreamAudio" by themselves might be referring to sources of audio (i.e. outputs from streams) or sinks for audio (i.e. inputs to streams). When choosing node names (i.e. the words in the X3D language) we strive for clarity and want to avoid ambiguity. So it may make sense to emphasize purpose by keeping the suffix "Source" for these nodes.
>>
>> Am confident that when we start putting Web Audio graphs together using this new set of nodes, implementation/evaluation results will either make good sense (like the Web Audio javascript) or else confusing gaps will become more evident. Example usage is important to consider.
>>
>> We'll discuss and reach initial consensus on good names during Thursday morning's teleconference, 0900-1030 pacific)
>
> We discussed this topic in exhilarating detail.
>
> Naming heuristic: "when the going gets tough, the tough get... verbose."
>
> Suggestion: for the near term we go for longer names for clarity, but once we have examples in hand, we reconsider whether shorter names can work.
>
>> 2. Dick and I made excellent progress updating the Concepts 4.4.2.3 Interface Hierarchy to match the latest meeting notes.
>>
>> We are ready to move acoustic fields from Material to AcousticProperties node in Shape component when Michalis finishes his merge of glTF PBR Pull Request 8.
>
> confirmed
>
>> Several editorial issues are pending, including:
>>
>>> The concrete node derivations need to replace “AudioNode” with the appropriate abstract node type.
>>>
>>> nDick
>>
>> Our agenda for Thursday is to finalize node list and interfaces, then start dropping in prose from Efi's detailed documentation report.
> a. Agreed on abstract types:
>
> 16.3 Abstract types
> 16.3.1 X3DAudioListenerNode
> 16.3.2 X3DSoundAnalysisNode
> 16.3.3 X3DSoundChannelNode
> 16.3.4 X3DSoundDestinationNode
> 16.3.5 X3DSoundProcessingNode
> 16.3.6 X3DSoundNode
> 16.3.7 X3DSoundSourceNode
>
> Triage follows for nodes, in order to edit document further for draft release:
>
> 16.4 Node reference
>
> b. *Accepted*
>
> 16.4.1 AudioClip
> 16.4.2 Sound
> 16.4.3 SpatialSound
> 16.4.4 ListenerPoint
> 16.4.4 AudioBufferSource
[... snip duplicate ..]
> 16.4.2 OscillatorSource
> 16.4.7 BiquadFilter
> 16.4.8 Convolver
> 16.4.9 Delay
> 16.4.10 DynamicsCompressor
> 16.4.12 WaveShaper
> 16.4.13 PeriodicWave
> 16.4.17 Analyser
> 16.4.** StreamAudioSource
>
> 16.4.14 AudioDestination
> 16.4.xx StreamAudioDestination
>
> 16.4.yy MicrophoneSource (physical hardware device)
>
> c. *Not Included*
>
> 16.4.5 MediaElementAudioSource (same as AudioClip)
> 16.4.6 MediaStreamAudioSource (same as StreamAudioSource)
>
> 16.4.15 MediaStreamAudioDestination (same as StreamAudioDestination)
> 16.4.16 MediaStreamTrack
> 16.4.1 AudioParam (merged functionality? not used)
> 16.4.21 Panner (merged functionality in SpatialSound)
>
> d. *Pending major issues, decision TBD*
>
> 16.4.5 BinauralListenerPoint (will attempt to merge with ListenerPoint)
> (SurroundSound might also be a variation on ListenerPoint)
>
> 16.4.18 ChannelSplitter (likely, need channel design discussion, next session)
> 16.4.19 ChannelMerger (likely, need channel design discussion, next session)
>
> 16.4.1 AudioContext (perhaps integrated within interface hierarchy, more discussion)
>
> 16.4.11 Gain (included as a field in many nodes, or within interface hierarchy)
>
> Virtual Microphone Sensor (perhaps same as ListenerPoint or media stream?)
>
> How to simply pan or rebalance left/right?
>
> SFNode/MFNode fields defining parent-child node relationships, allowing tree-like construction of a W3C Audio API graph using X3D nodes.
>
> ==================
>
> 3. Next steps:
>
> a. Request: Efi please confirm all field definitions are correct for the Accepted nodes - so far so great.
>
> b. TODO Don and Dick will update draft specification in github, specifically nodes fields and prose, using Efi's two provided analysis/report documents.
>
> c. Objective: shift sharpest focus from our draft notes to actual github spec as our outcome document.
>
> d. What are our example audio graphs to represent in X3D?Looking at examples will help us confirm that the right fields and parent-child node relationships are described well. Efi has a detailed example comparing earlier/evolved examples that will help. She can (and will soon) release this. As they are each reviewed and considered mature, we will place them online in version control as part of X3D Examples Archive.
>
> e. It is clear that a good amount of work remains, but at least 80% is well defined and sensible. We need to ship a draft X3D4 Sound Component that reflects this progress, noting more work to follow. This will allow review and assessment (and possibly engagement) by many others.
>
> f. It is not completely certain yet, but we are very close hope to resolve this sufficiently to support a Web3D 2020 Conference paper. This is important and primarily a restructuring of Efi's excellent documents updates in paper form. We can certainly support a Web3D 2020 Conference tutorial.
>
> * Statistics joke: "The first 90% of the work takes 90% of the time. The last 10% of the work takes the other 90% of the time." (perhaps similar to Zeno's Paradox?!)
>
> * Zeno's Paradoxes
> https://en.wikipedia.org/wiki/Zeno%27s_paradoxes
>
> Summary of our near-term objective: produce a sufficiently advanced X3D4 Sound Component that enables meaningful understanding, implementation, evaluation and finalization.
>
>> We are getting pretty close to releasing the Sound component for inclusion in draft X3D4 Specification.
>>
>> Due diligence continues. All feedback remains helpful and welcome.
>
> Next meeting of group: back to Wednesday 15 July, same time.
>
> "What say?" Have fun with X3D Audio and Sound! 8)
>
> all the best, Don
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SoundPropagationPhenomena.png
Type: image/png
Size: 337473 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200715/5d1d5bfd/attachment-0001.png>
More information about the x3d-public
mailing list