[x3d-public] X3D Audio meeting minutes 28 OCT 2020

Don Brutzman brutzman at nps.edu
Wed Oct 28 11:06:53 PDT 2020 

Minutes for today.

Attendees Athanasio Malamos, Dick Puk, Don Brutzman.  Regrets Efi Lakka.

1. Discussed passthrough... looked at Web Audio API.  Modified

> TODO: if enabled FALSE, does signal pass through unmodified or is it blocked? Perhaps an additional boolean is needed for pass-through state? Modeling the 'connect' attribute and defining defaults is necessary for each case.

to

"If <i>enabled</i> field is FALSE, the audio signal passes through unmodified and is not blocked."

---

2. smoothingTimeConstant

https://www.w3.org/TR/webaudio/#fft-windowing-and-smoothing-over-time

It looks like something set by author, having accessType [in out] with range [0,1]

So our draft spec is OK.

---

3. interaural distance and binary recording

* https://www.w3.org/TR/webaudio/#ref-for-dom-panningmodeltype-hrtf

Proposed:

"The <i>interauralDistance</i> field can be used for binaural recording or precision sound-reproduction headgear."

This is not explicitly listed in Web Audio API, and that seems like an omission.  Important, they talk about an HRTF database and left/right "convolution engine" in a figure but provide no details and no way to set parameters in the Convolver node.  We can add this to our list of comments for them.

This feature can be implemented today simply by creating two ListenerPointSource nodes for left and right ears, separated by the interauralDistance.

We know that we want such a capability in X3D players.  No reason to limit ourselves to single-point audio recording or listening.

If we say nothing, then X3D authors have no way to make a binaural recording of virtual sound.  So let's keep it in with a TODO.

We expect to be able to engage with W3C Audio Working Group after the Web3D 2020 Conference in November, after hearing and reconciling everyone's comments about X3D4.

---

4. Surround sound, Dolby ATMOS, other advanced scenarios. etc.

Dick proposed and we discussed some interesting examples related to surround sound.

It appears that W3C Audio API, and our corresponding representation, don't necessarily have direct support or specific parameters for each of these examples.  Nevertheless it appears like there might be sufficient expressive power when putting together components in the API.

It would be helpful to describe the desired use-case capabilities in detail and see if we can model them.

It appears that we will be able to build a very interesting library of examples that either works (most likely for what we've imagined) or else shows necessary gaps.

---

5. Draft specification cleanup.

Looks like we can clean up the existing draft without any loss of information.  Makes it more reviewable.

We agreed that was OK, Don will execute.  Efi please holler if you have any problems.

----

6. All the infrastructure is in place to test more examples.

Efi tell me which ones you want for the tutorial, I will do them next.

We hope to rehearse and record and post our respective Web3D 2020 tutorial sections next week.

* Tutorial #5, X3D4 Spatial Sound, Web Audio API and Acoustic Properties
   https://web3d.siggraph.org/x3d-audio-graphs-acoustic-properties-3d-spatial-sound/

----

7. Conversions and loading native Web Audio API via url?

a. Of note is that we can create a Web Audio API graph as X3D, that an X3D4 player can implement via Web Audio API itself.

Efi's examples use a modified version (extensions) of X3DOM.

We expect that these changes can someday be added to the X3DOM trunk.

b. Likely a similar approach can be done with X_ITE.  (Hint hint...)

c. We are interested in knowing what other players might be planning such support some day, for example Castle Game Engine or FreeWrl.

d. Don plans to write (by end of year) an XSLT stylesheet that takes X3D in XML format and converts audio nodes there to a pure Web Audio API Javascript source.

e. A converter going the other way (Web Audio JavaScript into X3D) is possible.  Nobody has written it yet (to our direct knowledge) but that seems possible, and perhaps even likely.

f. While our new X3D Sound Component is written with Web Audio API in mind, we do not require that API explicitly in the X3D Architecture Specification, and it seems quite conceivable that a programmer might implement support for this component using another different audio API.  (Indeed that seem plausible since many different implementers helped design the Web Audio API.)  For X3D players that aren't running in a Web browser, this flexibility is important.

g. It is interesting therefore to consider an optional way to load a web audio JavaScript source code via url, either in SpatialSound or perhaps an Inline.

h. We have no other node with a children field that contains a URL field. This may cause all sorts of perturbations to implementations. Also if the implementation is providing an alternative sound implementation, requiring that the Web Audio API implementation would add an unnecessary amount of work for those implementers.

i.  We're not ready to define such an optional capability yet, and certainly don't want to require it.  Nevertheless interesting to consider.

OK nearly complete.  Thanks for steady progress, all feedback welcome.


On 10/27/2020 9:40 PM, Brutzman, Donald (Don) (CIV) wrote:
> [trying again]
> 
> Looking at new X3DSoundSourceNode nodes for X3D4 audio graphs:
> 
> <WaveShaper>
>    <!-- X3DSoundSourceNode nodes have no children -->
>    <ListenerPoint/>
>    <MicrophoneSource/>
>    <OscillatorSource/>
>    <AudioBufferSource/>
>    <StreamAudioSource/>
> </WaveShaper>
> 
> For consistency and clarity (avoiding some of the twisty turns we took last week, I think a better name for ListenerPoint is ListenerPointSource.
> 
> We still need to look at those use cases, but I think it is clearer.  Am applying this change (and various other review corrections) tonight for our group review tomorrow, since it each full deployment is quite a bit of work.  we can always change it back.

Applied and reviewed without complaint.

Tested:  X3D Validator + everything else is updated and matching.


> ________________________________________
> From: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> Sent: Sunday, October 25, 2020 22:01
> To: Efi Lakka; Athanasios Malamos; Richard F. Puk
> Cc: X3D Graphics public mailing list
> Subject: X3D Audio meeting minutes, updated sound component, XML/java/python/turtle/validator updates
> 
> 1. Thanks for the excellent meeting of the Audio and Sound team Friday.  Steady bashing is knocking this into steadily improving state.
> 
> The refinements we discussed were applied to good effect, passing all X3D infrastructure tests associated with X3DUOM.
> 
> Synopsis changelog:
> 
> X3D XML Schema update activity:
> https://www.web3d.org/specifications/x3d-schema-changelog.txt
> 
> 24 OCT 2020, brutzman, puk, athanasios, lakka, kamburelis, plesch
> - (v4.0) rename ChannelSelector field channelNumber as channelSelection for clarity
> - (v4.0) AudioBufferSource includes boolean field 'loop'
> - (v4.0) pitch moved from abstract type X3DTimeDependentNode to
>            AudioClip, MovieTexture with no change to X3D3 functionality
> - (v4.0) duration_changed moved from abstract type X3DSoundSourceNode to
>            AudioClip, MovieTexture with no change to X3D3 functionality
> - (v4.0) loop moved from abstract type X3DTimeDependentNode to
>            TimeSensor, AudioClip, MovieTexture with no change to X3D3 functionality
> - (v4.0) ListenerPoint not X3DBindableNode
> - (v4.0) X3DLightNode shadowIntensity, shadows (enabled); X3DShapeNode castShadow
> - (v4.0) TextureProjectorPerspective renamed as TextureProjector,
>            TextureProjectorParallel    remains as originally named
> - (v4.0) Sound component audio graphs: all node types and nodes
> - (v4.0) Shape component audio graphs: AcousticProperties
> - (v4.0) PointProperties pointSizeAttenuation -> attenuation, type SFVec3f
> - (v4.0) PointProperties omit colorMode, colorModeChoices
> - (v4.0) revise default values for nearDistance/farDistance
> 
> X3D4 XML Schema, XML DOCTYPE and X3DUOM sources and documentation all updated, deployed and available at
> 
> [1] X3D Specifications: Schema and DOCTYPE Validation
>       https://www.web3d.org/specifications/
> 
> Attached in follow-on message is PDF of github draft for your reviewing convenience.  As before, have applied light-blue "editorsNote" CSS styling to changes I've made.  These changes are quite detailed but reviewable.
> 
> Following our weekly review this coming Wednesday, I hope to earn each of your permissions to clean up this entire document by sorting fields and removing strikeouts of our previous draft strategies.  No more "blue" after that in preparation for next phase of complete document cleanup next month.
> 
> ---
> 
> 2. I've created an initial test scene AllAudioGraphNodes.x3d that has been helpful for debugging.
> 
> Attached, with a bit more work this example (and then your excerpted examples Efi) will be getting pushed to
> 
> [2.1] X3D for Advanced Modeling (X3D4AM) Examples Archive
>         https://x3dgraphics.com/examples/X3dForAdvancedModeling
>         https://x3dgraphics.com/examples/X3dForAdvancedModeling/AudioSpatialSound
> 
> All audio tooltips also updated, which provides a number of further checks on correctness and clarity.  That was a big pile of work, and helpful for all of us as a ready reference to ensure that we have a common understanding.  Incidentally those tooltips (describing each node and field) also become embedded documentation for Java, Python and Turtle libraries.
> 
> [2] X3D4 Tooltips
>       https://www.web3d.org/x3d/tooltips/X3dTooltips.html
> 
> Sound component / audio-graph nodes of interest:
> 
> Analyser
> AudioBufferSource
> AudioClip
> AudioDestination
> BiquadFilter
> ChannelMerger
> ChannelSelector
> ChannelSplitter
> Convolver
> Delay
> DynamicsCompressor
> Gain
> ListenerPoint
> MicrophoneSource
> MovieTexture
> OscillatorSource
> PeriodicWave
> Sound
> SpatialSound
> StreamAudioDestination
> StreamAudioSource
> WaveShaper
> 
> ---
> 
> 3. Java X3DJSAIL built, passing tests and deployed.  Now testing under new OpenJDK 15.0.1.  8)
> 
> [3] X3D Java Scene Access Interface Library (X3DJSAIL)
>       https://www.web3d.org/specifications/java/X3dJavaSceneAuthoringInterface.html
> 
> (Sorry John but nashorn and jjs are past deprecation and completely gone now, so we'll have to figure out another connection.)
> 
> ---
> 
> 4. Python x3d.py X3DPSAIL built, passing tests and deployed in Python Packaging Index (PyPi).  Now testing under new Python 3.9.  8)
> 
> [4.1] X3D Python Scene Access Interface Library (X3DPSAIL)
>         https://www.web3d.org/x3d/stylesheets/python/python.html
> 
> [4.2] PyPi Python package x3d
>         https://pypi.org/project/x3d
> 
> ---
> 
> 5. X3D Ontology also updated, passing tests and deployed.
> 
> [5.1] X3D Ontology for Semantic Web
>         https://www.web3d.org/x3d/content/semantics/semantics.html
> 
> Now includes papers and slideset for upcoming conference presentation.
> 
> [5.2] X3D Ontology for Querying 3D Models on the Semantic Web, Don Brutzman and Jakub Flotyński,
>         Web3D 2020 Conference, Seoul Korea, 9-13 November 2020, with accompanying presentation.
>         https://www.web3d.org/x3d/content/semantics/documentation/papers/Web3D2020BrutzmanFlotynskiX3dOntology.pdf
>         https://www.web3d.org/x3d/content/semantics/documentation/papers/Web3D2020BrutzmanFlotynskiX3dOntologyPresentation.pdf
> 
> ---
> 
> 6. X3D Validator was updated this morning, it will continue tracking these improvements.
> 
> [6] X3D Validator
>       https://savage.nps.edu/X3dValidator
> 
> ---
> 
> So... a lot is coming together and converging on final X3D4.  Having good quality X3D4 models should simplify other efforts.
> 
> Have fun with X3D4 and audio graphs!  8)


all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman

More information about the x3d-public mailing list