[x3d-public] X3D minutes 17 April 2020: HAnim2 approved by ISO; X3D4 Audio and Sound Component
Don Brutzman
brutzman at nps.edu
Sun Apr 19 18:41:33 PDT 2020
1. *Summary*. Today's session focused on X3D Sound Component and work by Efi Lakka and Athanasios Malamos to gain compatibility with W3C Web Audio components that are commonly being implemented in HTML5 browsers. Numerous references are also included.
We met Friday 17 April 2020 0800-0940 for regular weekly meeting of X3D Working Group.
Attendees: Efi Lakka, Athanasios Malamos, Vince Marchetti, Nicholas Polys, Dick Puk, Anita Havele, Don Brutzman.
[1.1] X3D Graphics Working Group
https://www.web3d.org/working-groups/x3d
Confirmed that no member-only information is present in these minutes. Noted that the information here:
- helps members pay full attention to the detailed improvement of X3D4,
- helps community understand and trust the diligence of the process, and
- helps everyone understand that Web3D Consortium offers serious member value!
https://www.web3d.org/join
Note that all Web3D zoom conference lines are now protected by password.
[1.2] Web3D Teleconference Information
https://www.web3d.org/member/teleconference-information
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2. *ISO Publication of HAnim*
Good news! Dick and I worked for about an hour during one of our twice-weekly X3D Specification Editor teleconferences to confirm that the ISO version (published November 2019) indeed matches (a) what we submitted, and (b) the Web3D version. Everything looked great. 8)
As most people know, these documents are available to members on github.
Github links:
[2.0] GitHub: Web3D Consortium
https://github.com/Web3DConsortium
[2.1] GitHub: Web3D Consortium members, HAnim
https://github.com/Web3DConsortium/HAnim
[2.2] GitHub: Web3D Consortium members, HAnim Specifications combined parts 1 and 2
https://github.com/Web3DConsortium/HAnim/raw/master/ISO-IEC19774-combined-IS.Web3D.zip
We've handed them off to Executive Director Anita and the Webmaster to post on web3d.org pages. These go on web3d.org under STANDARDS (All, Recommended, HAnim) replacing the v2.0 DIS version with this new v2.0 International Standard (IS) version.
Next steps:
- publish on web3d.org,
- public announcement (Anita will give initial draft to HAnim principals) with completion goal next week,
- track ISO correction of their catalog pages (H-Anim to HAnim),
- track ISO addition of HAnim specifications to their freely available ISO specifications,
- Joe and Don are now reviewing HAnim 1 and 2 examples for correctness. This take a number of weeks because the example version numbers need close review, and because not all prior changes were applied thoroughly).
- if needed, submit a corrigendum to ISO for updated HAnim examples. Dick can help with the process.
Suggested: followup with Medical Working Group about future strategies taking advantage of new HAnim capabilities; also DICOM HealthLevel7 and IEEE 3D Body Processing (3DBP). Perhaps OGC GeoPose too?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
2.5 *Web3DUX Working Group*
Announcement needed, Anita will contact working group chairs. Nicholas Anita Don and Dick will continue helping to encourage broader activity. They meet every other Wednesday on Web3D Teleconference line 1015-1100 pacific.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
3. *Audio Improvements and Sound Component*
Summary: Prior review by X3D Working Group accepted this work for X3D4. We are thrilled to see much progress today. Much detail follows.
This week's X3D Working Group teleconference features Efi Lakka and Professor Athanasios Malamos reviewing significant changes to X3D Sound Component for spatialization of sound and alignment of audio capabilities with current Web capabilities. This builds on capabilities first reported in the Web3D Conference held in Crete a few years ago. Her work includes updates to the X3D Sound Component (as a github pull request) plus demo scenes implemented in X3DOM.
Efi Lakka reviewed the overall design and summarized her prior work. Pull Request 9 information follows, also see attached notification from github.
[3.1] X3D4 PR9
===============================================================
You can view, comment on, or merge this pull request online at:
https://github.com/Web3DConsortium/X3D/pull/9
"Draft replacement to Sound Component for X3D4 Abstract Architecture, adding improved spatial audio rendering capabilities that take advantage of W3C Audio recommendations."
Patch Links:
* https://github.com/Web3DConsortium/X3D/pull/9.patch
* https://github.com/Web3DConsortium/X3D/pull/9.diff
===============================================================
[3.2] Multimedia Laboratory, Hellenic Mediterranean University
http://medialab.hmu.gr
[3.3] For everyone who wants to see how this first approach works, visit
http://medialab.hmu.gr/minipages/x3domAudio/index.html
Background document on X3D4 Implementations page is quite thorough. Small changes as version 3 will be released shortly.
[3.3] X3Dv4 Implementations Status
https://www.web3d.org/x3dv4-implementations
[3.3] Attachment X3dAudioExtension_v02.pdf
https://www.web3d.org/sites/default/files/page/X3Dv4%20Implementations/X3dAudioExtension_v02.pdf
[3.4] TODO needed: slideset shown during meeting
___________
Structure of hooking together various audio filters and sources is interesting. Generally design of an audio "chain" is accomplished primarily by hierarchical parent-child field references, not ROUTE connections.
Addition of AcousticMaterial and properties will soon be included - seems like the right idea to do now since it is part of the spatialization computations. Are extra properties needed? an important area for discussion and continued work. We will temporarily add to Sound component for initial review, then into Shape component when confirmed robust.
Implementation and evaluation will guide dialog and decision. Suggest specification prose sooner (at least as placeholders) and any additional implementation to follow shortly after. For now, "Better is the enemy of Good Enough" to keep building understanding and momentum.
X3DOM examples are more mature now, and can be found at
[3.5] TODO needed: latest examples link
___________
/Discussion/
Q: When can X_ITE plan to begin implementation work? Likely in another few months, probably best for everyone to focus completely on X3DOM implementation completely. (Andreas Plesch has visibility into both codebases and can advise.)
Q: Is JavaScript necessary for implementation support? Not necessarily. Most code so far has been moved into the X3DOM implementation. Some might be used for audio animation-style effects, see examples.
Q: So far, are you only using browser audio?
A: yes.
Q: Have you seen any audio implementations other than HTML browsers? This might be helpful for some of the non-HTML X3D players (view3dscene, FreeWrl, instantreality et al.). A: have not looked at that. More investigation by others is appropriate.
Q: How close is X3D audio content to HTML5 audio code? Can similar effects be achieved either inside X3D scene or above in enclosing HTML page? Will HTML audio authors find it familiar?
A: X3D4 authors will find the new component as easy to use as the existing X3D3 component. Implementation complexity is mostly hidden.
Q: What is your assessment of maturity of the W3C Audio candidate recommendations (CRs)?
A: They seem stable but not always complete. More study and feedback appears appropriate.
Background W3C resources of interest:
-----
[a.1] W3C » Standards » Web Design and Applications » Audio and Video
https://www.w3.org/standards/webdesign/audiovideo
[a.2] W3C Wiki: Audio
https://www.w3.org/wiki/Audio
[a.3] Making Audio and Video Media Accessible
https://www.w3.org/WAI/media/av
[a.4] W3C Standards and Drafts: Audio
https://www.w3.org/TR/?title=audio
[a.5] Web Audio API
W3C Candidate Recommendation, 18 September 2018
Latest published version:
https://www.w3.org/TR/webaudio
"Abstract. This specification describes a high-level Web API for processing and synthesizing audio in web applications. The primary paradigm is of an audio routing graph, where a number of AudioNode objects are connected together to define the overall audio rendering. The actual processing will primarily take place in the underlying implementation (typically optimized Assembly / C / C++ code), but direct script processing and synthesis is also supported.
The Introduction section covers the motivation behind this specification.
This API is designed to be used in conjunction with other APIs and elements on the web platform, notably: XMLHttpRequest [XHR] (using the responseType and response attributes). For games and interactive applications, it is anticipated to be used with the canvas 2D [2dcontext] and WebGL [WEBGL] 3D graphics APIs."
"Status of this document. [...] This document was published by the the Web Audio Working Group as a Candidate Recommendation. This document is intended to become a W3C Recommendation. [...] A preliminary implementation report is available."
[a.6] "CanIUseWebAudio"
https://caniuse.com/#feat=audio-api
[a.7] Audio Output Devices API
W3C Candidate Recommendation 03 October 2017
Latest published version:
https://www.w3.org/TR/audio-output
"Abstract. This document defines a set of JavaScript APIs that let a Web application manage how audio is rendered on the user audio output devices."
"Status of This Document. [...] While the specification is feature complete and is expected to be stable, it might evolve based on implementation feedback that will be gathered as its associated test suite evolves. This test suite will be used to build an implementation report of the API; to go into Proposed Recommendation status, the group expects to demonstrate implementation of each feature in at least two deployed browsers."
[a.8] Web MIDI API
W3C Working Draft 17 March 2015
Latest published version:
http://www.w3.org/TR/webmidi
[a.9] Web Audio Processing: Use Cases and Requirements
W3C Working Group Note 29 January 2013
Latest published version:
http://www.w3.org/TR/webaudio-usecases
includes
2.2 3D game with music and convincing sound effects
https://www.w3.org/TR/webaudio-usecases/#x3d-game-with-music-and-convincing-sound-effects
and
2.11 User Control of Audio
https://www.w3.org/TR/webaudio-usecases/#user-control-of-audio
[a.9] Audiobooks
W3C Candidate Recommendation 17 March 2020
Latest published version:
https://www.w3.org/TR/audiobooks
"Abstract. This specification describes the requirements for the creation of audiobooks, using a profile of the Publication Manifest specification."
[a.10] HTML5.2, section 4.7.11 The audio element
https://www.w3.org/TR/html52/semantics-embedded-content.html#the-audio-element
"An audio element represents a sound or audio stream." [...]
[a.11] Audio Working Group
https://www.w3.org/2011/audio
"Mission. The mission of the Audio Working Group is to add advanced sound and music synthesis capabilities to the Open Web Platform."
[a.12] Publishing Working Group
https://www.w3.org/publishing/groups/publ-wg
"Mission. The mission of the Publishing Working Group is to enable all publications—with all their specificities and traditions—to become first-class entities on the Web. The group will provide the necessary technologies on the Open Web Platform to make the combination of traditional publishing and the Web complete in terms of accessibility, usability, portability, distribution, archiving, offline access, and reliable cross referencing."
-----
Q: Is "chain" the right term for an effect? We should likely use same term as W3C Audio group uses.
A. TODO check/confirm.
Within the Web Audio CR, looks like the proper term is "modular routing" for composition of AudioNode source-node outputs and destination-object inputs.
[a.13] Web Audio API, Features: Modular Routing
https://www.w3.org/TR/webaudio/#ModularRouting
Q: Is the audio "chain" of source-filter-effect-spatialization effect represented as your .mp3 file?
A: TODO explain relationship.
Tool of interest:
[a.14] Web Audio Inspector (Chrome plugin)
https://opensource.google/projects/audion
"Audion (Web Audio Inspector) is a Chrome extension that adds a Web Audio panel to dev tools. This panel visualizes the web audio graph in real-time and lets users inspect nodes."
Erratum: for example shown during screenshare, <X3DSoundSourceNode> should be <AudioClip> (or perhaps another newer node)... the object interface might be functionally correct but we need to be careful about syntax, can't use abstract node types in X3D model files.
Don and Dick plan to work during Tuesday's X3D4 Specification Editors call to drop Efi's draft on top of existing X3D Sound component, see if we are ready to check in, if good enough then accept the merge and keep moving forward.
Next steps for action:
a. Marking up specification changes (css for proposed/yellow, deleted/orange) after including the Pull Request (PR9) in draft spec.
b. Are there any node name changes to consider? Create summary list for confirmation. Use Mantis to track specific changes.
c. Describing X3DOM implementation
d. Publishing example models, evaluate implementation correctness.
e. Encourage additional implementations X_iTE and other players - Web3D Call for Implementations in June, perhaps?
f. demonstration video for "Learning X3D4" later when stable, a draft video was created today. Thanks! Anita will publish privately with X3D Working Group.
g. prepare paper for Web3D 2020 in November? we hope so.
Of related note: Web3D Consortium has the opportunity to nominate individual member(s) to participation in W3C Spatial Audio group. This can really help us achieve all of our goals effectively.
[5.1] [x3d-public] Fwd: Audio Working Group Charter Approved; Call for Participation
https://web3d.org/pipermail/x3d-public_web3d.org/2020-April/012060.html
Subject to level of effort while implementing X3D4 Sound, Thanos and Efi are willing to participate. Don will participate as well and lead the initial engagement. Anyone else?
Time expectation for completion: primary improvements are complete and demonstrated. More incremental work will continue over a number of months. Goal completion for initial examples and specification is June with announcement of near-completion at SIGGRAPH in July. Working together with W3C colleagues will help.
Our plan is to check status on integrating PR9 Audio into the X3D4 draft in 2 weeks, 1 May 2000, at 0900 pacific, one hour into the regular X3D Working Group teleconference. (Please note that it is a bank holiday in Crete!)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
4. Fixing Mantis access: partially complete.
Webmaster has provided access to Mantis for following contributors:
> On 4/7/2020 11:54 AM, Don Brutzman wrote:
>> Webmaster, please add the following members to Mantis issue tracker:
>>
>> Athanasios Malamos <athanasios.malamos at gmail.com>
>>
>> Efi Lakka <efilakka at gmail.com>
>>
>> Jakub Flotyński <flotynski at kti.ue.poznan.pl>
>>
>> Thanks for all support.
However need to add them to X3D4 project. Didn't work for me so far, apparently due to lack of permissions. Have again requested webmaster support.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Suggested topic for next week: Cross Origin Resource Sharing (CORS), following up on prior meeting topics.
Onward we go. Have fun with X3D audio and sound! 8)
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
-------------- next part --------------
A non-text attachment was scrubbed...
Name: X3dSpecificationWebAudioApiPullRequest9.pdf
Type: application/pdf
Size: 99611 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200419/7ec90122/attachment-0001.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: HaveFunWithX3D.wav
Type: audio/wav
Size: 33306 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200419/7ec90122/attachment-0001.bin>
More information about the x3d-public
mailing list