[x3d-public] MIDI suggestions (GPU Group)

Andreas Plesch andreasplesch at gmail.com
Fri Aug 23 11:12:19 PDT 2024


I was curious and found there is a Web MIDI API:
https://developer.mozilla.org/en-US/docs/Web/API/Web_MIDI_API
which is supported on Browsers other than Safari.
There is also a support library: https://webmidijs.org/
This is focused on connected external input devices like a keyboard,
not necessarily on playing midi files.
There is a midi file player: https://cifkao.github.io/html-midi-player/
which uses https://github.com/magenta/magenta-js/tree/master/music/
which does its own TensorFlow(!) based tone generation.
I also found https://surikov.github.io/webaudiofont/examples/midiplayer.html
which seems capable.
-Andreas

On Fri, Aug 23, 2024 at 12:16 PM <x3d-public-request at web3d.org> wrote:
>
> Send x3d-public mailing list submissions to
>         x3d-public at web3d.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> or, via email, send a message with subject or body 'help' to
>         x3d-public-request at web3d.org
>
> You can reach the person managing the list at
>         x3d-public-owner at web3d.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of x3d-public digest..."
>
>
> Today's Topics:
>
>    1. Re: MIDI suggestions (GPU Group)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 23 Aug 2024 10:15:13 -0600
> From: GPU Group <gpugroup at gmail.com>
> To: "Extensible 3D (X3D) Graphics public discussion"
>         <x3d-public at web3d.org>
> Subject: Re: [x3d-public] MIDI suggestions
> Message-ID:
>         <CAM2ogRfZ8=+3K+pHdu2DiNLRB=A3WJyzoesn+UYUcPu_PYaHBg at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Don,
> I used MIDI Editor but just briefly, didn't master it.
> PROTOS > General Sequencing Node - something that sequences similar to MIDI
> that could be done in a PROTO is a general sequencing node, which could
> have the benefits of a text/string field for in-scene hacking and multiple
> output fields like the MIDI music octave style node type.
> GeneralSequencer {
> SFString in/out sequence
> SFTime in start
> SFBool out o1
> SFBool out o2
> SFBool out o3
> ...
> SFBool out o100
> }
> example
> DEF GS1 GeneralSequencer {
> # out field number, out field value, time offset from start
> sequence "
> 1 T 0.0
> 1 F 0.25
> 1 T 1.0
> 2 T 1.0
> 3 T 1.0
> 4 T 1.0
> "
> # or time offset from start, out value, list of fields
> "
> 0.0 T 1,
> .25 F 1,
> 1.0 T 1 2 3 4,
> "
> or something like that
> }
> The general sequencer could be convenient for naval simulations -with the
> string sequence being replaced according to different attack or submersible
> robot maneuvers.
> And convenient for musical HANIM sequences to synchronize musical riff play
> --using standard sound nodes-- and certain humanoid articulations.
> On each frame the Proto code would check if there are any in the sequence
> that have (start + offset) time > current time and haven't been played, and
> play them.
>
> -Doug
>
> On Fri, Aug 23, 2024 at 8:54?AM Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
> > OK very good, thanks for information Doug.
> >
> > Meanwhile, am hoping to improve the MIDI files that we have, and
> > potentially add more examples.  Wondering if there is a good open-source
> > software tool for MIDI?
> >
> > I looked again within Audacity, which is an excellent open source tool.
> > It does support MIDI but does not seem to give any information about what
> > type of MIDI file is loaded or exported.  Some vulnerabilities are noted in
> > NIST National Vulnerability Database but they are all on very old versions,
> > so it seems trustworthy.  Linked for installation and easy use within
> > X3D-Edit as a helper application.
> >
> >
> >    - https://www.audacityteam.org
> >    -
> >    https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=Audacity
> >
> >
> > Previous I was using MuseScore every now and then.  However an upgrade
> > earlier this year integrated some kind of p2p streaming activity (aka Muse
> > Hub) that triggered a variety of security warnings, so I stopped using it.
> > However, I now see that they offer a version without Muse Hub, will try
> > again.
> >
> >
> >    - https://musescore.org/en
> >    -
> >    https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=musescore
> >
> >
> > The midieditor looks possible but it is free software (hobbyist) and not
> > open source, so less appealing.  Also does not have an https server, which
> > is a red flag.
> >
> >
> >    - http://www.midieditor.org
> >
> >
> > Am hoping to test binary and text-based MIDI (and possibly MIDI version 1
> > and 2) using different versions of the example file.
> >
> >
> > Wondering if there are any open-source audio tools you can recommend that
> > support
> >
> >
> >    - Identifying MIDI version within a file,
> >    - Exporting between different MIDI formats
> >
> >
> > all the best, Don
> >
> > --
> >
> > Don Brutzman  Naval Postgraduate School, Code USW/Br
> > brutzman at nps.edu
> >
> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> > +1.831.656.2149
> >
> > X3D graphics, virtual worlds, navy robotics
> > https://faculty.nps.edu/brutzman
> >
> >
> > ------------------------------
> > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU Group
> > via x3d-public <x3d-public at web3d.org>
> > *Sent:* Friday, August 23, 2024 7:18 AM
> > *Cc:* GPU Group <gpugroup at gmail.com>; Extensible 3D (X3D) Graphics public
> > discussion <x3d-public at web3d.org>
> > *Subject:* Re: [x3d-public] MIDI suggestions
> >
> > Don,
> > MIDI via PROTOs > working in C I used threads and queues between the MIDI
> > port and the x3d rendering thread, and I don't know how to do that with
> > PROTOs. But it might be different with Web MIDI API. So one idea is to wait
> > until/in-case one of the web based x3d browsers implements a MIDI component
> > using PROTOs, and see how they did that, and then the native browsers can
> > adapt to that.
> > -Doug
> >
> > On Thu, Aug 22, 2024 at 6:35?PM Brutzman, Donald (Don) (CIV) <
> > brutzman at nps.edu> wrote:
> >
> > Understood.  The work you describe regarding MIDI nodes looks quite
> > involved.  Possible component extension if someone did the work involved.
> > Implementing as Prototype library using Script nodes might work.  Writing
> > extensions for X_ITE or X3DOM might work.  In each case that would be
> > proof-of-capability and maybe not a path to wide availability.
> >
> > I think it is doubtful that MIDI editing would be accepted as a high-value
> > use case and recommended practice for X3D 4.1, which might take an
> > undetermined length of time anyway.  Seems like a long, unlikely and
> > unsatisfying path to pursue.  Nevertheless possible.
> >
> > I looked in Audacity briefly for "saving as" MIDI ASCII but didn't find
> > it.  Perhaps I missed it or another tool is useful.
> >
> > If we do have a text-base example MIDI file archived, that might encourage
> > use of other online editors.  Note how X_ITE is already able to show XML,
> > ClassicVRML and JSON encodings equivalently (see links in upper-right
> > corner).
> >
> >
> >    - X_ITE Playground Editor
> >    - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
> >    Appearance Material Textures, Pixel Texture Snowman
> >    -
> >    https://create3000.github.io/x_ite/playground/?url=https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
> >
> >
> >
> >    - ... but has error:  Couldn't load URL '
> >    https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid'.EncodingError:
> >    Failed to execute 'decodeAudioData' on 'BaseAudioContext': Unable to decode
> >    audio data
> >
> >
> >
> >    - X3DOM similarly seems to have a problem paying the midi file
> >    -
> >    https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanX3dom.xhtml
> >
> >
> >
> >    - Nothing heard using latest FreeWrl
> >    - Nothing heard using current Castle Model Viewer
> >
> >
> > Audacity plays the .midi file OK.  Might there be a problem with the file?
> >
> > Regarding sound-related implementations, am hoping we get more examples
> > and start showing compelling implementation support for the Web Audio API
> > nodes that are already added to the X3D 4.0 Sound Component.  Huge but
> > unrealized potential is already available to us...
> >
> >
> >    -  Spatial Sound in X3DOM with Web Audio API
> >    - https://medialab.hmu.gr/minipages/x3domAudio
> >
> >
> >
> >    - X3D Example Archives: X3D4AM, X3D for Advanced Modeling, Audio
> >    Spatial Sound
> >    -
> >    https://www.web3d.org/x3d/content/examples/X3dForAdvancedModeling/AudioSpatialSound/
> >
> >
> > Again thanks for pushing on the limits, hope these potential paths forward
> > can help us focus further progress.
> >
> >
> > all the best, Don
> >
> > --
> >
> > Don Brutzman  Naval Postgraduate School, Code USW/Br
> > brutzman at nps.edu
> >
> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> > +1.831.656.2149
> >
> > X3D graphics, virtual worlds, navy robotics
> > https://faculty.nps.edu/brutzman
> >
> >
> >
> > ------------------------------
> > *From:* GPU Group <gpugroup at gmail.com>
> > *Sent:* Thursday, August 22, 2024 11:20 AM
> > *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> > *Cc:* Extensible 3D (X3D) Graphics public discussion <x3d-public at web3d.org
> > >
> > *Subject:* Re: [x3d-public] MIDI suggestions
> >  .
> > Don,
> > The current Sound component doesn't have a way to look at each MIDI note
> > as an event, nor to create MIDI notes with events during scene execution.
> > See this proposed separate 43 MIDI Component which allows that.
> > https://freewrl.sourceforge.io/tests/43_MIDI/
> > freewrl.sourceforge.io/tests/43_MIDI/NOTES_midi.txt
> > -Doug
> >
> > On Thu, Aug 22, 2024 at 11:16?AM Brutzman, Donald (Don) (CIV) <
> > brutzman at nps.edu> wrote:
> >
> > Doug, thanks for your suggestions.
> >
> > Regarding MIDI support:
> >
> >
> >    - *X3D 4.0 Architecture, 16 Sound component, 16.4.2 AudioClip*
> >    -
> >    https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioClip
> >    - The *url* field specifies the URL from which the sound file is
> >    loaded. 16.2.6 Audio encoding formats
> >    <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats> describes
> >    required and recommended file format support. 9.2.1 URLs, URNs and URIs
> >    <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/networking.html#URLs> contains
> >    details on the *url* field.
> >
> >
> >
> >    - *X3D 4.0 Architecture, 16 Sound component, 16.2.6 Audio encoding
> >    formats*
> >    -
> >    https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats
> >
> > X3D browsers shall support at least the *wavefile* format in uncompressed
> > PCM format (see [WAV]
> > <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/bibliography.html#WAV>
> > ).
> > It is recommended that X3D browsers support the MIDI file type 1 sound
> > format (see [MIDI 1.0]
> > <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-1>).
> > MIDI files are presumed to use the General MIDI patch set.
> > X3D browser support is also recommended for MIDI 2.0 (see [MIDI 2.0]
> > <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-2> and
> > Web MIDI API (see [W3C-WebMIDI]
> > <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#W3C-WebMIDI>
> > ).
> > It is also recommended that X3D browsers support the following formats:
> >
> >    - Format=MP3, container=MP3, compressed (see [I11172-1]
> >    <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#I11172_1>
> >    ).
> >    - Format=AAC, container=MP4 (see [14496-14]
> >    <https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#14496_14>
> >    ).
> >
> > Example X3Dscene with MIDI content:
> >
> >    - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
> >    Appearance Material Textures, Pixel Texture Snowman
> >    -
> >    https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanIndex.html
> >    -
> >    https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
> >    -
> >    https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid
> >
> >
> > Regarding MIDI content in ASCII form, it is certainly not disallowed in
> > the X3D specification.   Might you please send some references and an
> > example?  Perhaps an ASCII plaintext version of frosty2.
> >
> > Have fun with X3D and MIDI!  8)
> >
> > all the best, Don
> >
> > --
> >
> > Don Brutzman  Naval Postgraduate School, Code USW/Br
> > brutzman at nps.edu
> >
> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> > +1.831.656.2149
> >
> > X3D graphics, virtual worlds, navy robotics
> > https://faculty.nps.edu/brutzman
> >
> >
> >
> > ------------------------------
> > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU Group
> > via x3d-public <x3d-public at web3d.org>
> > *Sent:* Wednesday, August 21, 2024 3:14 PM
> > *To:* Extensible 3D (X3D) Graphics public discussion <x3d-public at web3d.org
> > >
> > *Cc:* GPU Group <gpugroup at gmail.com>
> > *Subject:* Re: [x3d-public] Web3D 2024: HAnim Music Animation competition
> > - Win cash prizes!
> >
> > MIDI > at least one web3d browser supports MIDI with nodes that give
> > events, which could be used to synchronize HAnim motions to music tones /
> > beats, or vice versa.
> > Something missing from prior work: a node that holds MIDI directly in
> > ASCII format, for easy tinkering in X3D scene file.
> > -Doug
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240823/e0f1181d/attachment.html>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
>
> ------------------------------
>
> End of x3d-public Digest, Vol 185, Issue 42
> *******************************************



-- 
Andreas Plesch
Waltham, MA 02453



More information about the x3d-public mailing list