[x3d-public] MIDI suggestions (GPU Group)
GPU Group
gpugroup at gmail.com
Fri Aug 23 13:40:18 PDT 2024
Very interesting Web MIDI API. I think the next move is up to the html
web3d browser gurus.
-Doug
In native C browser freewrl I made toneSplitter and toneMerger node types.
The toneSplitter would filter by octave and had out field for each note C
Cs D Ds E F Fs G Gs A As B so if you want to animate something when it hits
a certain note, you can catch the note, and in that way MIDI becomes a
general event sequencing method. I had a player-piano demo scene
that animated and played a note from a USB hardware midi keyboard. And
going the other way, a playable-piano scene where I could mouse-click a key
on a scene keyboard and it would route it to ToneMerger node and get to the
output MIDI port.
But to get between ports and the rendering thread I used threads and
message queues.
Q. for Web MIDI - is there a way to do something like the
ToneSplitter/ToneMerger without needing threading or extra message queues?
On Fri, Aug 23, 2024 at 12:13 PM Andreas Plesch via x3d-public <
x3d-public at web3d.org> wrote:
> I was curious and found there is a Web MIDI API:
> https://developer.mozilla.org/en-US/docs/Web/API/Web_MIDI_API
> which is supported on Browsers other than Safari.
> There is also a support library: https://webmidijs.org/
> This is focused on connected external input devices like a keyboard,
> not necessarily on playing midi files.
> There is a midi file player: https://cifkao.github.io/html-midi-player/
> which uses https://github.com/magenta/magenta-js/tree/master/music/
> which does its own TensorFlow(!) based tone generation.
> I also found
> https://surikov.github.io/webaudiofont/examples/midiplayer.html
> which seems capable.
> -Andreas
>
> On Fri, Aug 23, 2024 at 12:16 PM <x3d-public-request at web3d.org> wrote:
> >
> > Send x3d-public mailing list submissions to
> > x3d-public at web3d.org
> >
> > To subscribe or unsubscribe via the World Wide Web, visit
> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> > or, via email, send a message with subject or body 'help' to
> > x3d-public-request at web3d.org
> >
> > You can reach the person managing the list at
> > x3d-public-owner at web3d.org
> >
> > When replying, please edit your Subject line so it is more specific
> > than "Re: Contents of x3d-public digest..."
> >
> >
> > Today's Topics:
> >
> > 1. Re: MIDI suggestions (GPU Group)
> >
> >
> > ----------------------------------------------------------------------
> >
> > Message: 1
> > Date: Fri, 23 Aug 2024 10:15:13 -0600
> > From: GPU Group <gpugroup at gmail.com>
> > To: "Extensible 3D (X3D) Graphics public discussion"
> > <x3d-public at web3d.org>
> > Subject: Re: [x3d-public] MIDI suggestions
> > Message-ID:
> > <CAM2ogRfZ8=+3K+pHdu2DiNLRB=
> A3WJyzoesn+UYUcPu_PYaHBg at mail.gmail.com>
> > Content-Type: text/plain; charset="utf-8"
> >
> > Don,
> > I used MIDI Editor but just briefly, didn't master it.
> > PROTOS > General Sequencing Node - something that sequences similar to
> MIDI
> > that could be done in a PROTO is a general sequencing node, which could
> > have the benefits of a text/string field for in-scene hacking and
> multiple
> > output fields like the MIDI music octave style node type.
> > GeneralSequencer {
> > SFString in/out sequence
> > SFTime in start
> > SFBool out o1
> > SFBool out o2
> > SFBool out o3
> > ...
> > SFBool out o100
> > }
> > example
> > DEF GS1 GeneralSequencer {
> > # out field number, out field value, time offset from start
> > sequence "
> > 1 T 0.0
> > 1 F 0.25
> > 1 T 1.0
> > 2 T 1.0
> > 3 T 1.0
> > 4 T 1.0
> > "
> > # or time offset from start, out value, list of fields
> > "
> > 0.0 T 1,
> > .25 F 1,
> > 1.0 T 1 2 3 4,
> > "
> > or something like that
> > }
> > The general sequencer could be convenient for naval simulations -with the
> > string sequence being replaced according to different attack or
> submersible
> > robot maneuvers.
> > And convenient for musical HANIM sequences to synchronize musical riff
> play
> > --using standard sound nodes-- and certain humanoid articulations.
> > On each frame the Proto code would check if there are any in the sequence
> > that have (start + offset) time > current time and haven't been played,
> and
> > play them.
> >
> > -Doug
> >
> > On Fri, Aug 23, 2024 at 8:54?AM Brutzman, Donald (Don) (CIV) <
> > brutzman at nps.edu> wrote:
> >
> > > OK very good, thanks for information Doug.
> > >
> > > Meanwhile, am hoping to improve the MIDI files that we have, and
> > > potentially add more examples. Wondering if there is a good
> open-source
> > > software tool for MIDI?
> > >
> > > I looked again within Audacity, which is an excellent open source tool.
> > > It does support MIDI but does not seem to give any information about
> what
> > > type of MIDI file is loaded or exported. Some vulnerabilities are
> noted in
> > > NIST National Vulnerability Database but they are all on very old
> versions,
> > > so it seems trustworthy. Linked for installation and easy use within
> > > X3D-Edit as a helper application.
> > >
> > >
> > > - https://www.audacityteam.org
> > > -
> > >
> https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=Audacity
> > >
> > >
> > > Previous I was using MuseScore every now and then. However an upgrade
> > > earlier this year integrated some kind of p2p streaming activity (aka
> Muse
> > > Hub) that triggered a variety of security warnings, so I stopped using
> it.
> > > However, I now see that they offer a version without Muse Hub, will try
> > > again.
> > >
> > >
> > > - https://musescore.org/en
> > > -
> > >
> https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=musescore
> > >
> > >
> > > The midieditor looks possible but it is free software (hobbyist) and
> not
> > > open source, so less appealing. Also does not have an https server,
> which
> > > is a red flag.
> > >
> > >
> > > - http://www.midieditor.org
> > >
> > >
> > > Am hoping to test binary and text-based MIDI (and possibly MIDI
> version 1
> > > and 2) using different versions of the example file.
> > >
> > >
> > > Wondering if there are any open-source audio tools you can recommend
> that
> > > support
> > >
> > >
> > > - Identifying MIDI version within a file,
> > > - Exporting between different MIDI formats
> > >
> > >
> > > all the best, Don
> > >
> > > --
> > >
> > > Don Brutzman Naval Postgraduate School, Code USW/Br
> > > brutzman at nps.edu
> > >
> > > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
> > > +1.831.656.2149
> > >
> > > X3D graphics, virtual worlds, navy robotics
> > > https://faculty.nps.edu/brutzman
> > >
> > >
> > > ------------------------------
> > > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU
> Group
> > > via x3d-public <x3d-public at web3d.org>
> > > *Sent:* Friday, August 23, 2024 7:18 AM
> > > *Cc:* GPU Group <gpugroup at gmail.com>; Extensible 3D (X3D) Graphics
> public
> > > discussion <x3d-public at web3d.org>
> > > *Subject:* Re: [x3d-public] MIDI suggestions
> > >
> > > Don,
> > > MIDI via PROTOs > working in C I used threads and queues between the
> MIDI
> > > port and the x3d rendering thread, and I don't know how to do that with
> > > PROTOs. But it might be different with Web MIDI API. So one idea is to
> wait
> > > until/in-case one of the web based x3d browsers implements a MIDI
> component
> > > using PROTOs, and see how they did that, and then the native browsers
> can
> > > adapt to that.
> > > -Doug
> > >
> > > On Thu, Aug 22, 2024 at 6:35?PM Brutzman, Donald (Don) (CIV) <
> > > brutzman at nps.edu> wrote:
> > >
> > > Understood. The work you describe regarding MIDI nodes looks quite
> > > involved. Possible component extension if someone did the work
> involved.
> > > Implementing as Prototype library using Script nodes might work.
> Writing
> > > extensions for X_ITE or X3DOM might work. In each case that would be
> > > proof-of-capability and maybe not a path to wide availability.
> > >
> > > I think it is doubtful that MIDI editing would be accepted as a
> high-value
> > > use case and recommended practice for X3D 4.1, which might take an
> > > undetermined length of time anyway. Seems like a long, unlikely and
> > > unsatisfying path to pursue. Nevertheless possible.
> > >
> > > I looked in Audacity briefly for "saving as" MIDI ASCII but didn't find
> > > it. Perhaps I missed it or another tool is useful.
> > >
> > > If we do have a text-base example MIDI file archived, that might
> encourage
> > > use of other online editors. Note how X_ITE is already able to show
> XML,
> > > ClassicVRML and JSON encodings equivalently (see links in upper-right
> > > corner).
> > >
> > >
> > > - X_ITE Playground Editor
> > > - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
> > > Appearance Material Textures, Pixel Texture Snowman
> > > -
> > >
> https://create3000.github.io/x_ite/playground/?url=https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
> > >
> > >
> > >
> > > - ... but has error: Couldn't load URL '
> > >
> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid'.EncodingError
> :
> > > Failed to execute 'decodeAudioData' on 'BaseAudioContext': Unable
> to decode
> > > audio data
> > >
> > >
> > >
> > > - X3DOM similarly seems to have a problem paying the midi file
> > > -
> > >
> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanX3dom.xhtml
> > >
> > >
> > >
> > > - Nothing heard using latest FreeWrl
> > > - Nothing heard using current Castle Model Viewer
> > >
> > >
> > > Audacity plays the .midi file OK. Might there be a problem with the
> file?
> > >
> > > Regarding sound-related implementations, am hoping we get more examples
> > > and start showing compelling implementation support for the Web Audio
> API
> > > nodes that are already added to the X3D 4.0 Sound Component. Huge but
> > > unrealized potential is already available to us...
> > >
> > >
> > > - Spatial Sound in X3DOM with Web Audio API
> > > - https://medialab.hmu.gr/minipages/x3domAudio
> > >
> > >
> > >
> > > - X3D Example Archives: X3D4AM, X3D for Advanced Modeling, Audio
> > > Spatial Sound
> > > -
> > >
> https://www.web3d.org/x3d/content/examples/X3dForAdvancedModeling/AudioSpatialSound/
> > >
> > >
> > > Again thanks for pushing on the limits, hope these potential paths
> forward
> > > can help us focus further progress.
> > >
> > >
> > > all the best, Don
> > >
> > > --
> > >
> > > Don Brutzman Naval Postgraduate School, Code USW/Br
> > > brutzman at nps.edu
> > >
> > > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
> > > +1.831.656.2149
> > >
> > > X3D graphics, virtual worlds, navy robotics
> > > https://faculty.nps.edu/brutzman
> > >
> > >
> > >
> > > ------------------------------
> > > *From:* GPU Group <gpugroup at gmail.com>
> > > *Sent:* Thursday, August 22, 2024 11:20 AM
> > > *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> > > *Cc:* Extensible 3D (X3D) Graphics public discussion <
> x3d-public at web3d.org
> > > >
> > > *Subject:* Re: [x3d-public] MIDI suggestions
> > > .
> > > Don,
> > > The current Sound component doesn't have a way to look at each MIDI
> note
> > > as an event, nor to create MIDI notes with events during scene
> execution.
> > > See this proposed separate 43 MIDI Component which allows that.
> > > https://freewrl.sourceforge.io/tests/43_MIDI/
> > > freewrl.sourceforge.io/tests/43_MIDI/NOTES_midi.txt
> > > -Doug
> > >
> > > On Thu, Aug 22, 2024 at 11:16?AM Brutzman, Donald (Don) (CIV) <
> > > brutzman at nps.edu> wrote:
> > >
> > > Doug, thanks for your suggestions.
> > >
> > > Regarding MIDI support:
> > >
> > >
> > > - *X3D 4.0 Architecture, 16 Sound component, 16.4.2 AudioClip*
> > > -
> > >
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioClip
> > > - The *url* field specifies the URL from which the sound file is
> > > loaded. 16.2.6 Audio encoding formats
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats>
> describes
> > > required and recommended file format support. 9.2.1 URLs, URNs and
> URIs
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/networking.html#URLs>
> contains
> > > details on the *url* field.
> > >
> > >
> > >
> > > - *X3D 4.0 Architecture, 16 Sound component, 16.2.6 Audio encoding
> > > formats*
> > > -
> > >
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats
> > >
> > > X3D browsers shall support at least the *wavefile* format in
> uncompressed
> > > PCM format (see [WAV]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/bibliography.html#WAV
> >
> > > ).
> > > It is recommended that X3D browsers support the MIDI file type 1 sound
> > > format (see [MIDI 1.0]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-1
> >).
> > > MIDI files are presumed to use the General MIDI patch set.
> > > X3D browser support is also recommended for MIDI 2.0 (see [MIDI 2.0]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-2>
> and
> > > Web MIDI API (see [W3C-WebMIDI]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#W3C-WebMIDI
> >
> > > ).
> > > It is also recommended that X3D browsers support the following formats:
> > >
> > > - Format=MP3, container=MP3, compressed (see [I11172-1]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#I11172_1
> >
> > > ).
> > > - Format=AAC, container=MP4 (see [14496-14]
> > > <
> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#14496_14
> >
> > > ).
> > >
> > > Example X3Dscene with MIDI content:
> > >
> > > - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
> > > Appearance Material Textures, Pixel Texture Snowman
> > > -
> > >
> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanIndex.html
> > > -
> > >
> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
> > > -
> > >
> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid
> > >
> > >
> > > Regarding MIDI content in ASCII form, it is certainly not disallowed in
> > > the X3D specification. Might you please send some references and an
> > > example? Perhaps an ASCII plaintext version of frosty2.
> > >
> > > Have fun with X3D and MIDI! 8)
> > >
> > > all the best, Don
> > >
> > > --
> > >
> > > Don Brutzman Naval Postgraduate School, Code USW/Br
> > > brutzman at nps.edu
> > >
> > > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
> > > +1.831.656.2149
> > >
> > > X3D graphics, virtual worlds, navy robotics
> > > https://faculty.nps.edu/brutzman
> > >
> > >
> > >
> > > ------------------------------
> > > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU
> Group
> > > via x3d-public <x3d-public at web3d.org>
> > > *Sent:* Wednesday, August 21, 2024 3:14 PM
> > > *To:* Extensible 3D (X3D) Graphics public discussion <
> x3d-public at web3d.org
> > > >
> > > *Cc:* GPU Group <gpugroup at gmail.com>
> > > *Subject:* Re: [x3d-public] Web3D 2024: HAnim Music Animation
> competition
> > > - Win cash prizes!
> > >
> > > MIDI > at least one web3d browser supports MIDI with nodes that give
> > > events, which could be used to synchronize HAnim motions to music
> tones /
> > > beats, or vice versa.
> > > Something missing from prior work: a node that holds MIDI directly in
> > > ASCII format, for easy tinkering in X3D scene file.
> > > -Doug
> > >
> > >
> > -------------- next part --------------
> > An HTML attachment was scrubbed...
> > URL: <
> http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240823/e0f1181d/attachment.html
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > x3d-public mailing list
> > x3d-public at web3d.org
> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> >
> >
> > ------------------------------
> >
> > End of x3d-public Digest, Vol 185, Issue 42
> > *******************************************
>
>
>
> --
> Andreas Plesch
> Waltham, MA 02453
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240823/a91f73a4/attachment-0001.html>
More information about the x3d-public
mailing list