[x3d-public] MIDI suggestions (GPU Group)

John Carlson yottzumm at gmail.com
Sat Aug 24 01:24:07 PDT 2024 

Indeed, a MIDI SF field and MF field seems useful, if not covered by other
fields or nodes.   Perhaps an existing node or proto node could do.  I
would probably handle 100 SFBools and an MFBool plus SFInt32 length, but
checkable so not as to create buffer overruns.  I understand that fixed
size nodes are desirable, maybe an MFBool could be implemented like
MFString.

On Fri, Aug 23, 2024 at 3:41 PM GPU Group via x3d-public <
x3d-public at web3d.org> wrote:

> Very interesting Web MIDI API. I think the next move is up to the html
> web3d browser gurus.
> -Doug
> In native C browser freewrl I made toneSplitter and toneMerger node types.
> The toneSplitter would filter by octave and had out field for each note C
> Cs D Ds E F Fs G Gs A As B so if you want to animate something when it hits
> a certain note, you can catch the note, and in that way MIDI becomes a
> general event sequencing method. I had a player-piano demo scene
> that animated and played a note from a USB hardware midi keyboard. And
> going the other way, a playable-piano scene where I could mouse-click a key
> on a scene keyboard and it would route it to ToneMerger node and get to the
> output MIDI port.
> But to get between ports and the rendering thread I used threads and
> message queues.
> Q. for Web MIDI - is there a way to do something like the
> ToneSplitter/ToneMerger without needing threading or extra message queues?
>
> On Fri, Aug 23, 2024 at 12:13 PM Andreas Plesch via x3d-public <
> x3d-public at web3d.org> wrote:
>
>> I was curious and found there is a Web MIDI API:
>> https://developer.mozilla.org/en-US/docs/Web/API/Web_MIDI_API
>> which is supported on Browsers other than Safari.
>> There is also a support library: https://webmidijs.org/
>> This is focused on connected external input devices like a keyboard,
>> not necessarily on playing midi files.
>> There is a midi file player: https://cifkao.github.io/html-midi-player/
>> which uses https://github.com/magenta/magenta-js/tree/master/music/
>> which does its own TensorFlow(!) based tone generation.
>> I also found
>> https://surikov.github.io/webaudiofont/examples/midiplayer.html
>> which seems capable.
>> -Andreas
>>
>> On Fri, Aug 23, 2024 at 12:16 PM <x3d-public-request at web3d.org> wrote:
>> >
>> > Send x3d-public mailing list submissions to
>> >         x3d-public at web3d.org
>> >
>> > To subscribe or unsubscribe via the World Wide Web, visit
>> >         http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> > or, via email, send a message with subject or body 'help' to
>> >         x3d-public-request at web3d.org
>> >
>> > You can reach the person managing the list at
>> >         x3d-public-owner at web3d.org
>> >
>> > When replying, please edit your Subject line so it is more specific
>> > than "Re: Contents of x3d-public digest..."
>> >
>> >
>> > Today's Topics:
>> >
>> >    1. Re: MIDI suggestions (GPU Group)
>> >
>> >
>> > ----------------------------------------------------------------------
>> >
>> > Message: 1
>> > Date: Fri, 23 Aug 2024 10:15:13 -0600
>> > From: GPU Group <gpugroup at gmail.com>
>> > To: "Extensible 3D (X3D) Graphics public discussion"
>> >         <x3d-public at web3d.org>
>> > Subject: Re: [x3d-public] MIDI suggestions
>> > Message-ID:
>> >         <CAM2ogRfZ8=+3K+pHdu2DiNLRB=
>> A3WJyzoesn+UYUcPu_PYaHBg at mail.gmail.com>
>> > Content-Type: text/plain; charset="utf-8"
>> >
>> > Don,
>> > I used MIDI Editor but just briefly, didn't master it.
>> > PROTOS > General Sequencing Node - something that sequences similar to
>> MIDI
>> > that could be done in a PROTO is a general sequencing node, which could
>> > have the benefits of a text/string field for in-scene hacking and
>> multiple
>> > output fields like the MIDI music octave style node type.
>> > GeneralSequencer {
>> > SFString in/out sequence
>> > SFTime in start
>> > SFBool out o1
>> > SFBool out o2
>> > SFBool out o3
>> > ...
>> > SFBool out o100
>> > }
>> > example
>> > DEF GS1 GeneralSequencer {
>> > # out field number, out field value, time offset from start
>> > sequence "
>> > 1 T 0.0
>> > 1 F 0.25
>> > 1 T 1.0
>> > 2 T 1.0
>> > 3 T 1.0
>> > 4 T 1.0
>> > "
>> > # or time offset from start, out value, list of fields
>> > "
>> > 0.0 T 1,
>> > .25 F 1,
>> > 1.0 T 1 2 3 4,
>> > "
>> > or something like that
>> > }
>> > The general sequencer could be convenient for naval simulations -with
>> the
>> > string sequence being replaced according to different attack or
>> submersible
>> > robot maneuvers.
>> > And convenient for musical HANIM sequences to synchronize musical riff
>> play
>> > --using standard sound nodes-- and certain humanoid articulations.
>> > On each frame the Proto code would check if there are any in the
>> sequence
>> > that have (start + offset) time > current time and haven't been played,
>> and
>> > play them.
>> >
>> > -Doug
>> >
>> > On Fri, Aug 23, 2024 at 8:54?AM Brutzman, Donald (Don) (CIV) <
>> > brutzman at nps.edu> wrote:
>> >
>> > > OK very good, thanks for information Doug.
>> > >
>> > > Meanwhile, am hoping to improve the MIDI files that we have, and
>> > > potentially add more examples.  Wondering if there is a good
>> open-source
>> > > software tool for MIDI?
>> > >
>> > > I looked again within Audacity, which is an excellent open source
>> tool.
>> > > It does support MIDI but does not seem to give any information about
>> what
>> > > type of MIDI file is loaded or exported.  Some vulnerabilities are
>> noted in
>> > > NIST National Vulnerability Database but they are all on very old
>> versions,
>> > > so it seems trustworthy.  Linked for installation and easy use within
>> > > X3D-Edit as a helper application.
>> > >
>> > >
>> > >    - https://www.audacityteam.org
>> > >    -
>> > >
>> https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=Audacity
>> > >
>> > >
>> > > Previous I was using MuseScore every now and then.  However an upgrade
>> > > earlier this year integrated some kind of p2p streaming activity (aka
>> Muse
>> > > Hub) that triggered a variety of security warnings, so I stopped
>> using it.
>> > > However, I now see that they offer a version without Muse Hub, will
>> try
>> > > again.
>> > >
>> > >
>> > >    - https://musescore.org/en
>> > >    -
>> > >
>> https://nvd.nist.gov/vuln/search/results?form_type=Basic&results_type=overview&search_type=all&query=musescore
>> > >
>> > >
>> > > The midieditor looks possible but it is free software (hobbyist) and
>> not
>> > > open source, so less appealing.  Also does not have an https server,
>> which
>> > > is a red flag.
>> > >
>> > >
>> > >    - http://www.midieditor.org
>> > >
>> > >
>> > > Am hoping to test binary and text-based MIDI (and possibly MIDI
>> version 1
>> > > and 2) using different versions of the example file.
>> > >
>> > >
>> > > Wondering if there are any open-source audio tools you can recommend
>> that
>> > > support
>> > >
>> > >
>> > >    - Identifying MIDI version within a file,
>> > >    - Exporting between different MIDI formats
>> > >
>> > >
>> > > all the best, Don
>> > >
>> > > --
>> > >
>> > > Don Brutzman  Naval Postgraduate School, Code USW/Br
>> > > brutzman at nps.edu
>> > >
>> > > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>> > > +1.831.656.2149
>> > >
>> > > X3D graphics, virtual worlds, navy robotics
>> > > https://faculty.nps.edu/brutzman
>> > >
>> > >
>> > > ------------------------------
>> > > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU
>> Group
>> > > via x3d-public <x3d-public at web3d.org>
>> > > *Sent:* Friday, August 23, 2024 7:18 AM
>> > > *Cc:* GPU Group <gpugroup at gmail.com>; Extensible 3D (X3D) Graphics
>> public
>> > > discussion <x3d-public at web3d.org>
>> > > *Subject:* Re: [x3d-public] MIDI suggestions
>> > >
>> > > Don,
>> > > MIDI via PROTOs > working in C I used threads and queues between the
>> MIDI
>> > > port and the x3d rendering thread, and I don't know how to do that
>> with
>> > > PROTOs. But it might be different with Web MIDI API. So one idea is
>> to wait
>> > > until/in-case one of the web based x3d browsers implements a MIDI
>> component
>> > > using PROTOs, and see how they did that, and then the native browsers
>> can
>> > > adapt to that.
>> > > -Doug
>> > >
>> > > On Thu, Aug 22, 2024 at 6:35?PM Brutzman, Donald (Don) (CIV) <
>> > > brutzman at nps.edu> wrote:
>> > >
>> > > Understood.  The work you describe regarding MIDI nodes looks quite
>> > > involved.  Possible component extension if someone did the work
>> involved.
>> > > Implementing as Prototype library using Script nodes might work.
>> Writing
>> > > extensions for X_ITE or X3DOM might work.  In each case that would be
>> > > proof-of-capability and maybe not a path to wide availability.
>> > >
>> > > I think it is doubtful that MIDI editing would be accepted as a
>> high-value
>> > > use case and recommended practice for X3D 4.1, which might take an
>> > > undetermined length of time anyway.  Seems like a long, unlikely and
>> > > unsatisfying path to pursue.  Nevertheless possible.
>> > >
>> > > I looked in Audacity briefly for "saving as" MIDI ASCII but didn't
>> find
>> > > it.  Perhaps I missed it or another tool is useful.
>> > >
>> > > If we do have a text-base example MIDI file archived, that might
>> encourage
>> > > use of other online editors.  Note how X_ITE is already able to show
>> XML,
>> > > ClassicVRML and JSON encodings equivalently (see links in upper-right
>> > > corner).
>> > >
>> > >
>> > >    - X_ITE Playground Editor
>> > >    - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
>> > >    Appearance Material Textures, Pixel Texture Snowman
>> > >    -
>> > >
>> https://create3000.github.io/x_ite/playground/?url=https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
>> > >
>> > >
>> > >
>> > >    - ... but has error:  Couldn't load URL '
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid'.EncodingError
>> :
>> > >    Failed to execute 'decodeAudioData' on 'BaseAudioContext': Unable
>> to decode
>> > >    audio data
>> > >
>> > >
>> > >
>> > >    - X3DOM similarly seems to have a problem paying the midi file
>> > >    -
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanX3dom.xhtml
>> > >
>> > >
>> > >
>> > >    - Nothing heard using latest FreeWrl
>> > >    - Nothing heard using current Castle Model Viewer
>> > >
>> > >
>> > > Audacity plays the .midi file OK.  Might there be a problem with the
>> file?
>> > >
>> > > Regarding sound-related implementations, am hoping we get more
>> examples
>> > > and start showing compelling implementation support for the Web Audio
>> API
>> > > nodes that are already added to the X3D 4.0 Sound Component.  Huge but
>> > > unrealized potential is already available to us...
>> > >
>> > >
>> > >    -  Spatial Sound in X3DOM with Web Audio API
>> > >    - https://medialab.hmu.gr/minipages/x3domAudio
>> > >
>> > >
>> > >
>> > >    - X3D Example Archives: X3D4AM, X3D for Advanced Modeling, Audio
>> > >    Spatial Sound
>> > >    -
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForAdvancedModeling/AudioSpatialSound/
>> > >
>> > >
>> > > Again thanks for pushing on the limits, hope these potential paths
>> forward
>> > > can help us focus further progress.
>> > >
>> > >
>> > > all the best, Don
>> > >
>> > > --
>> > >
>> > > Don Brutzman  Naval Postgraduate School, Code USW/Br
>> > > brutzman at nps.edu
>> > >
>> > > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>> > > +1.831.656.2149
>> > >
>> > > X3D graphics, virtual worlds, navy robotics
>> > > https://faculty.nps.edu/brutzman
>> > >
>> > >
>> > >
>> > > ------------------------------
>> > > *From:* GPU Group <gpugroup at gmail.com>
>> > > *Sent:* Thursday, August 22, 2024 11:20 AM
>> > > *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> > > *Cc:* Extensible 3D (X3D) Graphics public discussion <
>> x3d-public at web3d.org
>> > > >
>> > > *Subject:* Re: [x3d-public] MIDI suggestions
>> > >  .
>> > > Don,
>> > > The current Sound component doesn't have a way to look at each MIDI
>> note
>> > > as an event, nor to create MIDI notes with events during scene
>> execution.
>> > > See this proposed separate 43 MIDI Component which allows that.
>> > > https://freewrl.sourceforge.io/tests/43_MIDI/
>> > > freewrl.sourceforge.io/tests/43_MIDI/NOTES_midi.txt
>> > > -Doug
>> > >
>> > > On Thu, Aug 22, 2024 at 11:16?AM Brutzman, Donald (Don) (CIV) <
>> > > brutzman at nps.edu> wrote:
>> > >
>> > > Doug, thanks for your suggestions.
>> > >
>> > > Regarding MIDI support:
>> > >
>> > >
>> > >    - *X3D 4.0 Architecture, 16 Sound component, 16.4.2 AudioClip*
>> > >    -
>> > >
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioClip
>> > >    - The *url* field specifies the URL from which the sound file is
>> > >    loaded. 16.2.6 Audio encoding formats
>> > >    <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats>
>> describes
>> > >    required and recommended file format support. 9.2.1 URLs, URNs and
>> URIs
>> > >    <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/networking.html#URLs>
>> contains
>> > >    details on the *url* field.
>> > >
>> > >
>> > >
>> > >    - *X3D 4.0 Architecture, 16 Sound component, 16.2.6 Audio encoding
>> > >    formats*
>> > >    -
>> > >
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/components/sound.html#AudioEncodingFormats
>> > >
>> > > X3D browsers shall support at least the *wavefile* format in
>> uncompressed
>> > > PCM format (see [WAV]
>> > > <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/bibliography.html#WAV
>> >
>> > > ).
>> > > It is recommended that X3D browsers support the MIDI file type 1 sound
>> > > format (see [MIDI 1.0]
>> > > <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-1
>> >).
>> > > MIDI files are presumed to use the General MIDI patch set.
>> > > X3D browser support is also recommended for MIDI 2.0 (see [MIDI 2.0]
>> > > <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#MIDI-2>
>> and
>> > > Web MIDI API (see [W3C-WebMIDI]
>> > > <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#W3C-WebMIDI
>> >
>> > > ).
>> > > It is also recommended that X3D browsers support the following
>> formats:
>> > >
>> > >    - Format=MP3, container=MP3, compressed (see [I11172-1]
>> > >    <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#I11172_1
>> >
>> > >    ).
>> > >    - Format=AAC, container=MP4 (see [14496-14]
>> > >    <
>> https://www.web3d.org/specifications/X3Dv4/ISO-IEC19775-1v4-IS/Part01/references.html#14496_14
>> >
>> > >    ).
>> > >
>> > > Example X3Dscene with MIDI content:
>> > >
>> > >    - X3D Example Archives: X3D4WA, X3D for Web Authors, Chapter 05
>> > >    Appearance Material Textures, Pixel Texture Snowman
>> > >    -
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowmanIndex.html
>> > >    -
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/PixelTextureSnowman.x3d
>> > >    -
>> > >
>> https://www.web3d.org/x3d/content/examples/X3dForWebAuthors/Chapter05AppearanceMaterialTextures/frosty2.mid
>> > >
>> > >
>> > > Regarding MIDI content in ASCII form, it is certainly not disallowed
>> in
>> > > the X3D specification.   Might you please send some references and an
>> > > example?  Perhaps an ASCII plaintext version of frosty2.
>> > >
>> > > Have fun with X3D and MIDI!  8)
>> > >
>> > > all the best, Don
>> > >
>> > > --
>> > >
>> > > Don Brutzman  Naval Postgraduate School, Code USW/Br
>> > > brutzman at nps.edu
>> > >
>> > > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
>> > > +1.831.656.2149
>> > >
>> > > X3D graphics, virtual worlds, navy robotics
>> > > https://faculty.nps.edu/brutzman
>> > >
>> > >
>> > >
>> > > ------------------------------
>> > > *From:* x3d-public <x3d-public-bounces at web3d.org> on behalf of GPU
>> Group
>> > > via x3d-public <x3d-public at web3d.org>
>> > > *Sent:* Wednesday, August 21, 2024 3:14 PM
>> > > *To:* Extensible 3D (X3D) Graphics public discussion <
>> x3d-public at web3d.org
>> > > >
>> > > *Cc:* GPU Group <gpugroup at gmail.com>
>> > > *Subject:* Re: [x3d-public] Web3D 2024: HAnim Music Animation
>> competition
>> > > - Win cash prizes!
>> > >
>> > > MIDI > at least one web3d browser supports MIDI with nodes that give
>> > > events, which could be used to synchronize HAnim motions to music
>> tones /
>> > > beats, or vice versa.
>> > > Something missing from prior work: a node that holds MIDI directly in
>> > > ASCII format, for easy tinkering in X3D scene file.
>> > > -Doug
>> > >
>> > >
>> > -------------- next part --------------
>> > An HTML attachment was scrubbed...
>> > URL: <
>> http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240823/e0f1181d/attachment.html
>> >
>> >
>> > ------------------------------
>> >
>> > Subject: Digest Footer
>> >
>> > _______________________________________________
>> > x3d-public mailing list
>> > x3d-public at web3d.org
>> > http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> >
>> >
>> > ------------------------------
>> >
>> > End of x3d-public Digest, Vol 185, Issue 42
>> > *******************************************
>>
>>
>>
>> --
>> Andreas Plesch
>> Waltham, MA 02453
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20240824/25b100b9/attachment-0001.html>

More information about the x3d-public mailing list