[X3D-Public] Fwd: HTML and Multimedia Synchronization

Sun Jan 6 11:29:36 PST 2013

Interesting dialog...

Sent from my handheld device

Begin forwarded message:

Resent-From: <public-html at w3.org<mailto:public-html at w3.org>>
From: Silvia Pfeiffer <silviapfeiffer1 at gmail.com<mailto:silviapfeiffer1 at gmail.com>>
Date: January 5, 2013, 18:43:54 PST
To: Adam Sobieski <adamsobieski at hotmail.com<mailto:adamsobieski at hotmail.com>>
Cc: "public-html at w3.org<mailto:public-html at w3.org>" <public-html at w3.org<mailto:public-html at w3.org>>
Subject: Re: HTML and Multimedia Synchronization

Hi Adam,

On Sun, Jan 6, 2013 at 2:57 AM, Adam Sobieski <adamsobieski at hotmail.com<mailto:adamsobieski at hotmail.com>> wrote:
Silvia,

I have implemented mathematics documents, document pages and portions of digital mathematics books and textbooks, including based upon pages and portions of existing documents, AMS documents, math books, and textbooks, including while looking at specific features and features in combination, for example CSS columns and MathML3 line wrapping (https://groups.google.com/forum/#!msg/mozilla.dev.platform/KCIMqFwVaIY/bkIfgjPERo0J).

On the topic of scholarly and scientific communication, an interesting use case is mathematics documents with device-specific formatting (http://www.w3.org/TR/css3-mediaqueries/), for example columnization (http://www.w3.org/TR/css3-multicol/) and widescreen displays, that print styled as per AMS document style (ftp://ftp.ams.org/pub/author-info/documentation/handbk.pdf).

I participate in some discussion groups specifically interested in digital STEM books and textbooks and, more specifically, digital mathematics books and textbooks; I could collect together their observations and invite them and other participants here to provide feedback upcoming;

That would be valuable.

a digital textbook team indicated to me, for example, an interest in the speech recognition of mathematics. Some more topics will be available upcoming with some available on the 15th when papers from the upcoming W3C digital publishing workshop will be available online.

Are you aware that the Speech API CG has developed a proposed specification for speech recognition and synthesis: http://www.w3.org/community/speech-api/ and http://dvcs.w3.org/hg/speech-api/raw-file/9a0075d25326/speechapi.html ? I am not sure about its status and whether it is being adopted into the work of a W3C WG. Is your suggestion to adopt it as a piece of work for this Working Group?

Some HTML and multimedia synchronization techniques are possible with with HTML, XHTML, and JavaScript. We can observe the timesheets.js project (http://wam.inrialpes.fr/timesheets/), for example, with internal timesheets (http://wam.inrialpes.fr/timesheets/markup/intTimesheet.xhtml), external timesheets (http://wam.inrialpes.fr/timesheets/markup/extTimesheet.html), smil-* attributes (http://wam.inrialpes.fr/timesheets/markup/smilPrefix.html), data-* attributes (http://wam.inrialpes.fr/timesheets/markup/dataPrefix.html), smil:* attributes (http://wam.inrialpes.fr/timesheets/markup/smilNS.xhtml), and inline SMIL (http://wam.inrialpes.fr/timesheets/markup/noPrefix.html).

Enhancing the Web-based and desktop-based indexing, searching, and retrieving of RDFa-enhanced hypertext-based time-based multimedia documents or configurations of such documents, or bookmarks, can become topical.

Indexing, search and retrieval are applications and don't need improvements to the HTML specification. The W3C Media Fragments WG also has developed specifications that allow the creation of hyperlinks into time-based multimedia documents, see http://www.w3.org/TR/media-frags/ . As URLs can be stored as bookmarks, these use cases are already possible.

Standardizing HTML and multimedia synchronization, for example with HTML 5.1, a new HTML+SMIL document (http://www.w3.org/TR/2000/WD-smil-boston-20000622/html-smil-profile.html), and/or a SMIL 4 specification, can enhance the indexing, searching and retrieving of, the searchability of, content with multimedia synchronization features. Some topics in multimedia synchronization include speech synthesis and 3D graphics.

3D graphics are indeed interesting. There is a whole consortium looking at 3D graphics on the Web: http://www.web3d.org/realtime-3d/ and X3D files can be used in browser applets. Are you suggesting we should make it more of a native Web format? Have browser vendors expressed interest in implementing 3D graphics support natively?

EPUB Media Overlays 3.0 (http://idpf.org/epub/30/spec/epub30-mediaoverlays.html) describes multimedia synchronization, specifically media overlays, with regard to the EPUB3 format.

I am not convinced that Web pages need media overlays. We have many years of experience in making Web pages accessible by proving text alternatives for certain pieces of content and by having AT communicate either existing text on a page or provided text alternatives to a11y users. Media overlays make sense for books where there is a linear reading experience. Web pages, and more so modern Web applications don't generally follow a linear reading experience and thus text and text alternatives that are associated with user's focus seem more adequate. I would, however, suggest discussing such needs with WAI.

Best Regards,
Silvia.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20130106/58b5fc64/attachment.html>