[x3d-public] The fundamental challenge left in computer science: unstructured, dangling media.

[Don Brutzman]

On 12/21/2015 10:55 PM, John Carlson wrote:
> Unstructured media: variable sized, new, hyper, dangling, broken, unfinished, context sensitive, made up, random, chaotic, ungrammatical, incorrect, paradoxical, and incomplete phonemes, words, sentences, grammars, and meta-grammars. Likewise with music, speech, drawings, points, vectors, surface, volumes, dimensions (geometry), painting, sculpture, pictures, video (pixels, voxels, time rate or size), taste, smell, motion, emotion, thought, haptics and feeling.
>
> If we can find a way to represent these in VRML/X3D/PostScript/PDF V4 or V5 or … I think we have a good chance of handling almost everything.
>
> Thoughts? What file formats or file structure are likely at this point?

Worthy questions indeed.

The X3D specifications can handle all manner of unstructured media, as follows:

a. url reference to each media resource, local or on the Web
b. Requiring or recommending support for certain formats
c. Allowing extensions, experimentation and customization

Any required format must be royalty free so that implementations and usage are unencumbered (just like HTML).

The X3D approach to your list of challenges includes efforts to preserve long-term stability of content, along with consistency of viewing & interaction by end users. These design fundamentals are likely to continue serving us well as we continue integration with the Web, and also continue integration with the real world as part of Mixed Augmented Reality (MAR) efforts.

The X3D Scene Authoring Hints include details about audio, image and movie formats.  They also cross-link to X3D specification sections.

	http://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#Audio
	http://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#Images (and Videos)

==============================
"X3D player support for the .wav format is required, while .midi and .mp3 support are recommended. Other audio formats are optional, you are welcome to check documentation for browsers of interest. So far, no streaming protocol is required to be supported in X3D players."
==============================
"Choice of image file formats:
     Portable network graphics (PNG) is the preferred image format for capturing screen images, providing the best support for lossless compression and transparency.
     Joint Photographic Experts Group (JPEG) format is also excellent. The preferred file extension is CoolImage.jpg. JPEG is the best choice for photographic images since it captures color gradations exceptionally well. Be careful to evaluate results when modifying JPEG compression quality, since it is lossy and highest image quality is often best. Note that JPEG does not support transparency.
     Tagged Image File Format (TIFF) is a high-quality format for images, photographs and line art. However it is proprietary and not always supported.
     Avoid using Graphics Interchange Format (GIF) if possible. It is technically inferior, larger size, and has licensing encumbrances. Rationale: PNG versus GIF considerations."
==============================
"Choice of video file formats:
     X3D browser support for the MPEG1 video file format format is required by the MovieTexture node definition in the X3D Specification.
     X3D browsers may offer to support multiple video formats. Since the X3D url field is an ordered list, authors can provide and link more than one format if they wish. X3D browsers will honor the first successfully playable link that they find.
     However consistent support for other formats beyond MPEG1 is not guaranteed since most popular video formats are highly encumbered by patents and licensing fees. Having authors or content providers account and pay for content is antithetical to the free growth of open standards and the World Wide Web.
     If a royalty-free (RF) video format becomes sufficiently popular and has multiple efficient open-source and commercial codec software implementations available, then the X3D Graphics Working Group and Web3D Consortium members will likely adopt its usage as a required format. Until then, support for any video format other than MPEG1 remains optional for X3D.
     Of related interest: the existence of the open X3D Graphics standard prevents a similar sad state of affairs from allowing patent-holding companies from "locking up" 3D graphics on the Web. Members of the Web3D Consortium deserve credit for maintaining this important achievement. Feel free to join or contribute so that such efforts can continue."
==============================

We have also looked at volume formats.  NRRD is a possibility.

Inclusion of Scalable Vector Graphics (SVG) as a recommended (or required) image format has been proposed for X3D version 4.  Since interaction and animation can be embedded in an SVG image, this might be represented as a new node - SvgImageTexture or somesuch.

As ever, our working group welcomes questions suggestions and improvements for the X3D specifications, X3D Resources and X3D Scene Authoring Hints pages.

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman