[X3D-Public] Fwd: Re: [X3D] X3D HTML5 meeting discussions: Declarative 3D interest group at W3C

[Chris Marrin]

On Dec 21, 2010, at 7:08 AM, Kristian Sons wrote:

> Dear all,
> 
> in preparation for our meeting today we worked on a draft for the
> "Declarative 3D" XG charter. We can use it as basis for the discussions
> today.


Hi,

For those of you who don't know me, I'm currently the editor of the WebGL spec and its main implementor on WebKit/Safari. I was involved with VRML from the beginning, but withdrew from the X3D efforts when our goals diverged.
==========

Before I start, let me give the executive summary (since this thing has gotten so long):
	- It's great to see this effort get underway
	- A "declarative 3D language" should avoid SVG's pitfalls and stay small and focused.
	- Use other web technologies (including WebGL) so you don't have to solve every problem.
	- Make your nodes work well with the rest of HTML, including the DOM and CSS.
	- Especially make sure your node set integrates well with WebGL to allow for extensions and modification of the rendering result.
	- Consider prototyping on top of WebGL as a way to test features and integration with the page.

==========
I applaud your effort to start an XG. I hope you go into the effort with an open mind and don't instantly attempt to ratify some form of the X3D spec. I have long felt that the main flaw of X3D is the fact that it stands alone. It has its own parser, its own scripting, its own image and media handling, it own event model and its own DOM. The event model in particular is completely foreign to the HTML model. It is also too big, even in its most modest forms. Look at SVG. It has taken many years for it to get even a toehold in modern browsers and even those implementations (including the one in WebKit) are inefficient and slow.'

I don't have insight to any browser other than those based on WebKit, but I can say that one of the main reasons for our implementation not being more performant is that the spec is too big. Implementing all those features on all supported platforms is extremely difficult. If you look at WebGL in contrast, there are now 7 implementations, all interoperable and performant, on 3 platforms (Windows, Mac, Linux). The Google Body example:

	http://bodybrowser.googlelabs.com/body.html

is rendering 1.5 million triangles per second and runs at frame rate on all 7 WebGL capable browsers. This is not bragging, I mention it because it took us less than 2 years to get to where we are. I believe the reasons for that are:

1) Our images are loaded via <img>, <video> or <canvas> elements. We hope in the future to be able to use rendered HTML as an image source as well.

2) We use the browser's JavaScript engine, so we get to take advantage of performance improvements currently being done by all the browsers. And since we're integrated, we can make improvements to the native interface between JavaScript and WebGL to further improve performance.

3) WebGL is created via the Canvas element, so we don't need to worry about compositing, layout or any other element level features.

4) WebGL is based on OpenGL ES 2.0. We didn't have to solve any of the hard 3D rendering problems, we just had to adapt an existing engine.

5) WebGL has wide acceptance among the browser vendors. Chrome, Firefox and Safari all support it, and an Opera version is in the works.

That last point has been incredibly helpful. Since Chrome and Safari are based on the same codebase, when you fix a bug for one, you fix it for all. The Google team has fixed a huge number of bugs that have benefited Safari. And Google also implemented ANGLE as a free an open toolkit which allows WebGL to run on D3D. The Firefox team is using ANGLE now and soon WebKit will be available on Windows because of that great effort.


	I want to put a caveat on all this. WebGL is in unreleased versions of all the browsers. I believe 
	Chrome and Firefox will have a version in they released products soon, but it is still only in 
	nightly builds of WebKit for Mac. There are stirrings about having it on mobile platforms but no 
	implementations yet. Mozilla showed an experimental version on a Nokia 900. I'm not sure if 
	that has deployed, or even if it will deploy. I don't know about progress on Android and I can't 
	talk about any Apple plans. But the spec is clearly aimed at being implementable on mobile 
	devices.


The biggest challenge in WebGL was adapting the OpenGL model, which is completely unsafe and left all of the resource management to the application developer, to a Web model where the browser must never do anything unsafe, insecure, or leak resources. We also tuned the API to make it possible to implement on D3D in addition to OpenGL.

WebGL is almost ready to be ratified by Khronos. We have the above mentioned implementations, with more on the way. We also have a conformance suite that is growing every day. And many bugs have been fixed, as more content is written to challenge the implementations, so WebGL is getting very stable.

I'm saying all this so hopefully you don't go down the path of trying to solve too many problems. One of the motivators of this message was because there are 2 references to WebGL in your document. In the first you stated how WebGL had deficiencies which motivated your XG, and in the other you state that APIs like WebGL are out of scope for your XG. This troubled me. New web technologies build on the existing specifications and implementations. You could base your work on WebGL and you would reduce your workload dramatically.

For instance, assuming you integrate shader technology into your proposal, what shader language will you use? In the past this has been undefined, so if someone wanted to run on OpenGL and D3D, they would need to write shaders in both GLSL and HLSL and possibly more than one variant to match the hardware being used. WebGL has solved that problem in a way that guarantees good cross platform fidelity and implementation feasibility. We use GLSL ES 1.0 with constraints listed in the spec so we translate to desktop GLSL (various versions) and HLSL using ANGLE. This restricts the shader capabilities to those in GLSL ES, but it ensures wide deployment, all the way down to mobile devices. We have an extension mechanism so we can add features from that base (texture_float is an early extension which has been implemented and tested by Google).

Another issue is how to handle binary data such as compressed textures, coordinates and animation keyframes. This is another area we knew early on would be an issue, so we created Typed Arrays (https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html) to deal with the problem. The next issue that came up was how to load binary data. Today's HTML browsers aren't very good at handling data types other than string and to convert the data to text would be very inefficient. So we've made a proposal to add the ability to load binary data using XMLHttpRequest. The data is loaded directly into a Typed Array so there is no conversion step getting in the way. The data can then be loaded into a VBO without any copying having been done.

I hope you can find all this existing work helpful to you. I agree that a declarative form is needed for 3D on the Web. I just hope it integrates well with the rest of the web technologies, including WebGL. For instance, wouldn't it be great if an author could write an extension node in JavaScript and WebGL. You could also allow authors to drop down into a WebGL API to make low-level adjustments to the state or to the buffers loaded in the GPU.

This brings me to my last point. Johannes and the others at Fraunhofer have done an incredible job of producing an X3D implementation on top of WebGL. So it is clearly possible to represent a complex declarative form on top of a JavaScript API. I know they experienced some pitfalls and deficiencies along the way. You can (or we all can) learn from the issues they found. To get around these issues your XG could avoid that element or feature of an element if possible. Or you could raise the issue with other W3C groups so browsers can be changed in the future to better suite your needs.

One area where WebGL is deficient is in its inability to use CSS directly. Even if you make a node set like X3DOM, being able to use a CSS color property on one of those nodes is very difficult. It's not hard to read the color and apply that to a 3D primitive. But being notified that the color had changed, either by programmatically changing the style, or by animating the color with CSS Animation, is impossible. A huge advantage of a native node set is that such notification is easy and would provide great integration with the CSS style system of today's browsers.

So my first hope is that you can use WebGL as a way to prototype design ideas and avoid having to tackle the difficult problems that have already been solved by other technologies. My second hope is that you make your node set small and well integrated with the HTML DOM and with CSS to make it an attractive addition to web browsers.

-----
~Chris
chris at marrin.com