[X3D-Public] Fwd: Re: [X3D] X3D HTML5 meeting discussions: Declarative 3D interest group at W3C

[Chris Marrin]

On Dec 22, 2010, at 12:43 AM, Kristian Sons wrote:

> Dear Chris,
> 
> thanks a lot for joining the discussion and for your precious input. Some of your points should definitly be integrated into the charter draft.
> 
> Let me comment some of your point from my view:
> 
>> 
>> I applaud your effort to start an XG. I hope you go into the effort with an open mind and don't instantly attempt to ratify some form of the X3D spec. I have long felt that the main flaw of X3D is the fact that it stands alone. It has its own parser, its own scripting, its own image and media handling, it own event model and its own DOM. The event model in particular is completely foreign to the HTML model. It is also too big, even in its most modest forms. Look at SVG. It has taken many years for it to get even a toehold in modern browsers and even those implementations (including the one in WebKit) are inefficient and slow.'
> 
> I am very glad you welcome our initiative. I guess most browser developers with 3D know-how are working on WebGL right now. That's probaby why a declarative approach is out of scope for most of them. I think WebGL lifts of in 2011 and applications and libraries pop up from everywhere. That is great! But according to the Hype Cycle there will also be the "Trough of Disillusionment". That might be a good starting point to deliver a declarative alternative for those applications that run better with it. But that will be soon. Thus I fully agree that we need a lean solution that leverages ALL existing technologies and is fastly implementable.

Actually, having been involved in non-X3D web efforts and especially since coming to Apple and working on WebKit, I think the hesitation about a 3D declarative language is more than just "we're too busy right now". WebGL worked its way into the browsers because it was a tractable problem. There were those who thought a 3D API on JavaScript would not fly (and some still do). But no one thought OpenGL ES 2.0 was too complex for today's systems. X3D (as the most popular example) is seen as too cumbersome and not integrated to be included. 

That's why I think the most important points to focus on in your XG are simplicity and integration. Mention SAI, routing or a separate JS engine and I believe the chances for acceptance are low. One other seemingly minor point is that we made our spec look and feel like a W3C spec. We mention other W3C technology and how we interact with it. We extended an existing node, CanvasElement, and so were asked by Ian Hickson to help make the CanvasElement spec more clear in how getContext() works. We did Typed Arrays, and are now involved with the XMLHttpRequest and File API standards to make them Typed Array aware. These are small things, but they are important because they let you integrate your efforts, as you integrate your spec, with the rest of W3C.

All that is my way of saying that you shouldn't hold off on making progress just because you think WebGL has the browsers up in the air. There's always room for good new ideas.

> 
> One of the reasons why we started the XG is, because we think that the HTML integration is the main focus point. It's not about ratifying X3D spec, X3DOM or XML3D. I think we have an excellent starting point with two approaches (XML3D and X3DOM), WebGL prototypes, native implementations in Chrome and Firefox and several plugin implementations. Thus we have some lessons learned. Now it's time to identify the requirements and based on that, we have to find an optimal solution for the Web. That's why we a looking for industry backup (probably another flaw of X3D) for our initiative and I think we are on a good way. So one topic will be to find a good technical solution, the other to show use-cases for declarative applications and to demonstrate the industry interest.

We did WebGL in Khronos because we wanted to take advantage of the expertise of the Khronos participants. I think you have good expertise in 3D declarative forms already, but I would recommend you stay close to other W3C efforts to be able to draw on their more general web expertise as well. As a WebKit developer, I would also recommend you pull a WebKit tree and understand it. We already have a cross-platform 3D rendering API (GraphicsContext3D). WebGL uses it already, and Google is beginning to use it for accelerated 2D rendering. GraphicsContext3D allows you to work on multiple platforms, including D3D via ANGLE, with a single implementation. Putting your declarative elements on top of it would be relatively simple and would speed development. The same may be true of Mozilla, but I'm not familiar with their codebase. But being completely unbiased, WebKIt is way better :-)

> 
>> 
>> I'm saying all this so hopefully you don't go down the path of trying to solve too many problems. One of the motivators of this message was because there are 2 references to WebGL in your document. In the first you stated how WebGL had deficiencies which motivated your XG, and in the other you state that APIs like WebGL are out of scope for your XG. This troubled me. New web technologies build on the existing specifications and implementations. You could base your work on WebGL and you would reduce your workload dramatically.
> 
> This is one of the main topics we discussed today during the telco. I think it was Anita who said mentioning WebGL in the "Out of Scope" section might be missunderstood, and here we go...  It's not WebGL that is out of scope. To develop an imperative API-approach is out of scope. Perhaps we should omit this point. The name of the XG already says it.

I agree that a pure API approach is out of scope. But you will have some sort of API for your nodes because of the DOM. The big question is how much API you expose in the DOM of your nodes. My personal opinion is that it would be valuable to let authors "crack through" into an underlying WebGL API to do advanced things. But that's just my opinion. And think about how you would use CSS in your nodes. Avoid DOM attributes like 'diffuseColor' or 'transform', and use the current CSS style for that info.

> 
> We should definitly take WebGL into account. I think WebGL is the enabler for our activities and without WebGL we would have a chance at all. I think especially the Typed Arrays is something that should work fluently between both approaches.

Yes and just FYI, there is a new Audio XG that we Apple is involved in. They are already using Typed Arrays for their binary audio data. So when you add that effort to XHR, the File API and of course WebGL, they are a powerful mechanism to deal with binary data. You should certainly use them if your API exposes binary data. There is also talk of using Typed Arrays as a way to more efficiently transfer data between threads in Web Workers. But that is still just talk.

> 
>> One area where WebGL is deficient is in its inability to use CSS directly. Even if you make a node set like X3DOM, being able to use a CSS color property on one of those nodes is very difficult. It's not hard to read the color and apply that to a 3D primitive. But being notified that the color had changed, either by programmatically changing the style, or by animating the color with CSS Animation, is impossible. A huge advantage of a native node set is that such notification is easy and would provide great integration with the CSS style system of today's browsers.
> 
> The CSS integration is a tricky thing. It's powerful and well-known. It's perfect for a fixed-function rendering like HTML. But hard to merge this with the programmable 3D pipeline. In the XML3D project we use CSS to assign shaders to geometry but not for the generic shader parameters. I would really like to see a stronger CSS usage. On the other hand it makes it harder to protoype it with JS/WebGL.

Yes, I don't know the details but I believe this was one of the most difficult parts of X3DOM, and I don't know how much integration they were able to achieve. I've thought about ways to expose CSS mutation to JS, but there is no such thing currently. But CSS would give you incredible power. Something like:

<head>
<style>
	.spin-animator {
		animation: spinner 2s linear;
	}
	@keyframes spinner {
		from: { transform:rotateY(0deg) }
		to: { transform:rotateY(360deg) }
	}
</style>
</head>
<body>
	<new3D ... maybe some namespace'y stuff here ... >
		<box class="spin-animator">
			<diffuseColor style="color:red" />
		</box>
	</new3D>
</body>

This is simple and clear (at least to me, having thought about this for many years), and is very easy with nodes integrated into the engine.
	
> 
> So with your experience it would be nice to have you being involved in some way with the XG. As an external professional or with Apple. I know that you are very busy with WebGL. But at least getting your input as kind of reviewer would be extremely helpful!

I'm always happy to help out. Fortunately XG's aren't so formal, so my affiliation isn't so important. I can't participate in any formal way because I'm busy (and not just with WebGL!). But I'd love to be on your mailing list.

-----
~Chris
chris at marrin.com