[x3d-public] towards concurrent X3D processing

[Andreas Plesch]

Please let me outline an idea which would help with concurrent and
asynchronous processing by making the position of a node in an MFNode
array explicit. A new attribute similar to "containerField" would hold
the index of the node in the array, a "containerFieldIndex" attribute.
This would probably only apply to MFNode fields, and allow a browser
to easily assign the intended array position to a node, even if its
loading and processing is delayed, and thus support parallel and out
of sequence processing better.

Feel free to skip the longer explanation below, Any feedback welcome.

X3D relies on the idea of sequential processing of a file. For
example, USE nodes can only occur after their DEFinitions, or
ProtoInstances can only occur after the corresponding
ProtoDeclarations.

This helps with processing since the browser can assume a sensible
sequence in the file, or complain otherwise to the author. It also
helps with authoring since it provides a clear paradigm an author can
understand and follow.

However, for nodes using remote resources sequential processing can
lead to complete blocking while waiting for timeouts, or due to slow
or unreliable server responses. Also, some nodes which use a lot of
data, eg. large IFS, may take significant time to parse and add to the
scene. In other words, for fast loading of large scenes it is
desirable to process nodes in parallel and out of order or
asynchronously.

This is possible even with a sequential loading paradigm, the browser
just has to carefully keep track of things, use placeholders etc. In
fact, x3dom loads Inline and ExternProtos asynchronously, adding
content and instances out of sequence to an already running scene.
This way a user can start to see and interact with a scene earlier.

To do this, x3dom uses the "containerFieldIndex" idea internally, to
first synchronously record the index position of an ExternProto
instance during sequential loading. Then, after out of sequence, slow
downloading and processing, the instance is added to the parent MFNode
field in the standard manner, eg. at the end of the array. But in a
final step, x3dom can use the recorded containerFieldIndex to
rearrange the array to have the correct order.

While this can be done internally, exposing a "containerFieldIndex"
attribute would have benefits. It would help authors to be both clear
and flexible about how they want nodes ordered, especially when
programmatically constructing scenes. It helps browsers since they
could rely on such an attribute in their processing. It would be
universal and apply to MFNode fields of any node. And it would promote
parallel processing since nodes become more self-describing. For
backwards compatibility, the default value of a containedFieldIndex
attribute would be the index of the node as given by its position in
the file.

"containerField" is already a strange concept (since it is unusual to
refer to something in parents), so "containerFieldIndex" may be even
stranger. But I could not think of a better idea to help with
asynchronous loading. I am pretty sure there are better ones.

Thanks for reading to the end ! Cheers, Andreas

-- 
Andreas Plesch
Waltham, MA 02453