[x3d-public] towards concurrent X3D processing

[John Carlson]

I couldn’t read all of this, but I recall keeping a counter of “children
loaded”. When I reached the number of children = counter, I would exit
loading.  The thing was, I would often hang on the last child.  I do not
recall how I solved this.

I can probably dig into this.  I don’t recall adding extra state to nodes,
rather I used a local variable, possibly passed as a parameter. I recall
the number of callback functions as horrendous, so doing something with
await async would probably be preferred?  Idk, sorry.

I get the heebee jeebees just thinking about it.

I think your containerFieldIndex may be a good idea, but I worry about
missing or removed indexes.

John

On Sun, May 1, 2022 at 8:48 PM Andreas Plesch <andreasplesch at gmail.com>
wrote:

> Please let me outline an idea which would help with concurrent and
> asynchronous processing by making the position of a node in an MFNode
> array explicit. A new attribute similar to "containerField" would hold
> the index of the node in the array, a "containerFieldIndex" attribute.
> This would probably only apply to MFNode fields, and allow a browser
> to easily assign the intended array position to a node, even if its
> loading and processing is delayed, and thus support parallel and out
> of sequence processing better.
>
> Feel free to skip the longer explanation below, Any feedback welcome.
>
> X3D relies on the idea of sequential processing of a file. For
> example, USE nodes can only occur after their DEFinitions, or
> ProtoInstances can only occur after the corresponding
> ProtoDeclarations.
>
> This helps with processing since the browser can assume a sensible
> sequence in the file, or complain otherwise to the author. It also
> helps with authoring since it provides a clear paradigm an author can
> understand and follow.
>
> However, for nodes using remote resources sequential processing can
> lead to complete blocking while waiting for timeouts, or due to slow
> or unreliable server responses. Also, some nodes which use a lot of
> data, eg. large IFS, may take significant time to parse and add to the
> scene. In other words, for fast loading of large scenes it is
> desirable to process nodes in parallel and out of order or
> asynchronously.
>
> This is possible even with a sequential loading paradigm, the browser
> just has to carefully keep track of things, use placeholders etc. In
> fact, x3dom loads Inline and ExternProtos asynchronously, adding
> content and instances out of sequence to an already running scene.
> This way a user can start to see and interact with a scene earlier.
>
> To do this, x3dom uses the "containerFieldIndex" idea internally, to
> first synchronously record the index position of an ExternProto
> instance during sequential loading. Then, after out of sequence, slow
> downloading and processing, the instance is added to the parent MFNode
> field in the standard manner, eg. at the end of the array. But in a
> final step, x3dom can use the recorded containerFieldIndex to
> rearrange the array to have the correct order.
>
> While this can be done internally, exposing a "containerFieldIndex"
> attribute would have benefits. It would help authors to be both clear
> and flexible about how they want nodes ordered, especially when
> programmatically constructing scenes. It helps browsers since they
> could rely on such an attribute in their processing. It would be
> universal and apply to MFNode fields of any node. And it would promote
> parallel processing since nodes become more self-describing. For
> backwards compatibility, the default value of a containedFieldIndex
> attribute would be the index of the node as given by its position in
> the file.
>
> "containerField" is already a strange concept (since it is unusual to
> refer to something in parents), so "containerFieldIndex" may be even
> stranger. But I could not think of a better idea to help with
> asynchronous loading. I am pretty sure there are better ones.
>
> Thanks for reading to the end ! Cheers, Andreas
>
> --
> Andreas Plesch
> Waltham, MA 02453
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220501/ea3b0af6/attachment.html>