[x3d-public] ProtoBody completeness in nested Protos

Fri Apr 29 23:16:52 PDT 2022

Thanks for pointing out how Inline addresses self-referential code.
Adopting the same rules and the suggested language for Protos would make a
lot sense and would completely answer the concerns raised.

Andreas

---on the phone---

On Sat, Apr 30, 2022, 12:28 AM Brutzman, Donald (Don) (CIV) <
brutzman at nps.edu> wrote:

> Good catch about circular definitions getting disallowed Andreas.  Besides
> bad practice and illogical, it is also a security vulnerability.
>
>
>
> We have such a statement for Inline nodes.
>
>
>
>    - X3D4 Architecture, clause 9 Networking component, 9.4.2 Inline
>    -
>    https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/networking.html#Inline
>
>
>
> "Security precaution: it is an error for a model to Inline itself,
> directly or indirectly, in order to avoid nonterminating recursion loops.
> X3D players browsers SHALL NOT honor self-referential loading of model
> loops in order to avoid security vulnerabilities."
>
>
>
> Now checking prototypes...  There is one simpler statement found as
> follows, added by Mantis 1395.
>
>
>
> “A prototype may not be instantiated inside its own implementation *(i.e.*,
> recursive prototypes are illegal).”
>
>
>
>    - X3D4 Architecture, clause 4 Concepts, 4.4.4.4 Prototype scoping rules
>    -
>    https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/concepts.html#PROTOdefinitionsemantics
>
>
>
> This is such an important point that I think we should repeat a similar
> Security precaution, proposed as follows.
>
>
>
>    - X3D4 Architecture, clause 4 Concepts, 4.4.4.3 PROTO definition
>    semantics
>    -
>    https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/concepts.html#PROTOdefinitionsemantics
>
>
>
> "Security precaution: it is an error for a prototype to reference an
> instance or repeated declaration of itself, directly or indirectly, in
> order to avoid nonterminating recursion loops. This error might occur with
> either local or external (PROTO or EXTERNPROTO) declarations.  X3D players
> browsers SHALL NOT honor self-referential loading of prototype declaration
> loops in order to avoid security vulnerabilities."
>
>
>
> Have re-opened Mantis 1395 and applied this proposed change.  As ever,
> review and feedback welcome.
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, Navy robotics https://
> faculty.nps.edu/brutzman
>
>
>
> -----Original Message-----
> From: Andreas Plesch <andreasplesch at gmail.com>
> Sent: Friday, April 29, 2022 12:46 PM
> To: John Carlson <yottzumm at gmail.com>
> Cc: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; X3D Graphics public
> mailing list <x3d-public at web3d.org>
> Subject: Re: [x3d-public] ProtoBody completeness in nested Protos
>
>
>
>
>
> On Fri, Apr 29, 2022 at 2:14 PM John Carlson <yottzumm at gmail.com> wrote:
>
> >
>
> > What I’m trying to get at is, as the browser is processing the files
> (serially), in one case, either A or B is instanced before it’s definition.
>
>
>
> Let's see hour serial processing would work:
>
>
>
> 1) Load ExternProto Declaration for B (but do not do anything with it
> since not instanced yet)
>
> 2) Define Proto Declaration for A
>
> 3) Try to Instance A
>
> 4) Expand A with ProtoBody for A (ok since it is defined)
>
> 5) ProtoBody for A tries to Instance B
>
> 6) Expand B with ProtoBody for B (ok since definition is loaded in step 1,
> eg. before instantiation)
>
> 7) Try to instance A
>
> 8) Expand A with ProtoBody for A (ok since A is defined in file B which is
> loaded) and so on.
>
>
>
> So I think instantiation never occurs before definition, even in serial
> processing. Probably I overlooked something ?
>
>
>
> >  I think your extra clause should be added for parallel loading.  In
> particular, I don’t think that extern PROTOs should be allowed to reference
> each other in a circular way (ignoring regular PROTO definitions and
> instances to be clear).   I thought we were talking about PROTO instances,
> not extern PROTOs.   Hopefully everyone will agree to add Andreas’
> clarification.
>
>
>
> It would probably be helpful to make circular definitions explicitly
> illegal. They may crop up more commonly by accident than single file
> recursive Protos which are explicitly illegal. But a browser implementer
> could already point at the spec. and argue that circular definitions are a
> covered case of recursion (and treat accordingly).
>
>
>
>
>
>
>
> > Thanks!
>
> >
>
> > John
>
> >
>
> > On Fri, Apr 29, 2022 at 9:52 AM Andreas Plesch <andreasplesch at gmail.com>
> wrote:
>
> >>
>
> >> Hi John,
>
> >>
>
> >> Here is a draft of an example of a circular definition:
>
> >>
>
> >> file A.x3d:
>
> >>
>
> >> ExternProtoDeclare name='B' url=' "B.x3d" '
>
> >> ProtoDeclare name='A'
>
> >>  ProtoBody
>
> >>   ProtoInstance name='B'
>
> >>   Shape DEF='sphere'
>
> >>  /ProtoBody
>
> >> /ProtoDeclare
>
> >> ProtoInstance name='A'
>
> >>
>
> >> file B.x3d:
>
> >>
>
> >> ExternProtoDeclare name='A' url=' "A.x3d" '
>
> >> ProtoDeclare name='B'
>
> >>  ProtoBody
>
> >>   ProtoInstance name='A'
>
> >>   Shape DEF='box'
>
> >>  /ProtoBody
>
> >> /ProtoDeclare
>
> >>
>
> >> ProtoInterfaces are omitted (and not used here).
>
> >>
>
> >> Note that Proto A does not use an instance (directly) of itself, and
>
> >> Proto B also does not use an instance of itself.
>
> >>
>
> >> Browser tries to display A.x3d .
>
> >>
>
> >> This case is not realistic since it cannot be resolved. But, like
>
> >> recursive cases, there are situations which could render in theory,
>
> >> if the circle is eventually broken for certain conditions. Since such
>
> >> a circular definition is also recursive, albeit indirectly, the
>
> >> existing recursive clause in the spec. may cover this as well. Such
>
> >> circular definitions leading to infinite loops would be hard to
>
> >> detect for a browser (perhaps by monitoring for stack overflows or by
> time outs).
>
> >>
>
> >> Andreas
>
> >>
>
> >> On Fri, Apr 29, 2022 at 12:00 AM John Carlson <yottzumm at gmail.com>
> wrote:
>
> >> >
>
> >> > One can’t instantiate until after definition.  I’m fairly sure that
> covers this case?  Maybe we can work with an example?
>
> >> >
>
> >> > Thanks,
>
> >> >
>
> >> > John
>
> >> >
>
> >> > On Thu, Apr 28, 2022 at 9:23 PM Andreas Plesch <
> andreasplesch at gmail.com> wrote:
>
> >> >>
>
> >> >> Thanks much for the review and revision effort.
>
> >> >>
>
> >> >> The recursive case brings up circular dependencies. Proto A may use
> an instance of Proto B, but Proto B in turn may use an instance of Proto A
> unless some initial condition occurs, so that in theory a result can be
> resolved.
>
> >> >>
>
> >> >> The recursive clause in a way covers this in my view but perhaps
> circular definitions could be made explicitly illegal.
>
> >> >>
>
> >> >> Just a thought,
>
> >> >>
>
> >> >> Andreas
>
> >> >>
>
> >> >> ---on the phone---
>
> >> >>
>
> >> >> On Thu, Apr 28, 2022, 8:45 PM Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
> >> >>>
>
> >> >>> We reviewed closely.  Prose definitions are correct.  Slight
> editorial changes (essentially re-ordering sentences) hopefully make things
> clearer.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Mantis 1395: improve clarity of prototype scoping rules
>
> >> >>> https://www.web3d.org/member-only/mantis/view.php?id=1395
>
> >> >>>
>
> >> >>> -
>
> >> >>>
>
> >> >>> 4.4.4.4 Prototype scoping rules
>
> >> >>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-
>
> >> >>> CD1/Part01/concepts.html#Prototypescopingrules
>
> >> >>>
>
> >> >>> -
>
> >> >>>
>
> >> >>> A prototype may be instantiated in a file anywhere after the
> completion of the prototype definition. Prototype definitions appearing
> inside a prototype definition (i.e., nested) have scope local to the
> enclosing prototype. IS statements inside a nested prototype's
> implementation may refer to the prototype declarations of the innermost
> prototype. A prototype may not be instantiated inside its own
> implementation (i.e., recursive prototypes are illegal).
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> A PROTO statement establishes a DEF/USE name scope separate from
> the rest of the scene and separate from any nested PROTO statements. Nodes
> given a name by a DEF construct inside the prototype may not be referenced
> in a USE construct outside of the prototype's scope. Nodes given a name by
> a DEF construct outside the prototype scope may not be referenced in a USE
> construct inside the prototype scope.
>
> >> >>>
>
> >> >>> -
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Again thanks for all efforts to make the X3D4 specification clear
> and consistently repeatable.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> all the best, Don
>
> >> >>>
>
> >> >>> --
>
> >> >>>
>
> >> >>> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> >> >>>
>
> >> >>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> >> >>>
>
> >> >>> X3D graphics, virtual worlds, Navy robotics https://
>
> >> >>> faculty.nps.edu/brutzman
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> From: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>
> >> >>> Sent: Wednesday, April 27, 2022 9:23 PM
>
> >> >>> To: GPU Group <gpugroup at gmail.com>; Andreas Plesch
>
> >> >>> <andreasplesch at gmail.com>; Richard F. Puk (puk at igraphics.com)
>
> >> >>> <puk at igraphics.com>
>
> >> >>> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>;
>
> >> >>> Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>
> >> >>> Subject: RE: [x3d-public] ProtoBody completeness in nested Protos
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Thanks for your precision and thoroughness Doug.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Seems clear and unambiguous that it is scoped, as previously
> described.  We’ll double check on phraseology.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> Dick can we please discuss in tomorrow’s meeting, TIA.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> all the best, Don
>
> >> >>>
>
> >> >>> --
>
> >> >>>
>
> >> >>> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> >> >>>
>
> >> >>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> >> >>>
>
> >> >>> X3D graphics, virtual worlds, Navy robotics https://
>
> >> >>> faculty.nps.edu/brutzman
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> From: x3d-public <x3d-public-bounces at web3d.org> On Behalf Of GPU
>
> >> >>> Group
>
> >> >>> Sent: Tuesday, April 26, 2022 9:57 AM
>
> >> >>> To: X3D Graphics public mailing list <x3d-public at web3d.org>
>
> >> >>> Subject: Re: [x3d-public] ProtoBody completeness in nested Protos
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> https://www.web3d.org/documents/specifications/19775-1/V4.0/Part0
>
> >> >>> 1/concepts.html#Prototypescopingrules
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> "A prototype may be instantiated in a file anywhere after the
> completion of the prototype definition."
>
> >> >>>
>
> >> >>> If the specs meant what you are suggesting --protos are self
> contained including proto definitions-- then it would have said "context"
> rather than "file"
>
> >> >>>
>
> >> >>> "Prototype definitions appearing inside a prototype definition (
> i.e., nested) are local to the enclosing prototype. "
>
> >> >>>
>
> >> >>> - refinement of the above file order rule.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> On Tue, Apr 26, 2022 at 9:51 AM GPU Group <gpugroup at gmail.com>
> wrote:
>
> >> >>>
>
> >> >>> "I really do
>
> >> >>> not think that proto declarations should be required to inherit
>
> >> >>> existing declarations from parent execution contexts"
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> If C++ required each class definition to be self-contained --to
> redefine all the types it depends on-- it would be a very messy and onerous
> system. If you want a system that can build up types - types on top of
> types to make more complex types - then you need a way for a type to use
> previously defined types.
>
> >> >>>
>
> >> >>> In x3d -a prototyping language- that looks like a new type being
> able to use a type defined previously in the file. Extern ProtoDeclares are
> stored in scene files as ProtoDeclares. As such they should be able to use
> type defined previously in their host file.
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>>
>
> >> >>> On Tue, Apr 26, 2022 at 9:37 AM Andreas Plesch <
> andreasplesch at gmail.com> wrote:
>
> >> >>>
>
> >> >>> > Date: Tue, 26 Apr 2022 07:56:47 -0600
>
> >> >>> > From: GPU Group <gpugroup at gmail.com>
>
> >> >>> > To: X3D Graphics public mailing list <x3d-public at web3d.org>
>
> >> >>>
>
> >> >>> > When I started with freewrl around 2009, it had a text-based
>
> >> >>> > system for
>
> >> >>> > protos:
>
> >> >>> > - when parsing a scene, if it found a proto declare, it would
>
> >> >>> > scrape the text into a buffer. Then as it continued parsing,
>
> >> >>> > when it hit a proto instance it pasted, and continued parsing
>
> >> >>> > over the pasted text as though it had been there all along.
>
> >> >>> > Extern protos similar, it would pause parsing, go to the file
>
> >> >>> > referenced, look for the proto declare by name, and scrape out
> the text of the ProtoDeclare, for pasting in-scene.
>
> >> >>> > And it worked a bit. But especially extern protos weren't working
> reliably.
>
> >> >>> > As a casual volunteer I decided to fix the bugs. How hard could
>
> >> >>> > it be? A little debugging and presto, I'd be a hero. As I got
>
> >> >>> > into it and saw more scene failure examples there were several
>
> >> >>> > emotional stages like grieving denial, shock, sadness,
>
> >> >>> > resistance etc. And a slow dawning realization the whole system
> needed a rework.
>
> >> >>> > And I didn't have support from others -just silence, perhaps
>
> >> >>> > they knew it was a mess and wanted to stay clear.
>
> >> >>> > It was a sickening feeling, and I needed to think for a few
>
> >> >>> > weeks whether I even wanted to be a volunteer programmer if the
>
> >> >>> > system was rotten from the core.
>
> >> >>> > So it can not only be a monster task, but precisely because
>
> >> >>> > it's a monster there's little support for those brave enough to
> tackle it.
>
> >> >>> > (After a few weeks I got to the Acceptance stage of grieving,
>
> >> >>> > and spent 6 months of hobby time building the new proto system).
>
> >> >>>
>
> >> >>> Thanks for the story, and for engaging. To me, I became
>
> >> >>> interested when I realized that x3dom is probably flexible enough
>
> >> >>> to allow for parameterized definition and registration of new
>
> >> >>> nodes dynamically which behave then exactly like native nodes.
>
> >> >>> This is different from filling in templates. For example, there
>
> >> >>> is no performance penalty during tree traversal and it enables
>
> >> >>> instancing using xml nodes with the new names, just as in x3dv.
>
> >> >>>
>
> >> >>> I am pretty happy with how it came out. An additional
>
> >> >>> complication is asynchronous loading of remote resources which is
> standard on the web.
>
> >> >>> X3D should rely less on how nodes or statements are ordered in a
>
> >> >>> scene.
>
> >> >>>
>
> >> >>> It is just frustrating that there are ill defined corners. I
>
> >> >>> really do not think that proto declarations should be required to
>
> >> >>> inherit existing declarations from parent execution contexts (
>
> >> >>> DEF/USE name scopes are different but there is still a separated
>
> >> >>> execution context ). That means the protos array of a new
>
> >> >>> execution context should be empty initially. The SAI spec. actually
> says so.
>
> >> >>>
>
> >> >>> Best, Andreas
>
> >> >>>
>
> >> >>> _______________________________________________
>
> >> >>> x3d-public mailing list
>
> >> >>> x3d-public at web3d.org
>
> >> >>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
> >> >>
>
> >> >> _______________________________________________
>
> >> >> x3d-public mailing list
>
> >> >> x3d-public at web3d.org
>
> >> >> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
> >>
>
> >>
>
> >>
>
> >> --
>
> >> Andreas Plesch
>
> >> Waltham, MA 02453
>
>
>
>
>
>
>
> --
>
> Andreas Plesch
>
> Waltham, MA 02453
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220430/629f1f8b/attachment-0001.html>