[x3d-public] x3dom prototypes, extern proto
John Carlson
yottzumm at gmail.com
Mon Jun 29 10:10:49 PDT 2020
Here's the issue I was unable to solve in 36 hours of debugging, still
related to protos.
https://github.com/x3dom/x3dom/issues/1044
I should be able to provide a data file, t1.x3d, if necessary.
John
On Mon, Jun 29, 2020 at 11:11 AM John Carlson <yottzumm at gmail.com> wrote:
> How does one handle scripts inside a ProtoBody (multiple instances of the
> same script) is what next on our (or just my) plate.
>
> On Mon, Jun 29, 2020 at 11:04 AM Andreas Plesch <andreasplesch at gmail.com>
> wrote:
>
>> Nothing is done to script elements. So the ones with a vrmlscript ( or
>> any non html standard ) mime type are ignored, and regular script
>> elements executed by the browser as dom scripts.
>>
>> -Andreas
>>
>> On Mon, Jun 29, 2020 at 11:57 AM John Carlson <yottzumm at gmail.com> wrote:
>> >
>> > Yeah! Andreas! Once you clean up, I will attempt to add some kind of
>> internal scripting to X3DOM. I’m curious though, what do you currently do
>> with Protos with Scripts? Where does the script go?
>> >
>> > John
>> >
>> > On Mon, Jun 29, 2020 at 10:31 AM Andreas Plesch <
>> andreasplesch at gmail.com> wrote:
>> >>
>> >> After too many hours in the debugger, I added not too many lines to
>> >> better deal with changes to node value fields, in particular on how to
>> >> properly transfer these from the proto instance node to the underlying
>> >> native (or other proto) node.
>> >>
>> >> rubikFurnace.x3d should work now, in parallel with the logo letter
>> >> example which was the main challenge.
>> >>
>> >>
>> https://andreasplesch.github.io/x3dom/test/functional/proto/inline.html
>> >> https://5efa06157960e80256fa5d6b--x3dom.netlify.app/
>> >>
>> https://5efa06157960e80256fa5d6b--x3dom.netlify.app/examples/functional/proto/inline.html
>> >>
>> >> I believe this concludes my substantive efforts. I will now focus on
>> >> separating out the proto code to a new file and some clean up.
>> >>
>> >> Andreas
>> >>
>> >>
>> >> On Sun, Jun 28, 2020 at 6:40 PM John Carlson <yottzumm at gmail.com>
>> wrote:
>> >> >
>> >> > I got some more time to work on this, and the new X3DOM proto code
>> does just as well with JSON as it does for XML. Congratulations!
>> >> >
>> >> > I think if Andreas fixes the outstanding rubik*'s errors (hopefully
>> this will stop the BoxEm errors), and the t1.json/t1.x3d error in the x3dom
>> issues, I believe we can declare this successful, and we can move onto
>> "scripts"--whatever we're going to call the new script node, if not
>> Script. If someone has a document on how X3DOM routes work (especially
>> with namescopes), that will help me help with scripts.
>> >> >
>> >> > I'm not checking in my X3DJSONLD proto expander disabling code for
>> now. For now, my copy will remain disabled for testing.
>> >> >
>> >> > There are errors to the X3DJSONLD console if someone wants me to
>> copy and paste.
>> >> >
>> >> > If you want me to check in code for this into x3dom/andreasplesch's
>> gh-pages branch, let me know. I only made change to the
>> PrototypeExpander.js code, which effectively disabled the JSON proto
>> expander, see below patch (patch to package.json unnecessary).
>> >> > diff --git a/package.json b/package.json
>> >> > index e1e0d501..7043f4f5 100644
>> >> > --- a/package.json
>> >> > +++ b/package.json
>> >> > @@ -10,7 +10,7 @@
>> >> > },
>> >> > "scripts": {
>> >> > "test": "echo \"Error: no test specified\" && exit 1",
>> >> > - "build": "node ./build/src-builder.js",
>> >> > + "build": "node --trace-warnings ./build/src-builder.js",
>> >> > "lint": "eslint \"src/**/*.js\"",
>> >> > "lint-fix": "eslint --fix \"src/**/*.js\""
>> >> > },
>> >> > diff --git a/src/util/json/PrototypeExpander.js
>> b/src/util/json/PrototypeExpander.js
>> >> > index 9bf64e6a..fc1e87e0 100644
>> >> > --- a/src/util/json/PrototypeExpander.js
>> >> > +++ b/src/util/json/PrototypeExpander.js
>> >> > @@ -740,6 +740,8 @@ x3dom.PROTOS.prototype = {
>> >> >
>> >> > prototypeExpander : function ( file, object )
>> >> > {
>> >> > + // Use Andreas' proto code
>> >> > + /*
>> >> > this.protos = {};
>> >> > this.names = {};
>> >> > this.protoField = {};
>> >> > @@ -756,6 +758,7 @@ x3dom.PROTOS.prototype = {
>> >> > object = this.flattener( object );
>> >> > // console.error("SCRIPTS",
>> JSON.stringify(this.scriptField));
>> >> > // console.error("PROTOS", JSON.stringify(this.protoField,
>> null, 2));
>> >> > + */
>> >> > return object;
>> >> > },
>> >> >
>> >> >
>> >> > On Sun, Jun 28, 2020 at 2:38 PM John Carlson <yottzumm at gmail.com>
>> wrote:
>> >> >>
>> >> >> I'm showing X_ITE sort of working, select rubikFurnace.json on the
>> below linked page. Perhaps it's the conversion to/from JSON that makes it
>> work? Or perhaps the JSON proto expander? Turning off the proto expander
>> shows spheres for X_ITE/JSON, but green cubes for X_ITE XML/DOM. It
>> appears that X_ITE/JSON/Protos/rubikFurnace needs some work which
>> X3DJSONLD/proto expander magically fixes. I usually run with proto
>> expansion enabled, so I wouldn't normally catch this error! Thanks for
>> the bug report!
>> >> >>
>> >> >> https://coderextreme.net/X3DJSONLD/src/main/html/index.html
>> >> >>
>> >> >> This uses your netlify version of x3dom.
>> >> >>
>> >> >> Pasting XML into the above page with proto expansion on, only the
>> X3DOM/XML/DOM version fails. Other versions show green cubes. Without the
>> proto expansion, the previous example mentioned fails, but the JSON X_ITE
>> version fails with white cubes. Loading XML with a local server shows no
>> differences. I guess I could check the console next. Looks okay.
>> >> >>
>> >> >> After working on t1.json for a while (see your x3dom issue related
>> to proto expander), I noticed that my changes broke some of the rubik*'s
>> examples. I was not successful at making both t1.json and rubik.json
>> working. I don't know if that helps or not. Making the .x3d versions of
>> these examples work may be tricky.
>> >> >>
>> >> >> I am following your lead on renaming "sphere" to "sphereproto" in
>> rubik.x3d
>> >> >>
>> >> >> I will start testing with your netlify version:
>> >> >>
>> >> >> Here is my proto check page with netlify (all examples seem to
>> work, json proto expander on):
>> http://coderextreme.net/X3DJSONLD/src/main/html/x3domproto.html for JSON.
>> >> >>
>> >> >> Here's the equivalent for XML Inlines (please try to get this page
>> or similar working like the previous!) with netlify.
>> https://coderextreme.net/X3DJSONLD/src/main/html/xmlproto.html
>> >> >>
>> >> >> The two files should be checked into github for your editing
>> convenience.
>> >> >>
>> >> >> The xmlproto.html is much better with a locally built x3dom
>> (gh-pages branch) than the remote netlify version. You might want to
>> update the netlify version?
>> >> >>
>> >> >> Note the presence of a Script in the flowerproto. I'm not
>> expecting that to work yet. It might be fun to get it working, though,
>> which I have kind of done:
>> >> >> https://coderextreme.net/X3DJSONLD/src/main/html/proto.html
>> (select ../data/x3domflowers.x3d).
>> >> >>
>> >> >> If you've got a script instance per flower from your proto
>> implementation, it might be possible. One just needs to rename
>> "scripts"/implement the routes to and from "scripts" (whatever you want to
>> call them), I am pretty sure. This becomes more and more doable, I am
>> thinking now, thanks to your effort with Protos.
>> >> >>
>> >> >> Don, can you add a check to the X3dToJson.xslt to throw a warning
>> if a proto declare name is the same as a tag?
>> >> >>
>> >> >> Thanks,
>> >> >>
>> >> >> John
>> >> >>
>> >> >> On Sun, Jun 28, 2020 at 12:28 AM Andreas Plesch <
>> andreasplesch at gmail.com> wrote:
>> >> >>>
>> >> >>> Thanks, very helpful. Two issues came up. Since 'sphere' is a name
>> of
>> >> >>> a regular node, but then was registered as a new proto node, things
>> >> >>> broke. Not sure what to do about it, maybe just documenting. HTML
>> has
>> >> >>> a requirement for names of custom nodes to avoid such conflicts. I
>> >> >>> renamed the proto to protosphere which fixed the scene. rubikOnFire
>> >> >>> was interesting because it is the only example which has an IS
>> >> >>> connection to a node field of a ProtoInstance. I found a workaround
>> >> >>> which should work most of the time. rubikFurnace does not work, it
>> >> >>> shows just the default spheres, not sure. x-ite has the same
>> problem
>> >> >>> with it, so maybe there is a deeper issue although I think the x3d
>> >> >>> looks ok.
>> >> >>>
>> >> >>>
>> https://andreasplesch.github.io/x3dom/test/functional/proto/inline.html
>> >> >>>
>> >> >>> PS: I started to use a chromebook and I think x-ite and x3dom are
>> the
>> >> >>> only x3d browsers for this platform. I looks like freeWrl for
>> android
>> >> >>> would need to be updated to work on it. I am getting used to the
>> >> >>> touchscreen,
>> >> >>>
>> >> >>> On Sat, Jun 27, 2020 at 9:38 PM John Carlson <yottzumm at gmail.com>
>> wrote:
>> >> >>> >
>> >> >>> > Here is an example to try:
>> >> >>> >
>> https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/data/rubik.x3d
>> >> >>> >
>> >> >>> > Other rubik*.x3d examples in same folder may be useful too, but
>> I can no longer remember all the differences. I know all shapes should be
>> the same in the result, cylinder results are not correct and there are 27
>> shapes in the result.
>> >> >>> >
>> >> >>> > The result of the one in the email should be 27 spheres.
>> >> >>> >
>> >> >>> > John
>> >> >>> >
>> >> >>> > On Sat, Jun 27, 2020 at 6:23 PM Andreas Plesch <
>> andreasplesch at gmail.com> wrote:
>> >> >>> >>
>> >> >>> >> I also start to think the main reason for the ExternProto
>> fields is to
>> >> >>> >> enable easier and more performant loading by browsers, using a
>> >> >>> >> template and fill in the details later approach.
>> >> >>> >>
>> >> >>> >> I expanded my working example list to a satisfactory number for
>> x3dom
>> >> >>> >> and will start to clean up and refactor a bit. Almost each
>> example
>> >> >>> >> needed additional attention to the processing so no doubt there
>> are
>> >> >>> >> gaps in coverage which soon will be discovered by actual usage.
>> But as
>> >> >>> >> long as the complexity in terms of nesting and async. loading
>> does not
>> >> >>> >> exceed the examples, the behaviour should be fairly robust.
>> The #name
>> >> >>> >> syntax works. The helicopter (Example16) is fairly complex and
>> works
>> >> >>> >> now, after replacing the script with event utilities. The
>> LogoLetter
>> >> >>> >> example unearthed another interesting bug which triggered
>> exponential
>> >> >>> >> doubling of shapes. Some castle engine examples stress the
>> limits,
>> >> >>> >> mostly by redefining DEFs (usually a no go) but do something
>> >> >>> >> reasonable now.
>> >> >>> >>
>> >> >>> >> The approach taken is to register actual new node types (which
>> >> >>> >> internally use other nodes) and then use more or less the
>> regular node
>> >> >>> >> creation and instancing for the ProtoInstances, after
>> converting them
>> >> >>> >> to a more readable syntax. I think this works as well as
>> expanding
>> >> >>> >> templates and feels more natural but tends to uncover implicit
>> >> >>> >> assumptions in the code. For example, x3dom assumes that
>> Material as a
>> >> >>> >> X3DAppearanceChild node is always a child of an Appearance
>> node. With
>> >> >>> >> protos, it can be a child of another node as well. So I had to
>> >> >>> >> eventually start to use a try/catch clause.
>> >> >>> >>
>> >> >>> >> Thanks for maintaining the example, they are critical to get
>> uniform behaviours.
>> >> >>> >>
>> >> >>> >> Here are the updated working examples:
>> >> >>> >>
>> >> >>> >>
>> https://andreasplesch.github.io/x3dom/test/functional/proto/inline.html
>> >> >>> >>
>> >> >>> >> I may be interested in trying a few more examples without
>> script nodes
>> >> >>> >> but I think these are a good selection.
>> >> >>> >>
>> >> >>> >> Any feedback welcome,
>> >> >>> >>
>> >> >>> >> -Andreas
>> >> >>> >>
>> >> >>> >> On Fri, Jun 26, 2020 at 2:09 PM Don Brutzman <brutzman at nps.edu>
>> wrote:
>> >> >>> >> >
>> >> >>> >> > On 6/26/2020 10:50 AM, Andreas Plesch wrote:
>> >> >>> >> > >
>> >> >>> >> > > Thanks for thinking this through. I am not seeing any
>> inconsistencies,
>> >> >>> >> > > only redundancies which could invite authoring errors in
>> the first
>> >> >>> >> > > place.
>> >> >>> >> > >
>> >> >>> >> > > I think for now, x3dom will have to go by the garbage in,
>> garbage out
>> >> >>> >> > > principle, meaning that inconsistent field statements may
>> cause
>> >> >>> >> > > problems. The spec. actually requires consistent naming.
>> >> >>> >> >
>> >> >>> >> > agreed
>> >> >>> >> >
>> >> >>> >> > > On Fri, Jun 26, 2020 at 10:06 AM Don Brutzman <
>> brutzman at nps.edu> wrote:
>> >> >>> >> > >>
>> >> >>> >> > >> Checking ProtoDeclare and ExternProtoDeclare can be
>> tricky, but I think it is correctly defined.
>> >> >>> >> > >>
>> >> >>> >> > >> My understanding of the intent of that section was to
>> prevent unexpected errors in the case of
>> >> >>> >> > >>
>> >> >>> >> > >> a. ProtoDeclare defined,
>> >> >>> >> > >> b. ExternProtoDeclare and ProtoInstance example work and
>> are deployed,
>> >> >>> >> > >> c. ProtoDeclare subsequently adds some additional fields
>> or changes default field values,
>> >> >>> >> > >
>> >> >>> >> > > How would that happen ? Externally, by editing the
>> ProtoDeclare in the
>> >> >>> >> > > referenced file ? That would seem like a situation which
>> should not be
>> >> >>> >> > > in the scope of x3d.
>> >> >>> >> >
>> >> >>> >> > correct. have seen this occur with long-term re-use of
>> valuable prototypes that continue to evolve, it is important to find
>> external instances or modify/evolve them with backwards compatibility in
>> mind.
>> >> >>> >> >
>> >> >>> >> > >> d. ExternProtoDeclare and ProtoInstance example still work
>> OK though new ProtoDeclare is retrieved at runtime.
>> >> >>> >> > >
>> >> >>> >> > > Hm, is there a requirement to reload already loaded
>> ProtoDeclare's
>> >> >>> >> > > when a new ProtoInstance is added to a scene ?
>> >> >>> >> >
>> >> >>> >> > no, that would be dangerous/unexpected. no hidden
>> dependencies here, just stepping through typical use.
>> >> >>> >> >
>> >> >>> >> > >> Certainly the browser loading the original/updated
>> ProtoDeclare must honor the behavior defined therein, including default
>> values.
>> >> >>> >> > >>
>> >> >>> >> > >> If the field interfaces within the ExternProtoDeclare
>> (which only contain name, type, accessType and not default values) are
>> different, that would be an error.
>> >> >>> >> > >
>> >> >>> >> > > yes, exactly, so why have those field interfaces ?
>> >> >>> >> >
>> >> >>> >> > Having ExternProtoDeclare allows a browser to load and set up
>> all connections with type information in mind, allowing remote loading of
>> ProtoDeclare to occur in parallel. Thus performance speedup.
>> >> >>> >> >
>> >> >>> >> > >> As above, if default values within the ProtoDeclare
>> change, this has no impact on ExternProtoDeclare field definitions because
>> they do not list default values.
>> >> >>> >> > >
>> >> >>> >> > > I am not sure how the default values could change.
>> >> >>> >> >
>> >> >>> >> > long-term evolution of a published prototype in a library,
>> for example.
>> >> >>> >> >
>> >> >>> >> > >> When a ProtoInstance provides fieldValue initializations,
>> they of course supersede whatever the default might be in the ProtoDeclare.
>> >> >>> >> > >>
>> >> >>> >> > >> ... so I think this all hangs together cleanly without
>> contradiction or ambiguity.
>> >> >>> >> > >
>> >> >>> >> > > Agreed, just potentially confusing redundancy.
>> >> >>> >> >
>> >> >>> >> > it takes some practice to get familiar since the capabilities
>> are powerful.
>> >> >>> >> >
>> >> >>> >> > >> Implementation-support notes:
>> >> >>> >> > >> - InstantReality handles cases well, although console
>> warnings sometimes include false positives.
>> >> >>> >> > >
>> >> >>> >> > > I am using view3dscene and freeWrl for testing. Most
>> examples work
>> >> >>> >> > > well though freeWrl seems to have a problem with the nested
>> spin group
>> >> >>> >> > > prototype example.
>> >> >>> >> >
>> >> >>> >> > As an author I avoid nested prototypes, they seem less robust
>> and more likely to fail.
>> >> >>> >> >
>> >> >>> >> > >> - X3D-Edit has a feature to check ExternProtoDeclare
>> interfaces against ProtoDeclare interfaces.
>> >> >>> >> > >> - Utility methods for such checking would be a good
>> feature to add to our Java, Python and JavaScript libraries.
>> >> >>> >> > >>
>> >> >>> >> > >> Loading and checking for such consistency is typically not
>> performed by any of our Quality Assurance (QA) tools since they tend to
>> perform validations in an offline manner. For X3DOM, I think this gap in
>> testing coverage means that you should carefully check for consistency
>> because if ProtoDeclare and ExternProtoDeclare differ then an incompatible
>> interface is expected and model errors are likely.
>> >> >>> >> > >>
>> >> >>> >> > >> Improvements always welcome. Thanks for close scrutiny
>> and thanks for tackling this super valuable capability for X3DOM.
>> >> >>> >> > >
>> >> >>> >> > > You can follow progress here:
>> >> >>> >> > >
>> >> >>> >> > >
>> https://andreasplesch.github.io/x3dom/test/functional/proto/inline.html#Front
>> >> >>> >> >
>> >> >>> >> > impressive setup
>> >> >>> >> >
>> >> >>> >> > > It will be interesting to see how Protos can be used in
>> combination
>> >> >>> >> > > with web js based templating.
>> >> >>> >> > >
>> >> >>> >> > > -Andreas
>> >> >>> >> >
>> >> >>> >> > thanks for taking the time to get this part right now. that
>> will make future HTML5-X3D4 patterns a lot more stable and understandable.
>> >> >>> >> >
>> >> >>> >> > >> On 6/23/2020 6:10 PM, Andreas Plesch wrote:
>> >> >>> >> > >>> ...
>> >> >>> >> > >>>> The next step would be to support the ExternProtoDeclare
>> statement.
>> >> >>> >> > >>>> The main question I have is about the function of the
>> additional field
>> >> >>> >> > >>>> statements under ExternProtoDeclare.
>> >> >>> >> > >>>>
>> >> >>> >> > >>>> - Do they replace ProtoInterface field statements ? (No.)
>> >> >>> >> > >>>> - Is the ProtoInterface element still required in the
>> external file ? (Yes.)
>> >> >>> >> > >>>> - Are they listed just for convenience (for the author
>> and the browser) ? (Yes?)
>> >> >>> >> > >>>> - Can they be ignored ? (Yes?)
>> >> >>> >> > >>>
>> >> >>> >> > >>> I did find the clause "The names and types of the fields
>> of the
>> >> >>> >> > >>> interface declaration shall be a subset of those defined
>> in the
>> >> >>> >> > >>> implementation." in 4.4.5.2 EXTERNPROTO interface
>> semantics. This
>> >> >>> >> > >>> means that an ExternProto can restrict access to fields
>> by not listing
>> >> >>> >> > >>> them in its field elements. So they should not be
>> ignored. On the
>> >> >>> >> > >>> other hand a browser which ignores them would still
>> generate the same
>> >> >>> >> > >>> behaviour, minus warnings or errors in an ill-constructed
>> scene when a
>> >> >>> >> > >>> ProtoInstance is trying to set non-accessible fields.
>> >> >>> >> > >>>
>> >> >>> >> > >>> So I think as a first cut, it is ok to just load the
>> external
>> >> >>> >> > >>> Protodeclaration and give it the name of the ExternProto
>> and not doing
>> >> >>> >> > >>> much or anything with the field elements.
>> >> >>> >> > >>>
>> >> >>> >> > >>>> Thanks for any insight,
>> >> >>> >> > >>>>
>> >> >>> >> > >>>> -Andreas
>> >> >>> >> >
>> >> >>> >> >
>> >> >>> >> > all the best, Don
>> >> >>> >> > --
>> >> >>> >> > Don Brutzman Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> >> >>> >> > Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>> >> >>> >> > X3D graphics, virtual worlds, navy robotics
>> http://faculty.nps.edu/brutzman
>> >> >>> >>
>> >> >>> >>
>> >> >>> >>
>> >> >>> >> --
>> >> >>> >> Andreas Plesch
>> >> >>> >> Waltham, MA 02453
>> >> >>> >>
>> >> >>> >> _______________________________________________
>> >> >>> >> x3d-public mailing list
>> >> >>> >> x3d-public at web3d.org
>> >> >>> >> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Andreas Plesch
>> >> >>> Waltham, MA 02453
>> >>
>> >>
>> >>
>> >> --
>> >> Andreas Plesch
>> >> Waltham, MA 02453
>>
>>
>>
>> --
>> Andreas Plesch
>> Waltham, MA 02453
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200629/56b90357/attachment-0001.html>
More information about the x3d-public
mailing list