[x3d-public] Possible issue with X3dToPython.xslt, newly tested file from X3D4WA: need x3d.py python support for Script CDATA
John Carlson
yottzumm at gmail.com
Sat Jan 15 13:06:26 PST 2022
It’s higher priority for me to fix the X3dToJson.xslt so that
Navigation.type produces a consistent type of data. At this time, I don’t
know if it produces a JSON string or a JSON array. I believe the JSON
Schema says array. If we want it to be a string, okay, but can we change
all other MFStrings to output JSON strings in X3dToJson.xslt?
Maybe I missed an email on this?
John
On Sat, Jan 15, 2022 at 2:52 PM John Carlson <yottzumm at gmail.com> wrote:
> It’s just that people may already have JSON files working with X_ITE. We
> will have to go through regression testing etc. We don’t have scripting
> with X3DOM working yet except on some of my published work. And even some
> of that is not working. AFAIK, all my scripts work ok in X_ITE. So I
> would encourage us to tread carefully when modifying JSONParser.js in
> X_ITE, and provide for a deprecation period.
>
> I don’t have to retest my shaders—they are in external files.
>
> If we decide to force a change to X3D JSON Schema, I will probably slowly
> deprecate #sourceText in favor of sourceCode, but I doubt if all this will
> happen overnight. My surgery was postponed and I want to minimize impact
> on my wrists right now. That is, I will support sourceCode and sourceText
> in parallel. I will have to figure out how the new X3DUOM impacts all my
> python generators as well.
>
> I’m guessing you don’t actually have any Scripting working on those older
> products you mention, so it didn’t really impact you until you wanted to do
> python+scripting in Jupyter.
>
> I will look a bit into X3DUOM changes to see how much my other code is
> impacted, or can be enhanced.
>
> There’s also probably X3DJSONLD.java and X3DJSONLD.cpp that need to be
> changed.
>
> I would encourage maintaining at least a synonym “#sourceText” for
> sourceCode.
>
> We should probably create a plan for updating JSONParser.js in X_ITE.
>
> I’m not really sure there will ever be a JSON encoding for X3D, but I’m
> glad X3dToJson.xslt is being maintained as I don’t have a good replacement
> yet.
>
> “someday” is today for me. What will it take to make X3D JSON a
> standard? We already have two working implementations, AFAIK.
>
> John
>
> On Sat, Jan 15, 2022 at 1:12 PM Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
>> I’ve got X3DUOM and Java working on my machine. Python, X3D Ontology and
>> Tooltips next.
>>
>>
>>
>> John, can we please change the JSON encoding to use “sourceCode” instead
>> of sourceText for consistency? I don’t think there is any need for
>> synonyms or backwards compatibility, the encoding hasn’t been formalized
>> yet and (someday) JSON Schema will help people confirm correctness.
>>
>>
>>
>> - X3D to JSON Stylesheet Converter: Data Types
>> - https://www.web3d.org/x3d/stylesheets/X3dToJson.html#DataTypes
>>
>>
>>
>> Embedded source code for
>> Script <https://www.web3d.org/x3d/content/X3dTooltips.html#Script>,
>> ShaderPart
>> <https://www.web3d.org/x3d/content/X3dTooltips.html#ShaderPart> and
>> ShaderProgram
>> <https://www.web3d.org/x3d/content/X3dTooltips.html#ShaderProgram> nodes
>>
>> CDATA (Character DATA) section
>> <[CDATA[ "world wild Web!" ]]>
>>
>> "#sourceText" string array containing original code, possibly escaped
>>
>>
>>
>>
>>
>> all the best, Don
>>
>> --
>>
>> Don Brutzman Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>>
>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>>
>> X3D graphics, virtual worlds, Navy robotics https://
>> faculty.nps.edu/brutzman
>>
>>
>>
>> *From:* John Carlson <yottzumm at gmail.com>
>> *Sent:* Friday, January 14, 2022 5:25 AM
>> *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> *Cc:* X3D Graphics public mailing list <x3d-public at web3d.org>
>> *Subject:* Re: Possible issue with X3dToPython.xslt, newly tested file
>> from X3D4WA: need x3d.py python support for Script CDATA
>>
>>
>>
>> I finally got a chance to think about this, and I think it will be good
>> to support legacy files by providing a synonym for
>> “#sourceText”—“sourceCode” as you say. If a particular binding can’t
>> provide for #sourceText, then sourceCode can be used as a fallback.
>>
>>
>>
>> We can discuss changes to the JSON WD Draft at some point.
>>
>>
>>
>> We will have to update X3D JSON schema as previously discussed implicitly.
>>
>>
>>
>> John
>>
>>
>>
>> On Tue, Jan 11, 2022 at 1:03 AM Brutzman, Donald (Don) (CIV) <
>> brutzman at nps.edu> wrote:
>>
>> 1. Following some initial testing, am happy to report that this approach
>> to embed source code in Script nodes seems consistently feasible for
>> X3DUOM, Python, Java and X3D Ontology all at once.
>>
>> It will take a little work to fully implement, but it should improve the
>> expressive power (meaning "fix a hole") in each of our programming-language
>> bindings.
>>
>> ------
>>
>> 2. In some sense embedding script code is already possible - you can put
>> such a program in first string contained in a Script url list - but in
>> practice that tends to be quite difficult because then all sorts of
>> obfuscating escaping of special characters has to take place.
>>
>> So having a separate place for source-code blocks is very helpful. The
>> X3D Architecture recognizes this possibility and defines corresponding
>> behaviors. Further details can be found in X3D XML Encoding.
>>
>> * X3D4 Architecture, clause 19 Scripting component, 29.4.1 Script
>> *
>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/scripting.html#Script
>>
>> * X3D4 Architecture, 9.2.3 Scripting language protocols
>>
>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/networking.html#ScriptingLanguageProtocols
>>
>> * X3D XML Encoding, Clause 4 Concepts, 4.3.13 Encapsulating Script node
>> code
>>
>> https://www.web3d.org/documents/specifications/19776-1/V3.3/Part01/concepts.html#EncapsulatingScriptNodeCode
>>
>> "If both a url field and a CDATA clause are encountered, the url field
>> is processed first. Thus, the CDATA construct can also be considered
>> equivalent to one additional value appended to the url MFString array. This
>> ordering allows an online script code url to take priority over fallback
>> default script code in the CDATA construct. This ordering also allows
>> run-time updates if a viewer is connected to the network, if so desired by
>> the originating author."
>>
>> ------
>>
>> 3. Next - naming. "CDATA" is pretty obscure as a field name, especially
>> since that is an XML datatype (for Character Data). Thus am thinking
>> "sourceCode" might be a better name... what do you think?
>>
>> The ability to embed safe scripting code within any X3D Script is a
>> powerful capability that also improves security possibilities for advanced
>> X3D... it is easier to secure and safely transport an embedded script than
>> it is to have multiple files handled safely.
>>
>> John, it looks like we already did something just like this in the JSON
>> encoding, we called it "sourceText", excerpt follows. (Excessive string
>> quoting was necessary to pass JSON parsing rules for multiline text.)
>>
>>
>> ----------------------------------------------------------------------------------------------------------------------------------------------
>> *
>> https://X3dGraphics.com/examples/X3dForWebAuthors/Chapter14Prototypes/MaterialModulator.json
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fx3dgraphics.com%2Fexamples%2FX3dForWebAuthors%2FChapter14Prototypes%2FMaterialModulator.json&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nBWBGWktMkBVA9XkqojDDqLZFbYrEIb3%2FGYH0za3FG0%3D&reserved=0>
>>
>> { "Script":
>> {
>> [....]
>> "#sourceText":[
>> "",
>> "",
>> "ecmascript:",
>> "function initialize ()",
>> "{",
>> " newColor = diffuseColor; // start with original color",
>> "}",
>> "function clockTrigger (timeValue)",
>> "{",
>> " if (!enabled) return;",
>> " red = newColor.r;",
>> " green = newColor.g;",
>> " blue = newColor.b;",
>> "",
>> " // note different modulation rates for each color component, % is
>> modulus operator",
>> " newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue
>> + 0.04) % 1);",
>> "\tif (enabled)",
>> "\t{",
>> "\t\tBrowser.print ('diffuseColor=(' + red +',' + green + ',' + blue + ')
>> newColor=' + newColor.toString() + '\n');",
>> "\t}",
>> "}",
>> "function set_enabled (newValue)",
>> "{",
>> "\tenabled = newValue;",
>> "}",
>> "",
>> ""
>> ]
>>
>> ----------------------------------------------------------------------------------------------------------------------------------------------
>> Which in the .x3d XML CDATA simply looks like
>>
>> <Script DEF='MaterialModulatorScript'>
>> <field accessType='inputOutput' name='enabled' type='SFBool'/>
>> <field accessType='inputOutput' name='diffuseColor'
>> type='SFColor'/>
>> <field accessType='outputOnly' name='newColor' type='SFColor'/>
>> <field accessType='inputOnly' name='clockTrigger'
>> type='SFTime'/>
>> <IS>
>> <connect nodeField='enabled' protoField='enabled'/>
>> <connect nodeField='diffuseColor' protoField='diffuseColor'/>
>> </IS>
>> <![CDATA[
>> ecmascript:
>> function initialize ()
>> {
>> newColor = diffuseColor; // start with original color
>> }
>> function clockTrigger (timeValue)
>> {
>> if (!enabled) return;
>> red = newColor.r;
>> green = newColor.g;
>> blue = newColor.b;
>>
>> // note different modulation rates for each color component, % is
>> modulus operator
>> newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue +
>> 0.04) % 1);
>> if (enabled)
>> {
>> Browser.print ('diffuseColor=(' + red +',' + green + ','
>> + blue + ') newColor=' + newColor.toString() + '\n');
>> }
>> }
>> function set_enabled (newValue)
>> {
>> enabled = newValue;
>> }
>> ]]>
>> </Script>
>>
>>
>> ----------------------------------------------------------------------------------------------------------------------------------------------
>>
>> So this possibility is a cool prospect, again thanks for flagging it.
>> I'll work further on X3DUOM and Python, Java, OWL compatibility next.
>>
>> Have fun with X3D! 8)
>>
>> all the best, Don
>> --
>> Don Brutzman Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics https://
>> faculty.nps.edu/brutzman
>>
>> -----Original Message-----
>> From: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>> Sent: Monday, January 10, 2022 6:37 PM
>> To: John Carlson <yottzumm at gmail.com>
>> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>; Brutzman,
>> Donald (Don) (CIV) <brutzman at nps.edu>
>> Subject: RE: Possible issue with X3dToPython.xslt, newly tested file from
>> X3D4WA: need x3d.py python support for Script CDATA
>>
>> Thanks for the interesting trouble report John.
>>
>> No changes needed to original X3D model, it passes all X3D Validator
>> tests (though X3D Schematron provides a few warnings).
>>
>> *
>> https://X3dGraphics.com/examples/X3dForWebAuthors/Chapter14Prototypes/MaterialModulator.x3d
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fx3dgraphics.com%2Fexamples%2FX3dForWebAuthors%2FChapter14Prototypes%2FMaterialModulator.x3d&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A57o96EWq%2B5YUt%2FtG81jPRexgtCi4BvN%2BnhpTrgV4OQ%3D&reserved=0>
>> * https://savage.nps.edu/X3dValidator
>>
>> XMLSpy was stricter about type handling than our Java-based Saxon10
>> conversions for this example. XMLSpy diagnosed some problems that I fixed.
>>
>> After also fixing the stylesheet name provided by a diagnostic, all
>> X3dToPython.xslt changes were checked into subversion.
>>
>> I now get the following conversion error:
>>
>> =====================
>> create python:
>> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors/Chapter14Prototypes//MaterialModulator.x3d
>> processing with X3dToPython stylesheet...
>> *** TODO x3d.py and X3dToPython.xslt need to handle embedded CDATA source
>> code for Script C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors/Chapter14Prototypes//MaterialModulator.py
>> self-validation tests...
>> validate python:
>> File "C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\Chapter14Prototypes\MaterialModulator.py",
>> line 74
>> *** TODO x3d.py and X3dToPython.xslt need to handle embedded CDATA
>> source code for Script
>> ^
>> SyntaxError: invalid syntax
>> Result: 1
>> ===================================
>>
>> So, embedded CDATA scripting code inside a Script node (and similar
>> Shader nodes) remains an omission in our Python support.
>>
>> To remedy this, we need to create a special field in the X3DPSAIL class
>> Script. *What shall we call it?* Creating a Script member named CDATA
>> seems unambiguous and eponymous with the XML encoding...
>>
>> * X3D Tooltips, Script and Script url
>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#Script
>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#Script.url
>>
>> * X3D Tooltips, XML Data Types
>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#XML
>>
>> " CDATA is an XML term for Character Data. The base type for all XML
>> attributes consists of string-based CDATA characters. CDATA is used
>> throughout the X3D DOCTYPE definitions, which can only check for the
>> presence of legal strings and thus are not able to validate numeric type
>> information. XML Schema provides stricter validation based on data types."
>>
>> * XML, 2.4 Character Data and Markup
>> https://www.w3.org/TR/REC-xml/#syntax
>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2FREC-xml%2F%23syntax&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=biLfaypaJLVx%2BbLUJycGeGJcGNvsOlc%2FV0NtjL7vNHA%3D&reserved=0>
>>
>> all the best, Don
>> --
>> Don Brutzman Naval Postgraduate School, Code USW/Br
>> brutzman at nps.edu
>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>> +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics https://
>> faculty.nps.edu/brutzman
>>
>> -----Original Message-----
>> From: John Carlson <yottzumm at gmail.com>
>> Sent: Monday, January 10, 2022 4:06 PM
>> To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; X3D Graphics public
>> mailing list <x3d-public at web3d.org>
>> Subject: Possible issue with X3dToPython.xslt, newly tested file from
>> X3D4WA
>>
>> X3D XML file attached. From X3DJSONLD's maven build log:
>>
>> [INFO] --- xml-maven-plugin:1.0.2:transform (default) @ X3DJSONLD ---
>> [INFO] Transforming file:
>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulatorExamples.x3d
>> [INFO] Transforming file:
>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulator.x3d
>> Type error at char 37 in xsl:value-of/@select on line 610 column 120 of
>> X3dToPython.xslt:
>> XPTY0004: A sequence of more than one item is not allowed as the first
>> argument of
>> normalize-space() ("", "", ...)
>> at xsl:apply-templates
>>
>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>> processing /X3D/Scene[1]/ProtoDeclare[1]/ProtoBody[1]/Script[1]
>> at xsl:apply-templates
>>
>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>> processing /X3D/Scene[1]/ProtoDeclare[1]/ProtoBody[1]
>> at xsl:apply-templates
>>
>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>> processing /X3D/Scene[1]/ProtoDeclare[1]
>> at xsl:apply-templates
>>
>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>> processing /X3D/Scene[1]
>> at xsl:apply-templates
>>
>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#123)
>> processing /X3D
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 2.840 s
>> [INFO] Finished at: 2022-01-10T17:59:22-06:00 [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.codehaus.mojo:xml-maven-plugin:1.0.2:transform (default) on project
>> X3DJSONLD: Failed to transform input file
>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulator.x3d: A
>> sequence of more than one item is not allowed as the first argument of
>> normalize-space() ("", "", ...) -> [Help 1] [ERROR] [ERROR] To see the
>> full stack trace of the errors, re-run Maven with the -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>>
>> https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FMAVEN%2FMojoExecutionException&data=04%7C01%7Cbrutzman%40nps.edu%7C79b6d0212127425f31b408d9d4ab3773%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637774654099751000%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VYVojdsW4QsyjrkZGnqfPOPmUp7x5ftTGE3H0TzrTts%3D&reserved=0
>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FMAVEN%2FMojoExecutionException&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=EVAYH1SE%2FC6vXPYV9v0Zi5rmvMiki8JEsb2CwoyP6XQ%3D&reserved=0>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220115/3ef03e1c/attachment-0001.html>
More information about the x3d-public
mailing list