[x3d-public] Possible issue with X3dToPython.xslt, newly tested file from X3D4WA: need x3d.py python support for Script CDATA
John Carlson
yottzumm at gmail.com
Sat Jan 15 13:12:21 PST 2022
In other words, let’s fix known problems before creating new ones.
Thanks!
John
On Sat, Jan 15, 2022 at 3:06 PM John Carlson <yottzumm at gmail.com> wrote:
> It’s higher priority for me to fix the X3dToJson.xslt so that
> Navigation.type produces a consistent type of data. At this time, I don’t
> know if it produces a JSON string or a JSON array. I believe the JSON
> Schema says array. If we want it to be a string, okay, but can we change
> all other MFStrings to output JSON strings in X3dToJson.xslt?
>
> Maybe I missed an email on this?
>
> John
>
> On Sat, Jan 15, 2022 at 2:52 PM John Carlson <yottzumm at gmail.com> wrote:
>
>> It’s just that people may already have JSON files working with X_ITE.
>> We will have to go through regression testing etc. We don’t have
>> scripting with X3DOM working yet except on some of my published work. And
>> even some of that is not working. AFAIK, all my scripts work ok in X_ITE.
>> So I would encourage us to tread carefully when modifying JSONParser.js
>> in X_ITE, and provide for a deprecation period.
>>
>> I don’t have to retest my shaders—they are in external files.
>>
>> If we decide to force a change to X3D JSON Schema, I will probably slowly
>> deprecate #sourceText in favor of sourceCode, but I doubt if all this will
>> happen overnight. My surgery was postponed and I want to minimize impact
>> on my wrists right now. That is, I will support sourceCode and sourceText
>> in parallel. I will have to figure out how the new X3DUOM impacts all my
>> python generators as well.
>>
>> I’m guessing you don’t actually have any Scripting working on those older
>> products you mention, so it didn’t really impact you until you wanted to do
>> python+scripting in Jupyter.
>>
>> I will look a bit into X3DUOM changes to see how much my other code is
>> impacted, or can be enhanced.
>>
>> There’s also probably X3DJSONLD.java and X3DJSONLD.cpp that need to be
>> changed.
>>
>> I would encourage maintaining at least a synonym “#sourceText” for
>> sourceCode.
>>
>> We should probably create a plan for updating JSONParser.js in X_ITE.
>>
>> I’m not really sure there will ever be a JSON encoding for X3D, but I’m
>> glad X3dToJson.xslt is being maintained as I don’t have a good replacement
>> yet.
>>
>> “someday” is today for me. What will it take to make X3D JSON a
>> standard? We already have two working implementations, AFAIK.
>>
>> John
>>
>> On Sat, Jan 15, 2022 at 1:12 PM Brutzman, Donald (Don) (CIV) <
>> brutzman at nps.edu> wrote:
>>
>>> I’ve got X3DUOM and Java working on my machine. Python, X3D Ontology
>>> and Tooltips next.
>>>
>>>
>>>
>>> John, can we please change the JSON encoding to use “sourceCode”
>>> instead of sourceText for consistency? I don’t think there is any need
>>> for synonyms or backwards compatibility, the encoding hasn’t been
>>> formalized yet and (someday) JSON Schema will help people confirm
>>> correctness.
>>>
>>>
>>>
>>> - X3D to JSON Stylesheet Converter: Data Types
>>> - https://www.web3d.org/x3d/stylesheets/X3dToJson.html#DataTypes
>>>
>>>
>>>
>>> Embedded source code for
>>> Script <https://www.web3d.org/x3d/content/X3dTooltips.html#Script>,
>>> ShaderPart
>>> <https://www.web3d.org/x3d/content/X3dTooltips.html#ShaderPart> and
>>> ShaderProgram
>>> <https://www.web3d.org/x3d/content/X3dTooltips.html#ShaderProgram> nodes
>>>
>>> CDATA (Character DATA) section
>>> <[CDATA[ "world wild Web!" ]]>
>>>
>>> "#sourceText" string array containing original code, possibly escaped
>>>
>>>
>>>
>>>
>>>
>>> all the best, Don
>>>
>>> --
>>>
>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>>
>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>>
>>> X3D graphics, virtual worlds, Navy robotics https://
>>> faculty.nps.edu/brutzman
>>>
>>>
>>>
>>> *From:* John Carlson <yottzumm at gmail.com>
>>> *Sent:* Friday, January 14, 2022 5:25 AM
>>> *To:* Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>>> *Cc:* X3D Graphics public mailing list <x3d-public at web3d.org>
>>> *Subject:* Re: Possible issue with X3dToPython.xslt, newly tested file
>>> from X3D4WA: need x3d.py python support for Script CDATA
>>>
>>>
>>>
>>> I finally got a chance to think about this, and I think it will be good
>>> to support legacy files by providing a synonym for
>>> “#sourceText”—“sourceCode” as you say. If a particular binding can’t
>>> provide for #sourceText, then sourceCode can be used as a fallback.
>>>
>>>
>>>
>>> We can discuss changes to the JSON WD Draft at some point.
>>>
>>>
>>>
>>> We will have to update X3D JSON schema as previously discussed
>>> implicitly.
>>>
>>>
>>>
>>> John
>>>
>>>
>>>
>>> On Tue, Jan 11, 2022 at 1:03 AM Brutzman, Donald (Don) (CIV) <
>>> brutzman at nps.edu> wrote:
>>>
>>> 1. Following some initial testing, am happy to report that this approach
>>> to embed source code in Script nodes seems consistently feasible for
>>> X3DUOM, Python, Java and X3D Ontology all at once.
>>>
>>> It will take a little work to fully implement, but it should improve the
>>> expressive power (meaning "fix a hole") in each of our programming-language
>>> bindings.
>>>
>>> ------
>>>
>>> 2. In some sense embedding script code is already possible - you can put
>>> such a program in first string contained in a Script url list - but in
>>> practice that tends to be quite difficult because then all sorts of
>>> obfuscating escaping of special characters has to take place.
>>>
>>> So having a separate place for source-code blocks is very helpful. The
>>> X3D Architecture recognizes this possibility and defines corresponding
>>> behaviors. Further details can be found in X3D XML Encoding.
>>>
>>> * X3D4 Architecture, clause 19 Scripting component, 29.4.1 Script
>>> *
>>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/scripting.html#Script
>>>
>>> * X3D4 Architecture, 9.2.3 Scripting language protocols
>>>
>>> https://www.web3d.org/specifications/X3Dv4Draft/ISO-IEC19775-1v4-CD1/Part01/components/networking.html#ScriptingLanguageProtocols
>>>
>>> * X3D XML Encoding, Clause 4 Concepts, 4.3.13 Encapsulating Script node
>>> code
>>>
>>> https://www.web3d.org/documents/specifications/19776-1/V3.3/Part01/concepts.html#EncapsulatingScriptNodeCode
>>>
>>> "If both a url field and a CDATA clause are encountered, the url
>>> field is processed first. Thus, the CDATA construct can also be considered
>>> equivalent to one additional value appended to the url MFString array. This
>>> ordering allows an online script code url to take priority over fallback
>>> default script code in the CDATA construct. This ordering also allows
>>> run-time updates if a viewer is connected to the network, if so desired by
>>> the originating author."
>>>
>>> ------
>>>
>>> 3. Next - naming. "CDATA" is pretty obscure as a field name, especially
>>> since that is an XML datatype (for Character Data). Thus am thinking
>>> "sourceCode" might be a better name... what do you think?
>>>
>>> The ability to embed safe scripting code within any X3D Script is a
>>> powerful capability that also improves security possibilities for advanced
>>> X3D... it is easier to secure and safely transport an embedded script than
>>> it is to have multiple files handled safely.
>>>
>>> John, it looks like we already did something just like this in the JSON
>>> encoding, we called it "sourceText", excerpt follows. (Excessive string
>>> quoting was necessary to pass JSON parsing rules for multiline text.)
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------------------
>>> *
>>> https://X3dGraphics.com/examples/X3dForWebAuthors/Chapter14Prototypes/MaterialModulator.json
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fx3dgraphics.com%2Fexamples%2FX3dForWebAuthors%2FChapter14Prototypes%2FMaterialModulator.json&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=nBWBGWktMkBVA9XkqojDDqLZFbYrEIb3%2FGYH0za3FG0%3D&reserved=0>
>>>
>>> { "Script":
>>> {
>>> [....]
>>> "#sourceText":[
>>> "",
>>> "",
>>> "ecmascript:",
>>> "function initialize ()",
>>> "{",
>>> " newColor = diffuseColor; // start with original color",
>>> "}",
>>> "function clockTrigger (timeValue)",
>>> "{",
>>> " if (!enabled) return;",
>>> " red = newColor.r;",
>>> " green = newColor.g;",
>>> " blue = newColor.b;",
>>> "",
>>> " // note different modulation rates for each color component, % is
>>> modulus operator",
>>> " newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue
>>> + 0.04) % 1);",
>>> "\tif (enabled)",
>>> "\t{",
>>> "\t\tBrowser.print ('diffuseColor=(' + red +',' + green + ',' + blue +
>>> ') newColor=' + newColor.toString() + '\n');",
>>> "\t}",
>>> "}",
>>> "function set_enabled (newValue)",
>>> "{",
>>> "\tenabled = newValue;",
>>> "}",
>>> "",
>>> ""
>>> ]
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------------------
>>> Which in the .x3d XML CDATA simply looks like
>>>
>>> <Script DEF='MaterialModulatorScript'>
>>> <field accessType='inputOutput' name='enabled' type='SFBool'/>
>>> <field accessType='inputOutput' name='diffuseColor'
>>> type='SFColor'/>
>>> <field accessType='outputOnly' name='newColor' type='SFColor'/>
>>> <field accessType='inputOnly' name='clockTrigger'
>>> type='SFTime'/>
>>> <IS>
>>> <connect nodeField='enabled' protoField='enabled'/>
>>> <connect nodeField='diffuseColor' protoField='diffuseColor'/>
>>> </IS>
>>> <![CDATA[
>>> ecmascript:
>>> function initialize ()
>>> {
>>> newColor = diffuseColor; // start with original color
>>> }
>>> function clockTrigger (timeValue)
>>> {
>>> if (!enabled) return;
>>> red = newColor.r;
>>> green = newColor.g;
>>> blue = newColor.b;
>>>
>>> // note different modulation rates for each color component, % is
>>> modulus operator
>>> newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue
>>> + 0.04) % 1);
>>> if (enabled)
>>> {
>>> Browser.print ('diffuseColor=(' + red +',' + green + ','
>>> + blue + ') newColor=' + newColor.toString() + '\n');
>>> }
>>> }
>>> function set_enabled (newValue)
>>> {
>>> enabled = newValue;
>>> }
>>> ]]>
>>> </Script>
>>>
>>>
>>> ----------------------------------------------------------------------------------------------------------------------------------------------
>>>
>>> So this possibility is a cool prospect, again thanks for flagging it.
>>> I'll work further on X3DUOM and Python, Java, OWL compatibility next.
>>>
>>> Have fun with X3D! 8)
>>>
>>> all the best, Don
>>> --
>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>> X3D graphics, virtual worlds, navy robotics https://
>>> faculty.nps.edu/brutzman
>>>
>>> -----Original Message-----
>>> From: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
>>> Sent: Monday, January 10, 2022 6:37 PM
>>> To: John Carlson <yottzumm at gmail.com>
>>> Cc: X3D Graphics public mailing list <x3d-public at web3d.org>; Brutzman,
>>> Donald (Don) (CIV) <brutzman at nps.edu>
>>> Subject: RE: Possible issue with X3dToPython.xslt, newly tested file
>>> from X3D4WA: need x3d.py python support for Script CDATA
>>>
>>> Thanks for the interesting trouble report John.
>>>
>>> No changes needed to original X3D model, it passes all X3D Validator
>>> tests (though X3D Schematron provides a few warnings).
>>>
>>> *
>>> https://X3dGraphics.com/examples/X3dForWebAuthors/Chapter14Prototypes/MaterialModulator.x3d
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fx3dgraphics.com%2Fexamples%2FX3dForWebAuthors%2FChapter14Prototypes%2FMaterialModulator.x3d&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=A57o96EWq%2B5YUt%2FtG81jPRexgtCi4BvN%2BnhpTrgV4OQ%3D&reserved=0>
>>> * https://savage.nps.edu/X3dValidator
>>>
>>> XMLSpy was stricter about type handling than our Java-based Saxon10
>>> conversions for this example. XMLSpy diagnosed some problems that I fixed.
>>>
>>> After also fixing the stylesheet name provided by a diagnostic, all
>>> X3dToPython.xslt changes were checked into subversion.
>>>
>>> I now get the following conversion error:
>>>
>>> =====================
>>> create python:
>>> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors/Chapter14Prototypes//MaterialModulator.x3d
>>> processing with X3dToPython stylesheet...
>>> *** TODO x3d.py and X3dToPython.xslt need to handle embedded CDATA
>>> source code for Script C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors/Chapter14Prototypes//MaterialModulator.py
>>> self-validation tests...
>>> validate python:
>>> File "C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\Chapter14Prototypes\MaterialModulator.py",
>>> line 74
>>> *** TODO x3d.py and X3dToPython.xslt need to handle embedded CDATA
>>> source code for Script
>>> ^
>>> SyntaxError: invalid syntax
>>> Result: 1
>>> ===================================
>>>
>>> So, embedded CDATA scripting code inside a Script node (and similar
>>> Shader nodes) remains an omission in our Python support.
>>>
>>> To remedy this, we need to create a special field in the X3DPSAIL class
>>> Script. *What shall we call it?* Creating a Script member named CDATA
>>> seems unambiguous and eponymous with the XML encoding...
>>>
>>> * X3D Tooltips, Script and Script url
>>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#Script
>>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#Script.url
>>>
>>> * X3D Tooltips, XML Data Types
>>> https://www.web3d.org/x3d/tooltips/X3dTooltips.html#XML
>>>
>>> " CDATA is an XML term for Character Data. The base type for all XML
>>> attributes consists of string-based CDATA characters. CDATA is used
>>> throughout the X3D DOCTYPE definitions, which can only check for the
>>> presence of legal strings and thus are not able to validate numeric type
>>> information. XML Schema provides stricter validation based on data types."
>>>
>>> * XML, 2.4 Character Data and Markup
>>> https://www.w3.org/TR/REC-xml/#syntax
>>> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.w3.org%2FTR%2FREC-xml%2F%23syntax&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=biLfaypaJLVx%2BbLUJycGeGJcGNvsOlc%2FV0NtjL7vNHA%3D&reserved=0>
>>>
>>> all the best, Don
>>> --
>>> Don Brutzman Naval Postgraduate School, Code USW/Br
>>> brutzman at nps.edu
>>> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
>>> +1.831.656.2149
>>> X3D graphics, virtual worlds, navy robotics https://
>>> faculty.nps.edu/brutzman
>>>
>>> -----Original Message-----
>>> From: John Carlson <yottzumm at gmail.com>
>>> Sent: Monday, January 10, 2022 4:06 PM
>>> To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>; X3D Graphics
>>> public mailing list <x3d-public at web3d.org>
>>> Subject: Possible issue with X3dToPython.xslt, newly tested file from
>>> X3D4WA
>>>
>>> X3D XML file attached. From X3DJSONLD's maven build log:
>>>
>>> [INFO] --- xml-maven-plugin:1.0.2:transform (default) @ X3DJSONLD ---
>>> [INFO] Transforming file:
>>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulatorExamples.x3d
>>> [INFO] Transforming file:
>>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulator.x3d
>>> Type error at char 37 in xsl:value-of/@select on line 610 column 120 of
>>> X3dToPython.xslt:
>>> XPTY0004: A sequence of more than one item is not allowed as the
>>> first argument of
>>> normalize-space() ("", "", ...)
>>> at xsl:apply-templates
>>>
>>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>>> processing /X3D/Scene[1]/ProtoDeclare[1]/ProtoBody[1]/Script[1]
>>> at xsl:apply-templates
>>>
>>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>>> processing /X3D/Scene[1]/ProtoDeclare[1]/ProtoBody[1]
>>> at xsl:apply-templates
>>>
>>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>>> processing /X3D/Scene[1]/ProtoDeclare[1]
>>> at xsl:apply-templates
>>>
>>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#606)
>>> processing /X3D/Scene[1]
>>> at xsl:apply-templates
>>>
>>> (file:/home/coderextreme/X3DJSONLD/src/main/lib/stylesheets/X3dToPython.xslt#123)
>>> processing /X3D
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] BUILD FAILURE
>>> [INFO]
>>> ------------------------------------------------------------------------
>>> [INFO] Total time: 2.840 s
>>> [INFO] Finished at: 2022-01-10T17:59:22-06:00 [INFO]
>>> ------------------------------------------------------------------------
>>> [ERROR] Failed to execute goal
>>> org.codehaus.mojo:xml-maven-plugin:1.0.2:transform (default) on project
>>> X3DJSONLD: Failed to transform input file
>>> /home/coderextreme/X3DJSONLD/src/main/data/MaterialModulator.x3d: A
>>> sequence of more than one item is not allowed as the first argument of
>>> normalize-space() ("", "", ...) -> [Help 1] [ERROR] [ERROR] To see the
>>> full stack trace of the errors, re-run Maven with the -e switch.
>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>>> [ERROR]
>>> [ERROR] For more information about the errors and possible solutions,
>>> please read the following articles:
>>> [ERROR] [Help 1]
>>>
>>> https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FMAVEN%2FMojoExecutionException&data=04%7C01%7Cbrutzman%40nps.edu%7C79b6d0212127425f31b408d9d4ab3773%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637774654099751000%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VYVojdsW4QsyjrkZGnqfPOPmUp7x5ftTGE3H0TzrTts%3D&reserved=0
>>> <https://nam10.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcwiki.apache.org%2Fconfluence%2Fdisplay%2FMAVEN%2FMojoExecutionException&data=04%7C01%7Cbrutzman%40nps.edu%7C015d3b7123844f4c7d7508d9d761502e%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637777635530499448%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=EVAYH1SE%2FC6vXPYV9v0Zi5rmvMiki8JEsb2CwoyP6XQ%3D&reserved=0>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20220115/91eae6cb/attachment-0001.html>
More information about the x3d-public
mailing list