[x3d-public] HelloWorldProgramOutputCanonical.x3d Report/Discussion on X3DJSONLD serializers (not checked in). Problem with X3dToJson.xslt as run by X3DJSAIL.
Don Brutzman
brutzman at nps.edu
Thu May 13 14:40:25 PDT 2021
thanks for your note. response:
On 5/9/2021 3:17 PM, John Carlson wrote:
> Don, hope to cover this in our JSON/JS meeting on Monday.
>
> X3DXML attached.
>
> ==
>
> Should we add "addMetadata" to MetadataSet.java?
>
> Used addValue instead of setValue (needs to be fully tested, was getting void value returned from setValue), changes to Java and Node serializers.
* X3DJSAIL MetadataSet javadoc
https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html
The accessor (i.e. get and set) methods follow consistent patterns that match the field type.
MetadataSet (like every other node) can contain a single Metadata* node in the /metadata/ field to help describe the purpose of the parent.
* setMetadata() method is singular, it only takes an SFNode, since the /metadata/ field is an SFNode field
https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html#setMetadata(org.web3d.x3d.sai.Core.X3DMetadataObject)
MetadataSet setMetadata(X3DMetadataObject newValue)
Accessor method to assign org.web3d.x3d.sai.Core.X3DMetadataObject instance
(using a properly typed node) to inputOutput SFNode field metadata.
MetadataSet can also contain a /value/ field which is an MFNode array of Metadata* nodes that contain the metadata information of interest.
* addValue() has three methods which can append a single node or array of nodes to the field.
* setValue() has two methods which can set a single node or an array, replacing any prior contents in the /value/ field.
so I think that all of the cases you describe are covered satisfactorily now.
Certainly "metadata about metadata" and "metadata collections" are easily confused, so it is important to have consistent design patterns.
If you see a potential improvement to javadoc descriptions (which flow from X3DUOM and X3D Tooltips) please advise. Similarly am trying to embody these patterns in X3dToJava.xslt stylesheet conversions exactly, so that can help provide examples for any model when coding natively with X3DJSAIL.
Hope this explains the existing situation to your satisfaction.
> Discussion of whether addValue should be allowed after setValue or not. Here's current (not checked in) serializer Java translation:
>
> .addChild(new Shape()
> .setAppearance(new Appearance()
> .setMaterial(new Material().setUSE("GreenMaterial")))
> .setGeometry(new Text().setString(new org.web3d.x3d.jsail.fields.MFString(new MFString7().getArray()))
> .addComments(new CommentsBlock("Comment example A, plain quotation marks: He said, \"Immel did it!\""))
> .addComments(new CommentsBlock("Comment example B, XML character entities: He said, "Immel did it!""))
> .setMetadata(new MetadataSet().setName("EscapedQuotationMarksMetadataSet")
> .addValue(new MetadataString().setName("quotesTestC").setValue(new org.web3d.x3d.jsail.fields.MFString(new MFString8().getArray())))
> .addValue(new MetadataString().setName("extraChildTest").setValue(new org.web3d.x3d.jsail.fields.MFString(new MFString9().getArray()))))
> .setFontStyle(new FontStyle().setJustify(new org.web3d.x3d.jsail.fields.MFString(new MFString10().getArray())))))
>
> Here's the stylesheet translation:
>
> .addChild(new Shape()
> .setAppearance(new Appearance()
> .setMaterial(new Material().setUSE("GreenMaterial")))
> .setGeometry(new Text().setString(new String[] {"X3D Java","SAI Library","X3DJSAIL"})
> .addComments(" Comment example A, plain quotation marks: He said, \"Immel did it!\" ")
> .addComments(" Comment example B, XML character entities: He said, "Immel did it!" ")
> .setMetadata(new MetadataSet().setName("EscapedQuotationMarksMetadataSet")
> .setMetadata(new MetadataString().setName("quotesTestC").setValue(new String[] {"MFString example C, backslash-escaped quotes: He said, \"Immel did it!\""}))
> .setMetadata(new MetadataString().setName("extraChildTest").setValue(new String[] {"checks MetadataSetObject addValue() method"})))
> .setFontStyle(new FontStyle().setJustify(FontStyle.JUSTIFY_MIDDLE_MIDDLE))))
>
>
> This looks like it would overwrite the previous MetadataString, but it looks like it just sets the Parent? Would addMetadata be more descriptive?
>
>
> Here are methods on MetadataSet:
>
> public final MetadataSet setCssClass(String newValue)
> public final MetadataSet setCssStyle(String newValue)
> public final MetadataSet setDEF(String newValue)
> public final MetadataSet setName(String newValue)
> public final MetadataSet setUSE(String newValue)
> public MetadataSet addComments (CommentsBlock newCommentsBlock)
> public MetadataSet addComments (String newComment)
> public MetadataSet addComments (String[] newComments)
> public MetadataSet addValue(org.web3d.x3d.sai.Core.X3DNode newValue)
> public MetadataSet addValue(ProtoInstance newValue)
> public MetadataSet setCssClass(SFString newValue)
> public MetadataSet setCssStyle(SFString newValue)
> public MetadataSet setDEF(SFString newValue)
> */ public MetadataSet setIS(IS newValue)
> public MetadataSet setMetadata( org.web3d.x3d.sai.Core.X3DMetadataObject newValue)
> public MetadataSet setMetadata(ProtoInstance newValue)
> public MetadataSet setName(SFString newValue)
> public MetadataSet setReference(SFString newValue)
> public MetadataSet setReference(String newValue)
> public MetadataSet setUSE(MetadataSet DEFnode)
> public MetadataSet setUSE(SFString newValue)
> public MetadataSet setValue(ArrayList< org.web3d.x3d.sai.Core.X3DMetadataObject> newValue)
> public MetadataSet setValue(org.web3d.x3d.sai.Core.X3DNode[] newValue)
> public void addValue(org.web3d.x3d.sai.Core.X3DNode[] newValue)
> public void setValue(org.web3d.x3d.sai.Core.X3DNode newValue)
>
>
> what do people think? I'm think we should do addMetadata or setMetadata.
>
> ==
>
> No testing on python/x3dpsail/pyjnius or x3d.py done.
>
> ==
>
> Output from Java (.new.json) appears slightly different than stylesheet (diff below). This may be a concern. Here are the 2 versions of Java, one of which (coderextreme) outputs the .new.json.
>
>
> $ egrep -w 'Immel|it' net/*/data/Hello*cal.java|grep '\\\\\\'
> net/coderextreme/data/HelloWorldProgramOutputCanonical.java: .addComments("alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did it!\\\"\"\"})")
> net/x3djsonld/data/HelloWorldProgramOutputCanonical.java: .addComments(" alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did it!\\\"\"}) ")
>
> I will attempt to output JSON from the later Java to insure it does the right thing.
>
>
> The JSON output from Don's Java (x3djsonld above) code (from a recent version of X3dToJava.xslt) produced the following when run through jsonlint:
>
> Error: Parse error on line 60:
> ...iption","@content":"Example HelloWorldP
> ----------------------^
> Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 'undefined'
>
> Actual line is:
>
> 60 "@content":"Example HelloWorldProgram creates an X3D model using the X3D Java Scene
>
> My guess is there's a newline in there (after Scene), probably created by Saxon (default XSLT processor) in X3DJSAIL. Please migrate to a preferred stylesheet processor. The stylesheet how I run it in X3DJSONLD appears to be OK. Don's Java app looks ok, except for use of Saxon.
>
> Here's what I do in X3DJSONLD testing:
>
> java -cp ~/pythonSAI/X3DJSAIL.4.0.full.jar:../java net.coderextreme.RunSaxon --- ---overwrite --../lib/stylesheets/X3dToJson.xslt -json ../data/HelloWorldProgramOutputCanonical.x3d
>
> My RunSaxon.java is here:
>
> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcoderextreme%2FX3DJSONLD%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjava%2Fnet%2Fcoderextreme%2FRunSaxon.java&data=04%7C01%7Cbrutzman%40nps.edu%7C52c3f9901bc84f93e37408d913384dbc%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637561955256442500%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VWidf0rTpCqH8NUb%2BJ8eVptJyVaS4GL2kJMAoX2mbzA%3D&reserved=0
>
> Please review my code and try to duplicate in ant, or we can review jar versions. For people trying to generate X3D JSON from Java, this could be painful right now.
>
> For example, I put this in my Java apps:
>
> ConfigurationProperties.setXsltEngine(ConfigurationProperties.XSLT_ENGINE_NATIVE_JAVA);
>
> I can test this in Don's code. Works! Attached Don's Java code as modified by me. Suggested line above as addition to X3dToJava.xslt stylesheet.
>
>
> There are a lot of differences in JSON generated by Don's code and JSON generated by my code. I suggest a conference. What stylesheet processor works best?
>
> =============
>
> Extra quotes in comments:
>
>
> Running through my Java code:
>
> $ diff ../data/HelloWorldProgramOutputCanonical.*json
> 390c390
> < "#comment":"alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did it!\\\"\"\"})"
> ---
> > "#comment":"alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"\"Immel did it!\\\"\"\"\"})"
>
>
> My Java code, generated by converting JSON to XML then Java.
>
> .addComments("alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did it!\\\"\"\"})")
>
> Don's Java code:
>
> .addComments(" alternative Java source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did it!\\\"\"}) ")
>
> XML:
>
> <!-- alternative Java source: .setString(new String [] {"One, Two, Comment", "", "He said, \"Immel did it!\""}) -->
>
> JSON -> XML DOM dump prior to serialization:
>
> <!--alternative Java source: .setString(new String [] {"One, Two, Comment", "", "He said, \""Immel did it!\"""})-->
>
> Node/X3DJSAIL generated XML code:
>
> <!-- alternative Java source: .setString(new String [] {"One, Two, Comment", "", "He said, \""Immel did it!\"""}) -->
>
>
> It looks like whatever's generating JSON is producing extra quotes, and that's spreading to other code. X3dToJson.xslt???
>
> Don do you have any code downstream from X3dToJson.xslt? Everit? Can we get rid of the extra quotes in JSON? Thanks!
>
> Test harness (minus old versions of python) results below:
>
> $ bash several.sh ../data/HelloWorldProgramOutputCanonical.x3d
> BEGIN ../data/HelloWorldProgramOutputCanonical.x3d > json, Script DEF=colorTypeConversionScript contains CDATA source-code text, copied as "#sourceText" using "strings" mode
> Script DEF=MaterialModulatorScript contains CDATA source-code text, copied as "#sourceText" using "strings" mode
> END ../data/HelloWorldProgramOutputCanonical.x3d
> ================================================================================
> /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js ../data/HelloWorldProgramOutputCanonical.x3d ../data/HelloWorldProgramOutputCanonical.x3d.new
> @5 /X3D/Scene/0/NavigationInfo/0/$/avatarSize
> < "0.25 1.6 0.75"
> @5 /X3D/Scene/0/NavigationInfo/0/$/transitionType
> < "\"LINEAR\""
> @5 /X3D/Scene/0/LayerSet/0/$/order
> < "0"
> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Appearance/0/ImageTexture/0/$/containerField
> >"texture"
> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Box/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/$/containerField
> >"metadata"
> @4 /X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/MetadataString/1/$/containerField
> < "value"
> @5 /X3D/Scene/0/Transform/1/Shape/0/Text/0/FontStyle/0/$/family
> < "\"SERIF\""
> @6/X3D/Scene/0/Transform/1/Collision/0/Shape/0/Text/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/Coordinate/0/$/containerField
> >"coord"
> @6/X3D/Scene/0/Shape/1/Sphere/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Shape/2/Cone/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Shape/3/Cylinder/0/$/containerField
> >"geometry"
> @6/X3D/Scene/0/Shape/4/Extrusion/0/$/containerField
> >"geometry"
> @7/X3D/Scene/0/Shape/5/Appearance/0/ProgramShader/0/ShaderProgram/0/$/type
> >"VERTEX"
> @4 /X3D/Scene/0/Shape/5/Appearance/0/ProtoInstance/0/$/containerField
> < "shaders"
> @7/X3D/Scene/0/Shape/5/Appearance/0/ComposedShader/0/ShaderPart/0/$/type
> >"VERTEX"
> @1 /X3D/Scene/0/Group/0/Script/0/_ /X3D/Scene/0/Group/0/Script/0/_
> < "\n \n \n \necmascript:\n\nfunction colorInput (eventValue) // Example source code\n{\n colorsOutput = new MFColor(eventValue); // assigning value sends output event\n// Browser.print('colorInput=' + eventValue + ', colorsOutput=' + colorsOutput + '\\n');\n}\n\n "
> > "\n\necmascript:\r\n\r\nfunction colorInput (eventValue) // Example source code\r\n{\r\n colorsOutput = new MFColor(eventValue); // assigning value sends output event\r\n// Browser.print('colorInput=' + eventValue + ', colorsOutput=' + colorsOutput + '\\n');\r\n}"
> @6/X3D/Scene/0/Group/1/Shape/0/MetadataString/0/$/containerField
> >"metadata"
> @6/X3D/Scene/0/Group/1/Shape/0/Cone/0/$/containerField
> >"geometry"
> @1 /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_ /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_
> < "\n \n \n \n \n \n \necmascript:\nfunction initialize ()\n{\n newColor = diffuseColor; // start with correct color\n}\nfunction set_enabled (newValue)\n{\n\tenabled = newValue;\n}\nfunction clockTrigger (timeValue)\n{\n if (!enabled) return;\n red = newColor.r;\n green = newColor.g;\n blue = newColor.b;\n \n // note different modulation rates for each color component, % is modulus operator\n newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue + 0.04) % 1);\n\tif (enabled)\n\t{\n\t\tBrowser.print ('diffuseColor=(' + red + ',' + green + ',' + blue + ') newColor=' + newColor.toString() + '\\n');\n\t}\n}\n\n "
> > "\n\n\n\n\necmascript:\r\nfunction initialize ()\r\n{\r\n newColor = diffuseColor; // start with correct color\r\n}\r\nfunction set_enabled (newValue)\r\n{\r\n\tenabled = newValue;\r\n}\r\nfunction clockTrigger (timeValue)\r\n{\r\n if (!enabled) return;\r\n red = newColor.r;\r\n green = newColor.g;\r\n blue = newColor.b;\r\n\r\n // note different modulation rates for each color component, % is modulus operator\r\n newColor = new SFColor ((red + 0.02) % 1, (green + 0.03) % 1, (blue + 0.04) % 1);\r\n\tif (enabled)\r\n\t{\r\n\t\tBrowser.print ('diffuseColor=(' + red + ',' + green + ',' + blue + ') newColor=' + newColor.toString() + '\\n');\r\n\t}\r\n}"
> @6/X3D/Scene/0/Sound/0/AudioClip/0/$/containerField
> >"source"
>
> Different
> ~/X3DJSONLD/src/main/java/net/coderextreme/data ~/X3DJSONLD/src/main/shell
> ../java/net/coderextreme/data/HelloWorldProgramOutputCanonical.java
> ~/X3DJSONLD/src/main/shell
> ~/X3DJSONLD/src/main/java ~/X3DJSONLD/src/main/shell
> net/coderextreme/data/HelloWorldProgramOutputCanonical
> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3' USE='' is missing containerField relationship to parent node, assuming containerField='shaders' from initial node in corresponding ProtoDeclare. Need to fix ProtoInstance definition in model source.
> Note: toFileStylesheetConversion(X3dToJson.xslt) is overwriting prior file ../data/HelloWorldProgramOutputCanonical.new.json
> Script DEF=colorTypeConversionScript contains CDATA source-code text, copied as "#sourceText" using "strings" mode
> Script DEF=MaterialModulatorScript contains CDATA source-code text, copied as "#sourceText" using "strings" mode
> ~/X3DJSONLD/src/main/shell
> ================================================================================
> /home/coderextreme/X3DJSONLD/src/main/node/jsondiff.js ../data/HelloWorldProgramOutputCanonical.json ../data/HelloWorldProgramOutputCanonical.new.json
> @2 /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0 /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0
> < "\\\"\"Immel"
> > "\\\"\"\"Immel"
> @2 /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0 /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0
> < "it!\\\"\"\"})"
> > "it!\\\"\"\"\"})"
>
> Different
> ~/X3DJSONLD/src/main/node ~/X3DJSONLD/src/main/shell
> ../node/net/coderextreme/data/HelloWorldProgramOutputCanonical.js
> Warning: ../data/HelloWorldProgramOutputCanonical.new.x3d does not meet suggested X3D naming conventions, continuing...
> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3' USE='' is missing containerField relationship to parent node, assuming containerField='shaders' from initial node in corresponding ProtoDeclare. Need to fix ProtoInstance definition in model source.
> Note: toFileX3D() is overwriting prior file ../data/HelloWorldProgramOutputCanonical.new.x3d
> ~/X3DJSONLD/src/main/shell
> node /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js ../data/HelloWorldProgramOutputCanonical.x3d ../data/HelloWorldProgramOutputCanonical.new.x3d
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list