[x3d-public] HelloWorldProgramOutputCanonical.x3d Report/Discussion on X3DJSONLD serializers (not checked in). Problem with X3dToJson.xslt as run by X3DJSAIL.

John Carlson yottzumm at gmail.com
Thu May 13 15:45:53 PDT 2021 

I will attempt to copy your Java source, hopefully in the near term. I'm 
in search of a libodejava for amd64 right now.

John

On 5/13/21 4:40 PM, Don Brutzman wrote:
> thanks for your note. response:
>
> On 5/9/2021 3:17 PM, John Carlson wrote:
>> Don, hope to cover this in our JSON/JS meeting on Monday.
>>
>> X3DXML attached.
>>
>> ==
>>
>> Should we add "addMetadata" to MetadataSet.java?
>>
>> Used addValue instead of setValue (needs to be fully tested, was 
>> getting void value returned from setValue), changes to Java and Node 
>> serializers.
>
> * X3DJSAIL MetadataSet javadoc
> https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html
>
> The accessor (i.e. get and set) methods follow consistent patterns 
> that match the field type.
>
> MetadataSet (like every other node) can contain a single Metadata* 
> node in the /metadata/ field to help describe the purpose of the parent.
>
> * setMetadata() method is singular, it only takes an SFNode, since the 
> /metadata/ field is an SFNode field
> https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html#setMetadata(org.web3d.x3d.sai.Core.X3DMetadataObject)
>
>     MetadataSet setMetadata​(X3DMetadataObject newValue)
>     Accessor method to assign org.web3d.x3d.sai.Core.X3DMetadataObject 
> instance
>     (using a properly typed node) to inputOutput SFNode field metadata.
>
> MetadataSet can also contain a /value/ field which is an MFNode array 
> of Metadata* nodes that contain the metadata information of interest.
>
> * addValue() has three methods which can append a single node or array 
> of nodes to the field.
>
> * setValue() has two methods which can set a single node or an array, 
> replacing any prior contents in the /value/ field.
>
> so I think that all of the cases you describe are covered 
> satisfactorily now.
>
> Certainly "metadata about metadata" and "metadata collections" are 
> easily confused, so it is important to have consistent design patterns.
>
> If you see a potential improvement to javadoc descriptions (which flow 
> from X3DUOM and X3D Tooltips) please advise.  Similarly am trying to 
> embody these patterns in X3dToJava.xslt stylesheet conversions 
> exactly, so that can help provide examples for any model when coding 
> natively with X3DJSAIL.
>
> Hope this explains the existing situation to your satisfaction.
>
>> Discussion of whether addValue should be allowed after setValue or 
>> not. Here's current (not checked in) serializer Java translation:
>>
>>            .addChild(new Shape()
>>              .setAppearance(new Appearance()
>>                .setMaterial(new Material().setUSE("GreenMaterial")))
>>              .setGeometry(new Text().setString(new 
>> org.web3d.x3d.jsail.fields.MFString(new MFString7().getArray()))
>>                .addComments(new CommentsBlock("Comment example A, 
>> plain quotation marks: He said, \"Immel did it!\""))
>>                .addComments(new CommentsBlock("Comment example B, XML 
>> character entities: He said, "Immel did it!""))
>>                .setMetadata(new 
>> MetadataSet().setName("EscapedQuotationMarksMetadataSet")
>>                  .addValue(new 
>> MetadataString().setName("quotesTestC").setValue(new 
>> org.web3d.x3d.jsail.fields.MFString(new MFString8().getArray())))
>>                  .addValue(new 
>> MetadataString().setName("extraChildTest").setValue(new 
>> org.web3d.x3d.jsail.fields.MFString(new MFString9().getArray()))))
>>                .setFontStyle(new FontStyle().setJustify(new 
>> org.web3d.x3d.jsail.fields.MFString(new MFString10().getArray())))))
>>
>> Here's the stylesheet translation:
>>
>> .addChild(new Shape()
>>          .setAppearance(new Appearance()
>>            .setMaterial(new Material().setUSE("GreenMaterial")))
>>          .setGeometry(new Text().setString(new String[] {"X3D 
>> Java","SAI Library","X3DJSAIL"})
>>            .addComments(" Comment example A, plain quotation marks: 
>> He said, \"Immel did it!\" ")
>>            .addComments(" Comment example B, XML character entities: 
>> He said, "Immel did it!" ")
>>            .setMetadata(new 
>> MetadataSet().setName("EscapedQuotationMarksMetadataSet")
>>              .setMetadata(new 
>> MetadataString().setName("quotesTestC").setValue(new String[] 
>> {"MFString example C, backslash-escaped quotes: He said, \"Immel did 
>> it!\""}))
>>              .setMetadata(new 
>> MetadataString().setName("extraChildTest").setValue(new String[] 
>> {"checks MetadataSetObject addValue() method"})))
>>            .setFontStyle(new 
>> FontStyle().setJustify(FontStyle.JUSTIFY_MIDDLE_MIDDLE))))
>>
>>
>> This looks like it would overwrite the previous MetadataString, but 
>> it looks like it just sets the Parent?  Would addMetadata be more 
>> descriptive?
>>
>>
>> Here are methods on MetadataSet:
>>
>>          public final MetadataSet setCssClass(String newValue)
>>          public final MetadataSet setCssStyle(String newValue)
>>          public final MetadataSet setDEF(String newValue)
>>          public final MetadataSet setName(String newValue)
>>          public final MetadataSet setUSE(String newValue)
>>          public MetadataSet addComments (CommentsBlock newCommentsBlock)
>>          public MetadataSet addComments (String newComment)
>>          public MetadataSet addComments (String[] newComments)
>>          public MetadataSet addValue(org.web3d.x3d.sai.Core.X3DNode 
>> newValue)
>>          public MetadataSet addValue(ProtoInstance newValue)
>>          public MetadataSet setCssClass(SFString newValue)
>>          public MetadataSet setCssStyle(SFString newValue)
>>          public MetadataSet setDEF(SFString newValue)
>>           */     public MetadataSet setIS(IS newValue)
>>          public MetadataSet setMetadata( 
>> org.web3d.x3d.sai.Core.X3DMetadataObject newValue)
>>          public MetadataSet setMetadata(ProtoInstance newValue)
>>          public MetadataSet setName(SFString newValue)
>>          public MetadataSet setReference(SFString newValue)
>>          public MetadataSet setReference(String newValue)
>>          public MetadataSet setUSE(MetadataSet DEFnode)
>>          public MetadataSet setUSE(SFString newValue)
>>          public MetadataSet setValue(ArrayList< 
>> org.web3d.x3d.sai.Core.X3DMetadataObject> newValue)
>>          public MetadataSet setValue(org.web3d.x3d.sai.Core.X3DNode[] 
>> newValue)
>>          public void addValue(org.web3d.x3d.sai.Core.X3DNode[] newValue)
>>          public void setValue(org.web3d.x3d.sai.Core.X3DNode newValue)
>>
>>
>> what do people think? I'm think we should do addMetadata or setMetadata.
>>
>> ==
>>
>> No testing on python/x3dpsail/pyjnius or x3d.py done.
>>
>> ==
>>
>> Output from Java (.new.json) appears slightly different than 
>> stylesheet (diff below).  This may be a concern.  Here are the 2 
>> versions of Java, one of which (coderextreme) outputs the .new.json.
>>
>>
>> $ egrep -w 'Immel|it' net/*/data/Hello*cal.java|grep '\\\\\\'
>> net/coderextreme/data/HelloWorldProgramOutputCanonical.java: 
>> .addComments("alternative Java source: .setString(new String [] 
>> {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did it!\\\"\"\"})")
>> net/x3djsonld/data/HelloWorldProgramOutputCanonical.java: 
>> .addComments(" alternative Java source: .setString(new String [] 
>> {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did it!\\\"\"}) ")
>>
>> I will attempt to output JSON from the later Java to insure it does 
>> the right thing.
>>
>>
>> The JSON output from Don's Java (x3djsonld above) code (from a recent 
>> version of X3dToJava.xslt) produced the following when run through 
>> jsonlint:
>>
>> Error: Parse error on line 60:
>> ...iption","@content":"Example HelloWorldP
>> ----------------------^
>> Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got 
>> 'undefined'
>>
>> Actual line is:
>>
>>     60  "@content":"Example HelloWorldProgram creates an X3D model 
>> using the X3D Java Scene
>>
>> My guess is there's  a newline in there (after Scene), probably 
>> created by Saxon (default XSLT processor) in X3DJSAIL.  Please 
>> migrate to a preferred stylesheet processor.  The stylesheet how I 
>> run it in X3DJSONLD appears to be OK.  Don's Java app looks ok, 
>> except for use of Saxon.
>>
>> Here's what I do in X3DJSONLD testing:
>>
>> java -cp ~/pythonSAI/X3DJSAIL.4.0.full.jar:../java 
>> net.coderextreme.RunSaxon --- ---overwrite 
>> --../lib/stylesheets/X3dToJson.xslt -json 
>> ../data/HelloWorldProgramOutputCanonical.x3d
>>
>> My RunSaxon.java is here:
>>
>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcoderextreme%2FX3DJSONLD%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjava%2Fnet%2Fcoderextreme%2FRunSaxon.java&data=04%7C01%7Cbrutzman%40nps.edu%7C52c3f9901bc84f93e37408d913384dbc%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637561955256442500%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VWidf0rTpCqH8NUb%2BJ8eVptJyVaS4GL2kJMAoX2mbzA%3D&reserved=0 
>>
>>
>> Please review my code and try to duplicate in ant, or we can review 
>> jar versions.  For people trying to generate X3D JSON from Java, this 
>> could be painful right now.
>>
>> For example, I put this in my Java apps:
>>
>> ConfigurationProperties.setXsltEngine(ConfigurationProperties.XSLT_ENGINE_NATIVE_JAVA); 
>>
>>
>> I can test this in Don's code.   Works!   Attached Don's Java code as 
>> modified by me.  Suggested line above as addition to X3dToJava.xslt 
>> stylesheet.
>>
>>
>> There are a lot of differences in JSON generated by Don's code and 
>> JSON generated by  my code.   I suggest a conference.  What 
>> stylesheet processor works best?
>>
>> =============
>>
>> Extra quotes in comments:
>>
>>
>> Running through my Java code:
>>
>> $ diff ../data/HelloWorldProgramOutputCanonical.*json
>> 390c390
>> <                               "#comment":"alternative Java source: 
>> .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, 
>> \\\"\"Immel did it!\\\"\"\"})"
>> ---
>>  >                               "#comment":"alternative Java source: 
>> .setString(new String [] {\"One, Two, Comment\", \"\", \"He said, 
>> \\\"\"\"Immel did it!\\\"\"\"\"})"
>>
>>
>> My Java code, generated by converting JSON to XML then Java.
>>
>>                .addComments("alternative Java source: .setString(new 
>> String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did 
>> it!\\\"\"\"})")
>>
>> Don's Java code:
>>
>>            .addComments(" alternative Java source: .setString(new 
>> String [] {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did 
>> it!\\\"\"}) ")
>>
>> XML:
>>
>> <!-- alternative Java source: .setString(new String [] {"One, Two, 
>> Comment", "", "He said, \"Immel did it!\""}) -->
>>
>> JSON -> XML DOM dump prior to serialization:
>>
>> <!--alternative Java source: .setString(new String [] {"One, Two, 
>> Comment", "", "He said, \""Immel did it!\"""})-->
>>
>> Node/X3DJSAIL generated XML code:
>>
>> <!-- alternative Java source: .setString(new String [] {"One, Two, 
>> Comment", "", "He said, \""Immel did it!\"""}) -->
>>
>>
>> It looks like whatever's generating JSON is producing extra quotes, 
>> and that's spreading to other code.   X3dToJson.xslt???
>>
>> Don do you have any code downstream from X3dToJson.xslt? Everit?  Can 
>> we get rid of the extra quotes in JSON?   Thanks!
>>
>> Test harness (minus old versions of python) results below:
>>
>> $ bash several.sh ../data/HelloWorldProgramOutputCanonical.x3d
>> BEGIN ../data/HelloWorldProgramOutputCanonical.x3d > json, Script 
>> DEF=colorTypeConversionScript contains CDATA source-code text, copied 
>> as "#sourceText" using "strings" mode
>> Script DEF=MaterialModulatorScript contains CDATA source-code text, 
>> copied as "#sourceText" using "strings" mode
>> END ../data/HelloWorldProgramOutputCanonical.x3d
>> ================================================================================ 
>>
>> /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js 
>> ../data/HelloWorldProgramOutputCanonical.x3d 
>> ../data/HelloWorldProgramOutputCanonical.x3d.new
>> @5 /X3D/Scene/0/NavigationInfo/0/$/avatarSize
>> < "0.25 1.6 0.75"
>> @5 /X3D/Scene/0/NavigationInfo/0/$/transitionType
>> < "\"LINEAR\""
>> @5 /X3D/Scene/0/LayerSet/0/$/order
>> < "0"
>> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Appearance/0/ImageTexture/0/$/containerField 
>>
>>  >"texture"
>> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Box/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/$/containerField
>>  >"metadata"
>> @4 
>> /X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/MetadataString/1/$/containerField
>> < "value"
>> @5 /X3D/Scene/0/Transform/1/Shape/0/Text/0/FontStyle/0/$/family
>> < "\"SERIF\""
>> @6/X3D/Scene/0/Transform/1/Collision/0/Shape/0/Text/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/Coordinate/0/$/containerField
>>  >"coord"
>> @6/X3D/Scene/0/Shape/1/Sphere/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Shape/2/Cone/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Shape/3/Cylinder/0/$/containerField
>>  >"geometry"
>> @6/X3D/Scene/0/Shape/4/Extrusion/0/$/containerField
>>  >"geometry"
>> @7/X3D/Scene/0/Shape/5/Appearance/0/ProgramShader/0/ShaderProgram/0/$/type 
>>
>>  >"VERTEX"
>> @4 /X3D/Scene/0/Shape/5/Appearance/0/ProtoInstance/0/$/containerField
>> < "shaders"
>> @7/X3D/Scene/0/Shape/5/Appearance/0/ComposedShader/0/ShaderPart/0/$/type
>>  >"VERTEX"
>> @1 /X3D/Scene/0/Group/0/Script/0/_ /X3D/Scene/0/Group/0/Script/0/_
>> < "\n        \n        \n        \necmascript:\n\nfunction colorInput 
>> (eventValue) // Example source code\n{\n colorsOutput = new 
>> MFColor(eventValue); // assigning value sends output event\n// 
>> Browser.print('colorInput=' + eventValue + ', colorsOutput=' + 
>> colorsOutput + '\\n');\n}\n\n      "
>>  > "\n\necmascript:\r\n\r\nfunction colorInput (eventValue) // 
>> Example source code\r\n{\r\n   colorsOutput = new 
>> MFColor(eventValue); // assigning value sends output event\r\n// 
>> Browser.print('colorInput=' + eventValue + ', colorsOutput=' + 
>> colorsOutput + '\\n');\r\n}"
>> @6/X3D/Scene/0/Group/1/Shape/0/MetadataString/0/$/containerField
>>  >"metadata"
>> @6/X3D/Scene/0/Group/1/Shape/0/Cone/0/$/containerField
>>  >"geometry"
>> @1 /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_ 
>> /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_
>> < "\n          \n          \n          \n          \n \n 
>> \necmascript:\nfunction initialize ()\n{\n    newColor = 
>> diffuseColor; // start with correct color\n}\nfunction set_enabled 
>> (newValue)\n{\n\tenabled = newValue;\n}\nfunction clockTrigger 
>> (timeValue)\n{\n    if (!enabled) return;\n    red = newColor.r;\n 
>> green = newColor.g;\n    blue  = newColor.b;\n    \n    // note 
>> different modulation rates for each color component, % is modulus 
>> operator\n    newColor = new SFColor ((red + 0.02) % 1, (green + 
>> 0.03) % 1, (blue + 0.04) % 1);\n\tif 
>> (enabled)\n\t{\n\t\tBrowser.print ('diffuseColor=(' + red + ',' + 
>> green + ',' + blue + ') newColor=' + newColor.toString() + 
>> '\\n');\n\t}\n}\n\n        "
>>  > "\n\n\n\n\necmascript:\r\nfunction initialize ()\r\n{\r\n newColor 
>> = diffuseColor; // start with correct color\r\n}\r\nfunction 
>> set_enabled (newValue)\r\n{\r\n\tenabled = newValue;\r\n}\r\nfunction 
>> clockTrigger (timeValue)\r\n{\r\n    if (!enabled) return;\r\n    
>> red   = newColor.r;\r\n    green = newColor.g;\r\n    blue  = 
>> newColor.b;\r\n\r\n    // note different modulation rates for each 
>> color component, % is modulus operator\r\n    newColor = new SFColor 
>> ((red + 0.02) % 1, (green + 0.03) % 1, (blue + 0.04) % 1);\r\n\tif 
>> (enabled)\r\n\t{\r\n\t\tBrowser.print ('diffuseColor=(' + red + ',' + 
>> green + ',' + blue + ') newColor=' + newColor.toString() + 
>> '\\n');\r\n\t}\r\n}"
>> @6/X3D/Scene/0/Sound/0/AudioClip/0/$/containerField
>>  >"source"
>>
>> Different
>> ~/X3DJSONLD/src/main/java/net/coderextreme/data 
>> ~/X3DJSONLD/src/main/shell
>> ../java/net/coderextreme/data/HelloWorldProgramOutputCanonical.java
>> ~/X3DJSONLD/src/main/shell
>> ~/X3DJSONLD/src/main/java ~/X3DJSONLD/src/main/shell
>> net/coderextreme/data/HelloWorldProgramOutputCanonical
>> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3' 
>> USE='' is missing containerField relationship to parent node, 
>> assuming containerField='shaders' from initial node in corresponding 
>> ProtoDeclare. Need to fix ProtoInstance definition in model source.
>> Note: toFileStylesheetConversion(X3dToJson.xslt) is overwriting prior 
>> file ../data/HelloWorldProgramOutputCanonical.new.json
>> Script DEF=colorTypeConversionScript contains CDATA source-code text, 
>> copied as "#sourceText" using "strings" mode
>> Script DEF=MaterialModulatorScript contains CDATA source-code text, 
>> copied as "#sourceText" using "strings" mode
>> ~/X3DJSONLD/src/main/shell
>> ================================================================================ 
>>
>> /home/coderextreme/X3DJSONLD/src/main/node/jsondiff.js 
>> ../data/HelloWorldProgramOutputCanonical.json 
>> ../data/HelloWorldProgramOutputCanonical.new.json
>> @2 
>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0 
>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0 
>>
>> < "\\\"\"Immel"
>>  > "\\\"\"\"Immel"
>> @2 
>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0 
>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0 
>>
>> < "it!\\\"\"\"})"
>>  > "it!\\\"\"\"\"})"
>>
>> Different
>> ~/X3DJSONLD/src/main/node ~/X3DJSONLD/src/main/shell
>> ../node/net/coderextreme/data/HelloWorldProgramOutputCanonical.js
>> Warning: ../data/HelloWorldProgramOutputCanonical.new.x3d does not 
>> meet suggested X3D naming conventions, continuing...
>> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3' 
>> USE='' is missing containerField relationship to parent node, 
>> assuming containerField='shaders' from initial node in corresponding 
>> ProtoDeclare. Need to fix ProtoInstance definition in model source.
>> Note: toFileX3D() is overwriting prior file 
>> ../data/HelloWorldProgramOutputCanonical.new.x3d
>> ~/X3DJSONLD/src/main/shell
>> node /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js 
>> ../data/HelloWorldProgramOutputCanonical.x3d 
>> ../data/HelloWorldProgramOutputCanonical.new.x3d
>>
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>
>
> all the best, Don

More information about the x3d-public mailing list