[x3d-public] HelloWorldProgramOutputCanonical.x3d Report/Discussion on X3DJSONLD serializers (not checked in). Problem with X3dToJson.xslt as run by X3DJSAIL.
John Carlson
yottzumm at gmail.com
Thu May 13 17:04:52 PDT 2021
Hold my horses. My JSON version indicates that the JSON could be wrong
as well. Here is JSON in question:
"-metadata":
{ "MetadataSet":
{
"@name":"EscapedQuotationMarksMetadataSet",
"-value":[
{ "MetadataString":
{
"@name":"quotesTestC",
"@value":["MFString example C,
backslash-escaped quotes: He said, \"Immel did it!\""]
}
},
{ "MetadataString":
{
"@name":"extraChildTest",
"@value":["checks
MetadataSetObject addValue() method"]
}
}
]
}
},
Pulling up XML:
<MetadataSet name='EscapedQuotationMarksMetadataSet'>
<MetadataString containerField='value' name='quotesTestC'
value='"MFString example C, backslash-escaped quotes: He said, \"Immel
did it!\""'/>
<MetadataString containerField='value'
name='extraChildTest' value='"checks MetadataSetObject addValue() method"'/>
</MetadataSet>
So the JSON translation is correct. Please patch X3dToJava.xslt for
this case.
I'm seeing that by setValue()/addValue() code was correct to start with,
so I'm backing out changes (i didn't check them in).
On 5/13/21 5:45 PM, John Carlson wrote:
> I will attempt to copy your Java source, hopefully in the near term.
> I'm in search of a libodejava for amd64 right now.
>
> John
>
> On 5/13/21 4:40 PM, Don Brutzman wrote:
>> thanks for your note. response:
>>
>> On 5/9/2021 3:17 PM, John Carlson wrote:
>>> Don, hope to cover this in our JSON/JS meeting on Monday.
>>>
>>> X3DXML attached.
>>>
>>> ==
>>>
>>> Should we add "addMetadata" to MetadataSet.java?
>>>
>>> Used addValue instead of setValue (needs to be fully tested, was
>>> getting void value returned from setValue), changes to Java and Node
>>> serializers.
>>
>> * X3DJSAIL MetadataSet javadoc
>> https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html
>>
>>
>> The accessor (i.e. get and set) methods follow consistent patterns
>> that match the field type.
>>
>> MetadataSet (like every other node) can contain a single Metadata*
>> node in the /metadata/ field to help describe the purpose of the parent.
>>
>> * setMetadata() method is singular, it only takes an SFNode, since
>> the /metadata/ field is an SFNode field
>> https://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/Core/MetadataSet.html#setMetadata(org.web3d.x3d.sai.Core.X3DMetadataObject)
>>
>>
>> MetadataSet setMetadata(X3DMetadataObject newValue)
>> Accessor method to assign
>> org.web3d.x3d.sai.Core.X3DMetadataObject instance
>> (using a properly typed node) to inputOutput SFNode field metadata.
>>
>> MetadataSet can also contain a /value/ field which is an MFNode array
>> of Metadata* nodes that contain the metadata information of interest.
>>
>> * addValue() has three methods which can append a single node or
>> array of nodes to the field.
>>
>> * setValue() has two methods which can set a single node or an array,
>> replacing any prior contents in the /value/ field.
>>
>> so I think that all of the cases you describe are covered
>> satisfactorily now.
>>
>> Certainly "metadata about metadata" and "metadata collections" are
>> easily confused, so it is important to have consistent design patterns.
>>
>> If you see a potential improvement to javadoc descriptions (which
>> flow from X3DUOM and X3D Tooltips) please advise. Similarly am trying
>> to embody these patterns in X3dToJava.xslt stylesheet conversions
>> exactly, so that can help provide examples for any model when coding
>> natively with X3DJSAIL.
>>
>> Hope this explains the existing situation to your satisfaction.
>>
>>> Discussion of whether addValue should be allowed after setValue or
>>> not. Here's current (not checked in) serializer Java translation:
>>>
>>> .addChild(new Shape()
>>> .setAppearance(new Appearance()
>>> .setMaterial(new Material().setUSE("GreenMaterial")))
>>> .setGeometry(new Text().setString(new
>>> org.web3d.x3d.jsail.fields.MFString(new MFString7().getArray()))
>>> .addComments(new CommentsBlock("Comment example A,
>>> plain quotation marks: He said, \"Immel did it!\""))
>>> .addComments(new CommentsBlock("Comment example B,
>>> XML character entities: He said, "Immel did it!""))
>>> .setMetadata(new
>>> MetadataSet().setName("EscapedQuotationMarksMetadataSet")
>>> .addValue(new
>>> MetadataString().setName("quotesTestC").setValue(new
>>> org.web3d.x3d.jsail.fields.MFString(new MFString8().getArray())))
>>> .addValue(new
>>> MetadataString().setName("extraChildTest").setValue(new
>>> org.web3d.x3d.jsail.fields.MFString(new MFString9().getArray()))))
>>> .setFontStyle(new FontStyle().setJustify(new
>>> org.web3d.x3d.jsail.fields.MFString(new MFString10().getArray())))))
>>>
>>> Here's the stylesheet translation:
>>>
>>> .addChild(new Shape()
>>> .setAppearance(new Appearance()
>>> .setMaterial(new Material().setUSE("GreenMaterial")))
>>> .setGeometry(new Text().setString(new String[] {"X3D
>>> Java","SAI Library","X3DJSAIL"})
>>> .addComments(" Comment example A, plain quotation marks:
>>> He said, \"Immel did it!\" ")
>>> .addComments(" Comment example B, XML character entities:
>>> He said, "Immel did it!" ")
>>> .setMetadata(new
>>> MetadataSet().setName("EscapedQuotationMarksMetadataSet")
>>> .setMetadata(new
>>> MetadataString().setName("quotesTestC").setValue(new String[]
>>> {"MFString example C, backslash-escaped quotes: He said, \"Immel did
>>> it!\""}))
>>> .setMetadata(new
>>> MetadataString().setName("extraChildTest").setValue(new String[]
>>> {"checks MetadataSetObject addValue() method"})))
>>> .setFontStyle(new
>>> FontStyle().setJustify(FontStyle.JUSTIFY_MIDDLE_MIDDLE))))
>>>
>>>
>>> This looks like it would overwrite the previous MetadataString, but
>>> it looks like it just sets the Parent? Would addMetadata be more
>>> descriptive?
>>>
>>>
>>> Here are methods on MetadataSet:
>>>
>>> public final MetadataSet setCssClass(String newValue)
>>> public final MetadataSet setCssStyle(String newValue)
>>> public final MetadataSet setDEF(String newValue)
>>> public final MetadataSet setName(String newValue)
>>> public final MetadataSet setUSE(String newValue)
>>> public MetadataSet addComments (CommentsBlock
>>> newCommentsBlock)
>>> public MetadataSet addComments (String newComment)
>>> public MetadataSet addComments (String[] newComments)
>>> public MetadataSet addValue(org.web3d.x3d.sai.Core.X3DNode
>>> newValue)
>>> public MetadataSet addValue(ProtoInstance newValue)
>>> public MetadataSet setCssClass(SFString newValue)
>>> public MetadataSet setCssStyle(SFString newValue)
>>> public MetadataSet setDEF(SFString newValue)
>>> */ public MetadataSet setIS(IS newValue)
>>> public MetadataSet setMetadata(
>>> org.web3d.x3d.sai.Core.X3DMetadataObject newValue)
>>> public MetadataSet setMetadata(ProtoInstance newValue)
>>> public MetadataSet setName(SFString newValue)
>>> public MetadataSet setReference(SFString newValue)
>>> public MetadataSet setReference(String newValue)
>>> public MetadataSet setUSE(MetadataSet DEFnode)
>>> public MetadataSet setUSE(SFString newValue)
>>> public MetadataSet setValue(ArrayList<
>>> org.web3d.x3d.sai.Core.X3DMetadataObject> newValue)
>>> public MetadataSet
>>> setValue(org.web3d.x3d.sai.Core.X3DNode[] newValue)
>>> public void addValue(org.web3d.x3d.sai.Core.X3DNode[]
>>> newValue)
>>> public void setValue(org.web3d.x3d.sai.Core.X3DNode newValue)
>>>
>>>
>>> what do people think? I'm think we should do addMetadata or
>>> setMetadata.
>>>
>>> ==
>>>
>>> No testing on python/x3dpsail/pyjnius or x3d.py done.
>>>
>>> ==
>>>
>>> Output from Java (.new.json) appears slightly different than
>>> stylesheet (diff below). This may be a concern. Here are the 2
>>> versions of Java, one of which (coderextreme) outputs the .new.json.
>>>
>>>
>>> $ egrep -w 'Immel|it' net/*/data/Hello*cal.java|grep '\\\\\\'
>>> net/coderextreme/data/HelloWorldProgramOutputCanonical.java:
>>> .addComments("alternative Java source: .setString(new String []
>>> {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did
>>> it!\\\"\"\"})")
>>> net/x3djsonld/data/HelloWorldProgramOutputCanonical.java:
>>> .addComments(" alternative Java source: .setString(new String []
>>> {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did it!\\\"\"}) ")
>>>
>>> I will attempt to output JSON from the later Java to insure it does
>>> the right thing.
>>>
>>>
>>> The JSON output from Don's Java (x3djsonld above) code (from a
>>> recent version of X3dToJava.xslt) produced the following when run
>>> through jsonlint:
>>>
>>> Error: Parse error on line 60:
>>> ...iption","@content":"Example HelloWorldP
>>> ----------------------^
>>> Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got
>>> 'undefined'
>>>
>>> Actual line is:
>>>
>>> 60 "@content":"Example HelloWorldProgram creates an X3D model
>>> using the X3D Java Scene
>>>
>>> My guess is there's a newline in there (after Scene), probably
>>> created by Saxon (default XSLT processor) in X3DJSAIL. Please
>>> migrate to a preferred stylesheet processor. The stylesheet how I
>>> run it in X3DJSONLD appears to be OK. Don's Java app looks ok,
>>> except for use of Saxon.
>>>
>>> Here's what I do in X3DJSONLD testing:
>>>
>>> java -cp ~/pythonSAI/X3DJSAIL.4.0.full.jar:../java
>>> net.coderextreme.RunSaxon --- ---overwrite
>>> --../lib/stylesheets/X3dToJson.xslt -json
>>> ../data/HelloWorldProgramOutputCanonical.x3d
>>>
>>> My RunSaxon.java is here:
>>>
>>> https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcoderextreme%2FX3DJSONLD%2Fblob%2Fmaster%2Fsrc%2Fmain%2Fjava%2Fnet%2Fcoderextreme%2FRunSaxon.java&data=04%7C01%7Cbrutzman%40nps.edu%7C52c3f9901bc84f93e37408d913384dbc%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637561955256442500%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=VWidf0rTpCqH8NUb%2BJ8eVptJyVaS4GL2kJMAoX2mbzA%3D&reserved=0
>>>
>>>
>>> Please review my code and try to duplicate in ant, or we can review
>>> jar versions. For people trying to generate X3D JSON from Java,
>>> this could be painful right now.
>>>
>>> For example, I put this in my Java apps:
>>>
>>> ConfigurationProperties.setXsltEngine(ConfigurationProperties.XSLT_ENGINE_NATIVE_JAVA);
>>>
>>>
>>> I can test this in Don's code. Works! Attached Don's Java code
>>> as modified by me. Suggested line above as addition to
>>> X3dToJava.xslt stylesheet.
>>>
>>>
>>> There are a lot of differences in JSON generated by Don's code and
>>> JSON generated by my code. I suggest a conference. What
>>> stylesheet processor works best?
>>>
>>> =============
>>>
>>> Extra quotes in comments:
>>>
>>>
>>> Running through my Java code:
>>>
>>> $ diff ../data/HelloWorldProgramOutputCanonical.*json
>>> 390c390
>>> < "#comment":"alternative Java source:
>>> .setString(new String [] {\"One, Two, Comment\", \"\", \"He said,
>>> \\\"\"Immel did it!\\\"\"\"})"
>>> ---
>>> > "#comment":"alternative Java
>>> source: .setString(new String [] {\"One, Two, Comment\", \"\", \"He
>>> said, \\\"\"\"Immel did it!\\\"\"\"\"})"
>>>
>>>
>>> My Java code, generated by converting JSON to XML then Java.
>>>
>>> .addComments("alternative Java source: .setString(new
>>> String [] {\"One, Two, Comment\", \"\", \"He said, \\\"\"Immel did
>>> it!\\\"\"\"})")
>>>
>>> Don's Java code:
>>>
>>> .addComments(" alternative Java source: .setString(new
>>> String [] {\"One, Two, Comment\", \"\", \"He said, \\\"Immel did
>>> it!\\\"\"}) ")
>>>
>>> XML:
>>>
>>> <!-- alternative Java source: .setString(new String [] {"One, Two,
>>> Comment", "", "He said, \"Immel did it!\""}) -->
>>>
>>> JSON -> XML DOM dump prior to serialization:
>>>
>>> <!--alternative Java source: .setString(new String [] {"One, Two,
>>> Comment", "", "He said, \""Immel did it!\"""})-->
>>>
>>> Node/X3DJSAIL generated XML code:
>>>
>>> <!-- alternative Java source: .setString(new String [] {"One, Two,
>>> Comment", "", "He said, \""Immel did it!\"""}) -->
>>>
>>>
>>> It looks like whatever's generating JSON is producing extra quotes,
>>> and that's spreading to other code. X3dToJson.xslt???
>>>
>>> Don do you have any code downstream from X3dToJson.xslt? Everit?
>>> Can we get rid of the extra quotes in JSON? Thanks!
>>>
>>> Test harness (minus old versions of python) results below:
>>>
>>> $ bash several.sh ../data/HelloWorldProgramOutputCanonical.x3d
>>> BEGIN ../data/HelloWorldProgramOutputCanonical.x3d > json, Script
>>> DEF=colorTypeConversionScript contains CDATA source-code text,
>>> copied as "#sourceText" using "strings" mode
>>> Script DEF=MaterialModulatorScript contains CDATA source-code text,
>>> copied as "#sourceText" using "strings" mode
>>> END ../data/HelloWorldProgramOutputCanonical.x3d
>>> ================================================================================
>>>
>>> /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js
>>> ../data/HelloWorldProgramOutputCanonical.x3d
>>> ../data/HelloWorldProgramOutputCanonical.x3d.new
>>> @5 /X3D/Scene/0/NavigationInfo/0/$/avatarSize
>>> < "0.25 1.6 0.75"
>>> @5 /X3D/Scene/0/NavigationInfo/0/$/transitionType
>>> < "\"LINEAR\""
>>> @5 /X3D/Scene/0/LayerSet/0/$/order
>>> < "0"
>>> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Appearance/0/ImageTexture/0/$/containerField
>>>
>>> >"texture"
>>> @6/X3D/Scene/0/Transform/0/Anchor/0/Shape/0/Box/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/$/containerField
>>>
>>> >"metadata"
>>> @4
>>> /X3D/Scene/0/Transform/1/Shape/0/Text/0/MetadataSet/0/MetadataString/1/$/containerField
>>> < "value"
>>> @5 /X3D/Scene/0/Transform/1/Shape/0/Text/0/FontStyle/0/$/family
>>> < "\"SERIF\""
>>> @6/X3D/Scene/0/Transform/1/Collision/0/Shape/0/Text/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Shape/0/IndexedLineSet/0/Coordinate/0/$/containerField
>>> >"coord"
>>> @6/X3D/Scene/0/Shape/1/Sphere/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Shape/2/Cone/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Shape/3/Cylinder/0/$/containerField
>>> >"geometry"
>>> @6/X3D/Scene/0/Shape/4/Extrusion/0/$/containerField
>>> >"geometry"
>>> @7/X3D/Scene/0/Shape/5/Appearance/0/ProgramShader/0/ShaderProgram/0/$/type
>>>
>>> >"VERTEX"
>>> @4 /X3D/Scene/0/Shape/5/Appearance/0/ProtoInstance/0/$/containerField
>>> < "shaders"
>>> @7/X3D/Scene/0/Shape/5/Appearance/0/ComposedShader/0/ShaderPart/0/$/type
>>>
>>> >"VERTEX"
>>> @1 /X3D/Scene/0/Group/0/Script/0/_ /X3D/Scene/0/Group/0/Script/0/_
>>> < "\n \n \n \necmascript:\n\nfunction
>>> colorInput (eventValue) // Example source code\n{\n colorsOutput =
>>> new MFColor(eventValue); // assigning value sends output event\n//
>>> Browser.print('colorInput=' + eventValue + ', colorsOutput=' +
>>> colorsOutput + '\\n');\n}\n\n "
>>> > "\n\necmascript:\r\n\r\nfunction colorInput (eventValue) //
>>> Example source code\r\n{\r\n colorsOutput = new
>>> MFColor(eventValue); // assigning value sends output event\r\n//
>>> Browser.print('colorInput=' + eventValue + ', colorsOutput=' +
>>> colorsOutput + '\\n');\r\n}"
>>> @6/X3D/Scene/0/Group/1/Shape/0/MetadataString/0/$/containerField
>>> >"metadata"
>>> @6/X3D/Scene/0/Group/1/Shape/0/Cone/0/$/containerField
>>> >"geometry"
>>> @1 /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_
>>> /X3D/Scene/0/ProtoDeclare/1/ProtoBody/0/Script/0/_
>>> < "\n \n \n \n \n \n
>>> \necmascript:\nfunction initialize ()\n{\n newColor =
>>> diffuseColor; // start with correct color\n}\nfunction set_enabled
>>> (newValue)\n{\n\tenabled = newValue;\n}\nfunction clockTrigger
>>> (timeValue)\n{\n if (!enabled) return;\n red = newColor.r;\n
>>> green = newColor.g;\n blue = newColor.b;\n \n // note
>>> different modulation rates for each color component, % is modulus
>>> operator\n newColor = new SFColor ((red + 0.02) % 1, (green +
>>> 0.03) % 1, (blue + 0.04) % 1);\n\tif
>>> (enabled)\n\t{\n\t\tBrowser.print ('diffuseColor=(' + red + ',' +
>>> green + ',' + blue + ') newColor=' + newColor.toString() +
>>> '\\n');\n\t}\n}\n\n "
>>> > "\n\n\n\n\necmascript:\r\nfunction initialize ()\r\n{\r\n
>>> newColor = diffuseColor; // start with correct
>>> color\r\n}\r\nfunction set_enabled (newValue)\r\n{\r\n\tenabled =
>>> newValue;\r\n}\r\nfunction clockTrigger (timeValue)\r\n{\r\n if
>>> (!enabled) return;\r\n red = newColor.r;\r\n green =
>>> newColor.g;\r\n blue = newColor.b;\r\n\r\n // note different
>>> modulation rates for each color component, % is modulus
>>> operator\r\n newColor = new SFColor ((red + 0.02) % 1, (green +
>>> 0.03) % 1, (blue + 0.04) % 1);\r\n\tif
>>> (enabled)\r\n\t{\r\n\t\tBrowser.print ('diffuseColor=(' + red + ','
>>> + green + ',' + blue + ') newColor=' + newColor.toString() +
>>> '\\n');\r\n\t}\r\n}"
>>> @6/X3D/Scene/0/Sound/0/AudioClip/0/$/containerField
>>> >"source"
>>>
>>> Different
>>> ~/X3DJSONLD/src/main/java/net/coderextreme/data
>>> ~/X3DJSONLD/src/main/shell
>>> ../java/net/coderextreme/data/HelloWorldProgramOutputCanonical.java
>>> ~/X3DJSONLD/src/main/shell
>>> ~/X3DJSONLD/src/main/java ~/X3DJSONLD/src/main/shell
>>> net/coderextreme/data/HelloWorldProgramOutputCanonical
>>> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3'
>>> USE='' is missing containerField relationship to parent node,
>>> assuming containerField='shaders' from initial node in corresponding
>>> ProtoDeclare. Need to fix ProtoInstance definition in model source.
>>> Note: toFileStylesheetConversion(X3dToJson.xslt) is overwriting
>>> prior file ../data/HelloWorldProgramOutputCanonical.new.json
>>> Script DEF=colorTypeConversionScript contains CDATA source-code
>>> text, copied as "#sourceText" using "strings" mode
>>> Script DEF=MaterialModulatorScript contains CDATA source-code text,
>>> copied as "#sourceText" using "strings" mode
>>> ~/X3DJSONLD/src/main/shell
>>> ================================================================================
>>>
>>> /home/coderextreme/X3DJSONLD/src/main/node/jsondiff.js
>>> ../data/HelloWorldProgramOutputCanonical.json
>>> ../data/HelloWorldProgramOutputCanonical.new.json
>>> @2
>>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0
>>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/12/0
>>>
>>> < "\\\"\"Immel"
>>> > "\\\"\"\"Immel"
>>> @2
>>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0
>>> /X3D/Scene/-children/13/Transform/-children/1/Collision/-proxy/Shape/-children/2/#comment/14/0
>>>
>>> < "it!\\\"\"\"})"
>>> > "it!\\\"\"\"\"})"
>>>
>>> Different
>>> ~/X3DJSONLD/src/main/node ~/X3DJSONLD/src/main/shell
>>> ../node/net/coderextreme/data/HelloWorldProgramOutputCanonical.js
>>> Warning: ../data/HelloWorldProgramOutputCanonical.new.x3d does not
>>> meet suggested X3D naming conventions, continuing...
>>> WARNING_MESSAGE: ProtoInstance name='ShaderProto' DEF='TestShader3'
>>> USE='' is missing containerField relationship to parent node,
>>> assuming containerField='shaders' from initial node in corresponding
>>> ProtoDeclare. Need to fix ProtoInstance definition in model source.
>>> Note: toFileX3D() is overwriting prior file
>>> ../data/HelloWorldProgramOutputCanonical.new.x3d
>>> ~/X3DJSONLD/src/main/shell
>>> node /home/coderextreme/X3DJSONLD/src/main/node/xmldiff.js
>>> ../data/HelloWorldProgramOutputCanonical.x3d
>>> ../data/HelloWorldProgramOutputCanonical.new.x3d
>>>
>>>
>>> _______________________________________________
>>> x3d-public mailing list
>>> x3d-public at web3d.org
>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>>
>>
>> all the best, Don
More information about the x3d-public
mailing list