[x3d-public] X3dToJson.xslt still removing spaces from comments... fixed, please test

John Carlson yottzumm at gmail.com
Wed Jan 24 23:31:56 PST 2018


Okay, now you need to XML output to pay attention to normalizeCommentWhitespace.

Thanks,

John

Sent from Mail for Windows 10

From: Don Brutzman
Sent: Wednesday, January 24, 2018 2:11 AM
To: John Carlson
Cc: X3D Graphics public mailing list
Subject: Re: X3dToJson.xslt still removing spaces from comments... fixed,please test

OK.  Have added property normalizeCommentWhitespace and corresponding methods.  Modified X3dToJson.xslt to support this parameter, default is to normalize comment whitespace.  Also added support to CommentsBlock.toStringX3D() and toStringVRML() methods.

I don't think it is quite practical to modify the pretty-print HTML, since comments come out as plain text and not PRE-formatted text.  Shifting that around would be... uh, less pretty.  8)

Note that this properties work once per file/stream creation, and cannot be turned on/off in the middle of a file/stream output.  The original comment is retained in memory, comment values in the serialization output reflects the property setting.  So programmers are not losing any significant whitespace text within CommentsBlock objects.

Smoke tests passed, update deployed.

http://www.web3d.org/specifications/java/X3dJavaSceneAuthoringInterface.html#UtilityMethods

http://www.web3d.org/specifications/java/javadoc/org/web3d/x3d/jsail/ConfigurationProperties.html#isNormalizeCommentWhitespace--

As ever, further testing welcome to see if all code paths are working OK.  Again thanks for sticking with this one John, it is a useful refinement.


On 1/23/2018 11:25 AM, John Carlson wrote:
> I'm pretty sure it's XML -> JSON and XML -> HTML, as previously mentioned.
> 
> On Jan 23, 2018 10:49 AM, "Don Brutzman" <brutzman at nps.edu <mailto:brutzman at nps.edu>> wrote:
> 
>     Good point.  Probably there are other use cases too.
> 
>     So maybe we would do better to NOT require normalization of whitespace in comments, but keep it as an option.  That is do-able in the code without much fuss.
> 
>     In which case we are back to your original diff observation: what part of the existing code is normalizing whitespace in your comments?  DOM loader, perhaps?  If so, is there a similar switch we might apply to that library?
> 
> 
>     On 1/22/2018 12:22 PM, John Carlson wrote:
> 
>         A case where I wouldn’t want to normalize in comments is if someone is created a multicolumn table, or a hierarchy. Perhaps normalize spaces, but not whitespace.
> 
>         John
> 
>         Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986 <https://go.microsoft.com/fwlink/?LinkId=550986>> for Windows 10
> 
>         *From: *Don Brutzman <mailto:brutzman at nps.edu <mailto:brutzman at nps.edu>>
>         *Sent: *Monday, January 22, 2018 11:30 AM
> 
>         *To: *John Carlson <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>>
>         *Cc: *x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
>         *Subject: *Re: X3dToJson.xslt still removing spaces from comments. Seethisoutput from Java compared to JavaScript and JavaScript output
> 
> 
>         so in that case: i recommend we clarify in X3D Canonical Form that whitespace in comments is normalized.  in which case we are done with that code.
> 
>         our alternative to permit irregular whitespace in comments may not be consistently implementable, and certainly is confusing/deceiving when applying security measures.
> 
>         i will enter a specification comment and the working group can later discuss at the appropriate time.
> 
>         On 1/22/2018 3:40 AM, John Carlson wrote:
> 
>           > Then by all means, make the X3D and JSON output have the same normalization
> 
>           >
> 
>           > Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986 <https://go.microsoft.com/fwlink/?LinkId=550986>> for Windows 10
> 
>           >
> 
>           > *From: *Don Brutzman <mailto:brutzman at nps.edu <mailto:brutzman at nps.edu>>
> 
>           > *Sent: *Monday, January 22, 2018 12:01 AM
> 
>           > *To: *John Carlson <mailto:yottzumm at gmail.com <mailto:yottzumm at gmail.com>>
> 
>           > *Cc: *x3d-public at web3d.org <mailto:x3d-public at web3d.org> <mailto:x3d-public at web3d.org <mailto:x3d-public at web3d.org>>
> 
>           > *Subject: *Re: X3dToJson.xslt still removing spaces from comments. See thisoutput from Java compared to JavaScript and JavaScript output
> 
>           >
> 
>           > X3D Canonical Form says that whitespace is normalized, and X3DJSAIL output attempts to follow that.  The first of the following rules can be interpreted that way, though I'm not seeing explicit mention of comments.
> 
>           >
> 
>           > X3D Compressed Binary Encoding: 4.2.3 X3D canonical form
> 
>           >
> 
>           > http://www.web3d.org/documents/specifications/19776-3/V3.1/Part03/concepts.html#X3DCanonicalForm <http://www.web3d.org/documents/specifications/19776-3/V3.1/Part03/concepts.html#X3DCanonicalForm>
> 
>           >
> 
>           > ====================
> 
>           >
> 
>           > a. Whitespace rules:
> 
>           >
> 
>           >       All whitespace characters are converted to a normalized (single-occurrence) blank character.
> 
>           >
> 
>           >       Whitespace includes comma separators between all MF-type array values, including MFString arrays.
> 
>           >
> 
>           >       All literal characters within an SFString value are retained verbatim.
> 
>           >
> 
>           >       All literal characters within MFString array values are retained verbatim.
> 
>           >
> 
>           > ====================
> 
>           >
> 
>           > I thought another prior requirement might be in XML Recommendation or XML Infoset, but didn't find it...
> 
>           >
> 
>           > XML Information Set (Second Edition)
> 
>           >
> 
>           > http://www.w3.org/TR/xml-infoset <http://www.w3.org/TR/xml-infoset>
> 
>           >
> 
>           > Extensible Markup Language (XML) 1.0 (Fifth Edition)
> 
>           >
> 
>           > https://www.w3.org/TR/xml
> 
>           >
> 
>           > Not sure where the excess whitespace is getting stripped, didn't find anything like that in the CommentsBlock source.
> 
>           >
> 
>           > Wouldn't surprise me if DOM loader was doing that, in which case it can't be "fixed" or retained any differently - confirmation or further sleuthing welcome..
> 
>           >
> 
>           > In practice I have seen tools normalize whitespace.  Seems like a good practice, you don't want invisible characters (or tool idiosyncrasies independently deciding that strictly equivalent information in a model might vary in terms of whitespace.  That's bad for digital signature, encryption, security back doors, etc.
> 
>           >
> 
>           > So, if anything, I'd lean towards explicitly requiring the normalization of whitespace in comments for X3D Canonical Form.
> 
>           >
> 
>           > On 1/20/2018 10:29 PM, John Carlson wrote:
> 
>           >
> 
>           >  > See subject.  Excess spaces are being removed from comments in JSON output from X3DJSAIL.
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > $ diff HelloWorldProgramOutput.Nashorn.x3d HelloWorldProgramOutput.Nashorn.js.new.x3d
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 73c73
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <                     <!-- Comment example A, plain quotation marks:  He said, "Immel did it!" -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > ---
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >                     <!-- Comment example A, plain quotation marks: He said, "Immel did it!" -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 112a113
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 230a232
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 248,252c250
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success: x3dModel.findNodeByDEF(DeclarativeAppearanceExample) = <Appearance DEF='DeclarativeAppearanceExample'/> i.e.
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > < <Appearance DEF='DeclarativeAppearanceExample'>
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <     <!- - DeclarativeMaterialExample gets overridden by subsequently added MaterialModulator ProtoInstance - ->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <     <ProtoInstance DEF='MyMaterialModulator' name='MaterialModulator' containerField='material'/>
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > < </Appearance> -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > ---
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: x3dModel.findNodeByDEF(DeclarativeAppearanceExample) = <Appearance DEF='DeclarativeAppearanceExample'/> i.e. <Appearance DEF='DeclarativeAppearanceExample'> <!- - DeclarativeMaterialExample gets overridden by subsequently added MaterialModulator ProtoInstance - -> <ProtoInstance DEF='MyMaterialModulator' name='MaterialModulator' containerField='material'/> </Appearance> -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 255c253
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success: x3dModel.findElementByNameValue("MaterialModulator", "ProtoDeclare")  found -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > ---
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: x3dModel.findElementByNameValue("MaterialModulator", "ProtoDeclare") found -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > 279,284c277,282
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success:  AnchorObject.isNode()=true,            siteAnchor.isNode()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success:  AnchorObject.isStatement()=false,        siteAnchor.isStatement()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success:   ROUTEObject.isNode()=false,   orbitPositionROUTE.isNode()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success:   ROUTEObject.isStatement()=true, orbitPositionROUTE.isStatement()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test success: CommentsBlock.isNode()=false,           testComments.isNode()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > <         <!-- Test failure: CommentsBlock.isStatement()=true,      testComments.isStatement()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  > ---
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: AnchorObject.isNode()=true, siteAnchor.isNode()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: AnchorObject.isStatement()=false, siteAnchor.isStatement()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: ROUTEObject.isNode()=false, orbitPositionROUTE.isNode()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: ROUTEObject.isStatement()=true, orbitPositionROUTE.isStatement()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test success: CommentsBlock.isNode()=false, testComments.isNode()=false -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           >  >  >         <!-- Test failure: CommentsBlock.isStatement()=true, testComments.isStatement()=true -->
> 
>           >
> 
>           >  >
> 
>           >
> 
>           > all the best, Don
> 
>           >
> 
>           > --
> 
>           >
> 
>           > Don Brutzman  Naval Postgraduate School, Code USW/Br brutzman at nps.edu <mailto:brutzman at nps.edu>
> 
>           >
> 
>           > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 <tel:%2B1.831.656.2149>
> 
>           >
> 
>           > X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman <http://faculty.nps.edu/brutzman>
> 
>           >
> 
>         all the best, Don
> 
>         -- 
> 
>         Don Brutzman  Naval Postgraduate School, Code USW/Br brutzman at nps.edu <mailto:brutzman at nps.edu>
> 
>         Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 <tel:%2B1.831.656.2149>
> 
>         X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman <http://faculty.nps.edu/brutzman>
> 
> 
> 
>     all the best, Don
>     -- 
>     Don Brutzman  Naval Postgraduate School, Code USW/Br brutzman at nps.edu <mailto:brutzman at nps.edu>
>     Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149 <tel:%2B1.831.656.2149>
>     X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman <http://faculty.nps.edu/brutzman>
> 
> 


all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20180125/dac2f103/attachment-0001.html>


More information about the x3d-public mailing list