[x3d-public] [Junk] Re: MFString quoting [was: interesting sample]

[Don Brutzman]

[Summary: relationship between XML and also X3D XML Encoding for MFString representations]

On 4/25/2017 9:25 PM, Michalis Kamburelis wrote:
> The proposed change to the specification (all to the section
> http://www.web3d.org/documents/specifications/19776-1/V3.3/Part01/EncodingOfFields.html#SFString
> ) would be to change this part:
> 
> """""""
>    The MFString specifies zero or more SFStrings enclosed in single
> quotes (e.g., '"string1" "string2"').

Looks appealing, but can't quite be specified like that.  This is because XML Recommendation allows either attribute="value" or attribute='value' forms.  Further an intermediate User Agent (i.e. processor) gets to choose whatever form it prefers.

We can't legislate XML behavior.  Rather the X3D XML Specification defines how to represent X3D constructs using XML syntax.  The corresponding XML DTD, Schema and Schematron for X3D offer varying degrees of validation for such constructs.

The workaround is to use equivalent character-entity abbreviations for " or ' as appropriate (" or ' or equivalent numerical forms).  And again, a User Agent can swap around equivalences as it sees fit, attribute="value" or attribute='value' forms remain interchangeable and equivalent.  Once that file leaves your editor, most XML parsers are accommodating but frankly the author no longer have a vote in the matter.

Let's look at alternative representations for " character.  Helpful reference gives 4 alternatives (am not sure about the " version) that is interesting:

	https://dev.w3.org/html5/html-author/charref

	" "
	"
	"

Information theory: the W3C Post Schema Validation Infoset (PSVI) rules say that no information can be inferred from which choice is encountered.

And so we are right back to (a) XML single-attribute xs:string preparsing according to W3C Recommendation rules, followed by (b) X3D parsing of MFString [ "quoted" "attribute" "values." ]

We might well make the X3D XML Encoding Specification more explanatory or clear, perhaps using EXAMPLES such as yours above.  Perhaps even a data flow diagram?  First attempt:
a. XML parser reads attribute value and strips surrounding pair of single (or double) quote marks.
b. XML parser handles any equivalence swapping of individual characters that it desires (and which can vary).
c. X3D parser handles any equivalence swapping to UTF-8 characters, in particular " and \ characters.
d. X3D parser splits MFString array according to " delimiters while retaining escaped \" and \\ character pairs as content.
e. MFString array is result.

As before, if anyone has an MFString value that doesn't appear representable in X3D XML Encoding, please post it.  Typically there are actually multiple ways to express an MFString value in X3D XML Encoding, e.g.

	 '"string1" "string2"'
	 '"string1" "string2"'
	 ""string1" "string2""
	 ""string1" "string2""
	 ""string1" "string2""
	 '"string1" "string2"'

etc. etc. for various combinatorics of character equivalences.

If this seems hard, well yeah it is complicated.  But without such XML rules and Unicode representations there would be a heckuva lot more Babel out there, and probably not an international Web.  So that it is also an incredibly strong motivation for us to get it exactly compatibly correct.

Hope this helps makes MFString handling more understandable.  Looking forward to discussing those mantis issues in a future meeting, reconciling wherever possible, improving specification wording and examples for clarity.  X3D XML Schematron and X3D Tidy are likely a bit improvable too.  X3D-Edit usually makes it easy for an author to sort out but can likely improve as well.

Hugely valuable outcomes of such review will hopefully include more consistent support by tools and better understanding by all.

btw X3DOM gets a lot of this for free by using WebFonts (though an author or conversion tool has to add that support to the HTML).

	http://www.web3d.org/x3d/content/examples/HelloWorldX3dom.xhtml
	view-source:http://www.web3d.org/x3d/content/examples/HelloWorldX3dom.xhtml
	[lines 17..65 and 201]

also
	https://examples.x3dom.org/example/text-quality.html
	view-source:https://examples.x3dom.org/example/text-quality.html

Related note is that coherently applying this approach is fundamentally important for Canonical X3D Encoding, binary compression, XML Encryption (privacy) and XML Digital Signature (authentication).

p.s. Next challenge: continuing with better support for extended character sets (such as emoticons) and Internationalization (I18N).  For example, am hoping to test support for many languages here:

	X3D Example Archives: X3D for Advanced Modeling, Hello World Scenes
	http://x3dgraphics.com/examples/X3dForAdvancedModeling/HelloWorldScenes

p.p.s. Open challenge: a set of regular expressions (regexes) that validate MFString representations for X3D ClassicVRML Encoding and also X3D XML Encoding.  Any takers?  8)

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman