[x3d-public] [x3d] X3D WG May 14 minutes : SFString in XML encoding examples
Don Brutzman
brutzman at nps.edu
Sun May 23 12:43:28 PDT 2021
On 5/17/2021 5:14 PM, vmarchetti at kshell.com wrote:
> This issue is being looked at in more detail because of the proposal in Mantis issue 1320 https://www.web3d.org/member-only/mantis/view.php?id=1320 <https://www.web3d.org/member-only/mantis/view.php?id=1320> which would allow an optional variant in the encoding of of an MFString with one element.
yes
> As you point out, current browsers agree
(parenthetical: it is certainly a goal to reach agreement, but the ClassicVRML Grammar is the deciding reference)
> that the following are encodings of the same value when an SFString type
>
> field = 'field"value'
This is not a legal value since the grammar says, in second sentence of following paragraph,
* Any characters (including linefeed and '#') may appear within the quotes of SFString and MFString fields.
* A double quote character within a string shall be preceded with a backslash (e.g., "Each double quotes character \" shall have a backslash.").
* A backslash character within a string shall be preceded with a backslash forming two backslashes (e.g., "One backslash \\ character").
It is always helpful progress to discover an ambiguity or misunderstanding. Your preceding example is clearly illegal because it violates /shall be preceded with a backslash/.
For your preceding example here is a legal form, presumably meeting your intent,
field = 'field\"value'
and now continuing with that comparison,
> and when it is the only element of an MFString
>
> field = '"field\"value"'
The current X3D XML spec forbids an unquoted SFString value as single element within an MFString array. Mantis 1320 hopes to relax that excessive rule, without loss of strictly defined parsing.
The proposed change is to allow an unquoted (no inner quotation marks) SFString value as single element within an MFString array. This would match functionality of ClassicVRML encoding, which allows an SFString value to be provided for an MFString array..
> The question is;
Well uh there are (at least) two question in this example: legality of unescaped quotation mark " within an SFString (no, not legal); and then
==================================================================================
* Mantis 1320: relax requirement for quoted single-string value in MFString array
https://www.web3d.org/member-only/mantis/view.php?id=1320
Suggested XML Encoding specification prose addition:
"Unquoted MFString values are treated as a singleton SFString value."
==================================================================================
> if the option proposed in Mantis 1320 is adopted, would the allowed variant for a one-element MFString be
>
> field = 'field"value'
>
> or
>
> field = 'field\"value'
No, this is not correctly formed comparison for reasons above.
Again restating: according to 1320 the following would be legal and equivalent, first parsing XML and then taking SFString result:
XML {resulting SFString value}
someMFStringField = 'field\"value' {field"value} single-element array
someMFStringField = '"field\"value"' {field"value} single-element array
also allowed and equivalent in XML:
someMFStringField = "field\"value" {field"value} single-element array
someMFStringField = ""field\"value"" {field"value} single-element array
someMFStringField = ""field\"value"" {field"value} single-element array
also allowed and equivalent in XML, see character entity references https://dev.w3.org/html5/html-author/charref
someMFStringField = ""field\"value"" {field"value} single-element array
someMFStringField = ""field\"value"" {field"value} single-element array
someMFStringField = ""field\"value"" {field"value} single-element array
also allowed by XML parser are any combinations thereof of character entity references,
someMFStringField = ""field\"value"" {field"value} single-element array
For these examples, here are the XML parsing steps (a, b) and VRML parsing steps (c, d) going on:
a. [XML parse] outer delimiter for attribute must match, either ' or "
b. [XML parse] contained escape characters are converted, such as " or " or " to "
c. [VRML parse] embedded \" is treated as a solitary "
d. [VRML parse] SFString value allowed as single element of MFString array
Note result is the same in every case. Thinking of XML parsing (which most programs have no character-level control over, they use an XML API library) as precursor step to VRML parsing makes everything straightforward.
I am completely optimistic that we will reach consensus on this, for (at least) four reasons:
1. XML parsing is what it is, immensely stable and validated across a full range of languages,
2. VRML parsing is well supported and stable (plus we just found an omission the spec that includes clarity),
3. we can create any SFString or MFString value we want (keep those examples coming), and
4. validators can identify illegal strings, helping reduce model GIGO before bad strings reach X3D parsers.
Hope this helps...
>> On May 17, 2021, at 4:15 PM, Michalis Kamburelis <michalis.kambi at gmail.com <mailto:michalis.kambi at gmail.com>> wrote:
>>
>> Questions after (only briefly) reading this thread (I'm very sorry if
>> I misunderstood something):
>>
>> - Why are we wondering about SFString reading in X3D XML encoding?
>> - And why do we want to change and complicate it? :)
>>
>> In my view, the way how to handle SFString in X3D XML right now is
>> simple and that is good: The XML attribute content is the value of the
>> string. The end. There's no interpretation of double-quotes,
>> backslashes etc. or anything when dealing with SFString. This is also
>> what X3D browsers already do. Just take the XML attribute value from
>> your DOM, and there you go, you have your SFString value.
>>
>> To be clear, we have problems with MFString in X3D XML specification
>> (which is the point of my
>> https://github.com/michaliskambi/x3d-tests/wiki/Clarify-the-usage-of-quotes-and-backslashes-for-MFString-and-SFString-in-XML-encoding <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmichaliskambi%2Fx3d-tests%2Fwiki%2FClarify-the-usage-of-quotes-and-backslashes-for-MFString-and-SFString-in-XML-encoding&data=04%7C01%7Cbrutzman%40nps.edu%7C4cf964cecc024c6b11f308d91991fe99%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C637568937855145109%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0&sdata=%2BvC4LsRjOAKvtTecqz%2B62%2Bwf006jPhpe0NkqyNNlcTk%3D&reserved=0>
>> ), which I believe should be addressed by update to the specification.
>> Because my tests showed that all X3D browsers are consistent
>> in how they read SFString and MFString from X3D XML files. That is,
>> all browsers already agreed that SFString requires a bit different
>> code than MFString (for SFString, you just take attribute value; for
>> MFString, you deal with additional quotes and backslahes) and all
>> browsers behave the same. So we need
>> to fix the spec, which is a quite confusing, but the existing
>> implementations have actually already reached a consensus. We just
>> need to clarify it in the spec.
>>
>> So I would be a bit opposed to changing SFString logic in X3D XML.
>> Right now, backslash and double-quote inside SFString in X3D XML are
>> nothing special, and this is simple enough and already implemented
>> everywhere. What is the benefit of changing it?
>>
>> Regards,
>> Michalis
>>
>> pon., 17 maj 2021 o 21:17 vmarchetti at kshell.com
>> <vmarchetti at kshell.com> napisał(a):
>>
>>>
>>> This is a set of examples and proposed specification for XML encoding of SFString values in a pattern that is an extension of the VRML encoding of SFString values
>>>
>>> [To review: Curly brackets are being used to isolate text; and are not part of the text content. All other characters inside curly brackets are part of the text content,
>>> and no escaping or unescaping is done inside curly brackets ]
>>>
>>>
>>> XML Encoding to X3D values examples
>>>
>>> XML Encoding SFString value
>>> 1. {<MetadataSet name='items'/>} invalid encoding
>>> 2. {<MetadataSet name='"items"'/>} {items}
>>> 3. {<MetadataSet name='ite"ms'/>} invalid encoding
>>> 4. {<MetadataSet name='\items'/>} invalid encoding
>>> 5. {<MetadataSet name='"\"items"'/>} {"items}
>>>
>>>
>>> SFString values to XML encodings examples:
>>>
>>> SFString XML encoding
>>> 1. {items} {<MetadataSet name='"items"'/>}
>>> 2. {"items"} {<MetadataSet name='"\"items\""'/>}
>>> 3. {ite"ms} {<MetadataSet name='"ite\"ms"'/>}
>>> 4. {\items} {<MetadataSet name='"\\items"'/>}
>>> 5. {\"items} {<MetadataSet name='"\\\"items"'/>}
>>>
>>> Proposed specification prose:
>>>
>>> The determination of an SFString value from the XML encoding in an XML attribute is conducted in two steps:
>>>
>>> step 1. From the XML attribute text, determine the string referred to as normalized attribute value by the algorithm in Section 3.3.3 of the XML Recommentations document Edition 1. https://www.w3.org/TR/2008/REC-xml-20081126/#AVNormalize
>>>
>>> step 2. The resulting string from step 1 must be a valid VRML lexical element as defined in Annex A (normative) grammar for the Classic VRML Encoding : ISO/IEC 19776-2:2015 . From that document the following appears as the lexical element for an SFString value:
>>> ----------------------------------------------------------------------
>>> sfstringValue ::= string ;
>>> string ::= ".*" ... double-quotes must be \", backslashes must be \\...
>>> ----------------------------------------------------------------------
>>>
>>> If the normalized attribute value determined in step 1 does not match the sfstringValue lexical element then the source XML encoding is invalid. If the normalized attribute value does match the VRML sfstringValue lexical element, then the SFString value is determined by the following procedure
>>> step 2.1 strip off the leading and trailing quote characters.
>>> step 2.2 With the remaining text, working from left to right, replace the combination {\\} with single slash {\} and the combination {\"} with quote {"}
>>>
>>> The resulting text is the value of the SFString field.
>>>
>>>
>>> The reverse process, constructing a valid XML encoding for a determined SFString value, may be normatively specified as any XML text which gives the correct value on applying steps 1 and 2 above. Informative prose may give algorithms which achieve this end.
>>>
>>> Vince Marchetti
>>>
>>>
>>>> On May 16, 2021, at 7:40 PM, vmarchetti at kshell.com wrote:
>>>>
>>>> This refers to item 3 of the minutes of the May 14 X3D WG meeting minutes.
>>>>
>>>>
>>>> At the May 14 X3D WG meeting we again addressed the long-standing problem of defining the XML encoding of the SFString data type in node fields, particulalarly when the SFString containg quote characters. It was proposed to do two related tasks:
>>>> 1. Generate examples of XML encoding which clearly illustrate the correct encoding of these edge cases
>>>> 2. Write specification prose text which can be unambiguously interpreted by developers in their implementations, to ensure interoperability.
>>>>
>>>> A perennial issue in these discussions is that we're reasoning about text, in text. We usually use the quote characters to isolate the text we are writing about -- that doesn't work here, because the quotes are critical part of our discussion. For that reason, in the scope of this document, I use curly brackets to set off text examples. In the X3D specification the curly bracket characters cause no controversy, and I will not use any examples where curly brackets are part of the text values. So, if I were to say the the value of an X3D field of type SFString is {"Hello"}, I mean at least two things:
>>>> 1. If this value gets put in an MFString value and passed to the X3D Text node, it will be rendered as 7 character glyphs.
>>>> 2. In any language binding, the value is a 7 character string in the native string representation of that language.
>>>>
>>>> I propose that our examples should illustrate both issues the X3D implementors face: reading an XML encoding of an SFString value and determing the correct data for their implementation environment, and generating a correct XML encoding of an arbitrary SFString value. As Michalis K has pointed, there are relatively few usages of SFString has a field type, but one important one is as the name field in the Metadata nodes. So I will propose a set of decoding examples for determining the correct value of the name field in the XML encodings of an empty MetadataSet node. The result SFString for these examples should either be the value of the name field or the assertion that the XML encoding itself is not a syntactically valid encoding.
>>>>
>>>>
>>>> XML Encoding SFString value
>>>> 1. {<MetadataSet name='items'/>} ---
>>>> 2. {<MetadataSet name='"items"'/>} ---
>>>> 3. {<MetadataSet name='ite"ms'/>} ---
>>>> 4. {<MetadataSet name='\items'/>} ---
>>>> 5. {<MetadataSet name='"\"items"'/>} ---
>>>>
>>>>
>>>> The second set of examples should illustrate the encoding problem: To specify a correct XML encoding of the empty MetadataSet whose name field has the specified value. The XML encoding has multiple correct solutions because the XML format itself provides multiple representations of the same XML attribute.
>>>>
>>>>
>>>> SFString XML encoding
>>>> 1. {items} ---
>>>> 2. {"items"} ---
>>>> 3. {ite"ms} ---
>>>> 4. {\items} ---
>>>> 5. {\"items} ---
>>>>
>>>> A final part of this project is to prepare normative prose that determines these example patterns and can be applied to the general case.
>>>>
>>>>
>>>>
>>>> Vince Marchetti
>>>>
>>>>
>>>> _______________________________________________
>>>> x3d mailing list
>>>> x3d at web3d.org
>>>> http://web3d.org/mailman/listinfo/x3d_web3d.org
>>>
>>>
>>> _______________________________________________
>>> x3d mailing list
>>> x3d at web3d.org
>>> http://web3d.org/mailman/listinfo/x3d_web3d.org
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list