[x3d-public] Mantis 1320: relax requirement for quoted single-string value in MFString array
Don Brutzman
brutzman at nps.edu
Sun Jul 5 12:22:36 PDT 2020
Vince and Michalis: thanks for steady strain on the line! Responses follow in a block. We're having fun now... :0
a. Too complicated? OK. When in doubt, simplify. if any degree of embedded quotation marks are troublesome, then we simply add to XML encoding specification prose a statement like [1]:
[1] "Non-empty MFString values that do not contain any quotation marks are treated as a single string element."
Relevant pseudocode repeated, there are many ways to implement this in any language:
if (!value.isEmpty() and !value.contains('"')) // example approach #3
{
newMFStringValue = new MFString() of length 1;
newMFStringValue[0] = value; // processed text from XML parser
}
else // continue parsing as before
Am appreciating this ultra-simple approach even more than before since it can't mistakenly accept a regular MFString definition that erroneously omits a single quotation mark. Let's avoid any complicating details whatsoever.
Wondering, will we be able to live with something like that? Persevering forward...
---
b. Regarding pseudocode, not sure how it can be an "error" per se since it is just trying illustrate an algorithm.
If you mean the intended algorithm is insufficient, then OK I understand your reply.
---
c. Problem-solving alignment: we can sharpen focus of any questions like "How would [JavaScript/Java/Python/JSON/VRML/C++/ObjectPascal/etc.] handle this example?" solely on reading and parsing XML text into an MFString object.
Step by step parsing chain:
c.1. an MFString value in a text file is read and processed as text by XML parser, producing plain text (" becomes ", etc.).
c.2. That raw-string value is then handed off to X3D parser which applies X3D parsing rules to create MFString object.
Since XML is well defined and parsing libraries are well supported, high-quality implementation paths are expected to be possible for each programming language.
Side effect benefit: our large regression suite and diverse Quality Assurance (QA) tools means that any problematic or erroneous XML values are detectable and testable (and often fixable too).
---
d. Regarding your attempted addition of a second line to an unquoted value: that is clearly a 2-element MFString array and not relevant. Any such case is a regular MFString construct of quoted single-string elements, for example value='"Hello" "World"'
---
e. Emptiness:
e.1. MFString array of length 1 holding empty single-string remains unchanged:
value=' "" ' or value='""' or value='""' value="""" etc.
e.2. Empty MFString remains unchanged: since value='' remains the MFString empty array (i.e. empty list) and also the default MFString value. Equally legal is complete removal of the value='' empty attribute, which can be optionally omitted by any XML processor, which is therefore never seen by X3D parser.
Summary: no changes in existing MFString approach to (e.1) singleton empty string, or (e.2) empty list.
---
f. Whitespace.
We might optionally disallow whitespace for this approach (for example value=' ') by amending the [1] specification statement above as
[2] "Non-empty MFString values that do not contain any quotation marks, and containing characters other than whitespace, are treated as a single string element."
This might be more robust since it ignores fields with perhaps-inadvertent stray whitespace while honoring actual visible text. But we're talking about rules for plain-old whitespace, which seems like overreaching...
Equivalent existing (and clearly deliberate) example: value='" "'
If specification sentence [2] is considered complicated or troublesome, then let's stick with specification sentence [1].
---
g. I hope the recent HAnimHumanoid info -> MetadataSet example helps illustrate the significant merits of value terseness.
Getting this issue done well seems critical for future of metadata with X3D models.
---
h. Next steps: we are making progress but am thinking we need to work together more effectively on this issue so that we proceed towards resolution, rather than circling.
Mantis issue now raised, distilled issue summaries/resolutions can go there. Let's link prior emails back in there too.
[3] Mantis 1320: relax requirement for quoted single-string value in MFString array
https://www.web3d.org/member-only/mantis/view.php?id=1320
Let's link your pages in Mantis, Michalis. They are correct, as far as they go, but am still not seeing much detail on important considerations: XML preprocessing of characters before values are parsed as MFString.
Steadily improving our mutual understanding and XML-syntax examples will help.
---
i. Given the furious pace of current X3D4 activity, let's again defer any "deep dive" and have a meeting post-SIGGRAPH dedicated to reviewing this topic. Entry criteria:
- Mantis 1320 issues/alternatives/links updated in a distilled manner (e.g. your pages),
- Suggested specification prose (such as [1] and [2] above),
- Example XML content that we agree should either work or fail (such as values defined above).
---
j. Motivation revisited.
Whether any implementation is convinced about merits of defining a simple unquoted MFString value, or not, the X3D ecosystem will definitely encounter (likely many) such examples in actual practice. Being robust means handling them appropriately whenever reasonably possible, rather than failing (which is unpleasant to all concerned) or processing inconsistently (which is dangerous).
So we all share a strong incentive to find a solution here. We appear to have gotten a few steps closer on this current round of scrutiny, which is great!
Again thanks for close care and diligent examination of the issue alternatives.
v/r Don
On 7/5/2020 4:48 AM, Michalis Kamburelis wrote:
> Don,
>
> About your implementation and samples, there are 2 errors. Which shows my point -- this issue has complicated details :)
>
> 1.You need to account that the character may also be " (literal double quote), they don't need to be expressed using HTML entities when you surround the XML attribute using apostrophes. (One point of our proposed text, on https://github.com/michaliskambi/x3d-tests/wiki/Clarify-the-usage-of-quotes-and-backslashes-for-MFString-and-SFString-in-XML-encoding , was to clarify this).
>
> 2. Your example <Text string='we \"love\" you Ma!'/> does not work like you probably think. Since now this is treated like an SFString, the backslashes are *not* special, and so they would be visible. Please see my discussion of backslashes on the bottom of https://github.com/michaliskambi/x3d-tests/wiki/Clarify-the-usage-of-quotes-and-backslashes-for-MFString-and-SFString-in-XML-encoding . This was discussed on this mailing list, with a consensus, and my description there matches what browsers are already doing.
>
> And there remains a problem Vince noted: what is an empty string now? Previously it was MFString with 0 items. Now it will be MFString with 1 item equal to an empty string. This difference matters in some edge cases e.g. "Text" node will have a different size now, as it has 1 row.
>
> In general, the rationale """be liberal in what you accept""" is not convincing IMHO. That is not what a usual programming language is doing. You don't want to liberally accept everything that may make sense -- because this leads to a language with lots of special rules and allowances, which leads to a language that is hard to understand / predict. Instead, the usual guideline is that language should apply the rules consistently.
>
> My practical problem is that this new rule would create a confusion for authors. Your own example shows this, as you were confused with how <Text string='we \"love\" you Ma!'/> works.
>
> I imagine these scenarios:
>
> 1. I start with a <Text string="bla bla" /> . Cool, it just works.
>
> - But now I want to add a second line. So I need to learn to surround the first line in double quotes now anyway: <Text string=""bla bla" "another line"" />.
>
> 2. I start with a <Text string='we "love" you Ma!' />. Cool, it just works.
>
> - But now I want to add a second line. So I need to learn to surround the first line in double quotes now anyway: <Text string='"we "love" you Ma!" "another line"' />
>
> - And now I need to learn about backslashes, because the above example will fail. Both " and " inside delimit string parts, so I actually need <Text string='"we \"love\" you Ma!" "another line"' />
>
> These cases show how an author would get confused. Having to do the correct thing (with quotes and backslashes) at the beginning, means that you have a harder start (you need to understand these quotes and backslashes), but then you have a smooth way ahead, because it's obvious how to add a 2nd string.
>
> And existing tools (both writing and reading) have learned to cope with this throughout the years. I doubt that these additional quotes are a blocker for X3D adoption -- it's trivial to explain them, "they are there because you may have multiple strings, and we have a general way of writing many strings inside one XML attribute".
>
> All in all, I'm still not convinced.
>
> Regards,
> Michalis
>
> niedz., 5 lip 2020 o 05:01 vmarchetti at kshell.com <mailto:vmarchetti at kshell.com> <vmarchetti at kshell.com <mailto:vmarchetti at kshell.com>> napisał(a):
>
> Apologies, this fragment of a reply was sent prematurely and is incomplete, ignore it for now.
>
> Vince
>
>
> > On Jul 4, 2020, at 10:58 PM, vmarchetti at kshell.com <mailto:vmarchetti at kshell.com> wrote:
> >
> >
> >
> >> On Jul 4, 2020, at 9:26 PM, Don Brutzman <brutzman at nps.edu <mailto:brutzman at nps.edu>> wrote:
> >>
> >> Thanks for your note... yes we had a brief discussion some months ago, will look it up. Didn't want to dwell on the topic then (each of us too busy!!) but now more metadata is getting created and this relaxation will help.
> >>
> >> My understanding of years of implementation-complexity objections heard to this XML encoding relaxation have not appeared to be grounded by practice, which in this case is quite simple.
> >>
> >> For example, pseudocode adaptable to ~any language that is parsing text into X3D objects:
> >>
> >> if (!value.startsWith('"') and !value.endsWith('"')) // example approach #1
> >> {
> >> newMFStringValue = new MFString() of length 1;
> >> newMFStringValue[0] = value;
> >> }
> >> else // continue parsing as before
> >>
> >>
> >
> > However, I think this approach of coming up with code or pseudo code to explain or document the standard doesn't account that implementors will want to (and I think should) use standardized code to do all the XML processing, including applying the XML rules for un-encoding the way that attribute values can be encoded in XML.
> > For example, given an XML fragment
> > <Element fruit='"apple"'/>
> >
> > However, the same could be
> >
> > Someone using the standard Python module xml.etree to parse the document would only see a Python object with a (Python) attribute named fruit and a value which was the unicode value "apple" . If this occurred in an X3D document,
> > this would be interpreted as an SFString with value "apple" or an MFString of one element with value apple .
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list