[x3d-public] X3D regexes for colorRGBA, image need work

Andreas Plesch andreasplesch at gmail.com
Sun Jul 22 16:32:27 PDT 2018


I agree, once a single field pattern is established it is possible,
often straightforward to expand to a multi field pattern.

Another point is that for some types it may be necessary to define if
the goal of regex validation is to exclude all invalid content along
with some valid (but unusual) content, or to include all valid content
along with some invalid (but exotic) content. Eg. have overly strict
or overly permissive matching.

With scientific notation there is an infinite number of patterns to
write a number [0-1], for example:

...
500e-3 : \d\d\d\.?\d*[eE]-0*[3-9]\d*
50e-2 : \d\d\.?\d*[eE]-0*[2-9]\d*
5e-1 : \d\.?\d*[eE]-0*[1-9]\d*
0.5e0 : 0*\.\d+[eE](0+|-\d+)
0.05e1 : 0*\.0\d+[eE]((0*[0-1])|-\d+)
0.005e2 : 0*\.00\d+[eE]((0*[0-2])|-\d+)
...

I can think of regexes to match some of them (see above) but not all
of the possible patterns without also matching larger numbers since
the number of digits in the coefficient before (or after)  the decimal
point has to be taken into account when matching the exponent.

A simple solution would be to require that numbers in scientific
notation have coefficients that range from 0.01 to 99.9999.. which
would cover all useful uses of the notation but still seems of course
arbitrary. It would be somewhat similar to the leading zeroes
restriction. With such a restriction, (slightly complicated) regexes
are possible.

-Andreas

On Sun, Jul 22, 2018 at 2:40 PM Don Brutzman <brutzman at nps.edu> wrote:
>
> Thanks for continued work Andreas.  Will study closely.
>
> A good way to think about regular expressions is that they define grammars with arbitrary expressive power.  Thus we will eventually be able to devise regexes for any regular data type in X3D.  Truly anything repeatable/regular is possible with regex expressibility, and they have many aspects similar to a programming language.
>
> Am thinking that we might do well to first deconstruct, focusing on things like a single value [0..1] and then slowly building up, reviewing the existing regexes.  That way our floating-point regexes will build upon the same primitives throughout.
>
> Will shift focus to junit5 testing at this point to facilitate experimentation.  We can put our test cases both on regex101.com and also in test suite, both for coverage and for independent checks.
>
> On 7/22/2018 10:31 AM, Andreas Plesch wrote:
> > Another update for SFcolorRGBA
> >
> > https://regex101.com/r/R3LXnN/4/tests
> >
> > which deals with more complicated zeroes.
> >
> > Here is also the original but with anchors which are implied in xml:
> >
> > https://regex101.com/r/sy4qJ7/2
> >
> > and without the final '?' to make everything optional. This works with
> > the first unit tests so it was actually working pretty well.
> > I added some more tests for which it does not work.
> >
> > https://regex101.com/r/sy4qJ7/3 is the same regex written more
> > concisely, using the \d character class and {3} repetitions.
> >
> >
> >
> > On Sat, Jul 21, 2018 at 6:41 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
> >>
> >> I added some more unit tests and comments here:
> >>
> >> https://regex101.com/r/R3LXnN/2
> >>
> >> The same regex would work for SFColor with {2} repetitions instead of {3}.
> >>
> >> It looks like the original regex allowed for positive exponents as
> >> long as the mantissa was <1. But this would allow 0.9e99. It also did
> >> not match 10e-3, I think, which may actually occur.
> >>
> >>
> >>
> >>
> >> On Sat, Jul 21, 2018 at 4:37 PM Andreas Plesch <andreasplesch at gmail.com> wrote:
> >>>
> >>> I started from scratch and came up with this regex:
> >>>
> >>> https://regex101.com/r/R3LXnN/1/tests
> >>>
> >>> I also added a tricky unit test case which has 10e-3 as a valid but
> >>> hard to validate value. I cannot see how a regex can catch that but
> >>> still try to enforce the <=1.0 condition.
> >>>
> >>> Similary, 0.0001e+3 would be hard to validate but valid even though
> >>> unusual. Currently, the regex above does not allow for any positive
> >>> exponents.
> >>>
> >>> So perhaps the regex should not try and just accept all numbers with
> >>> scientific notation, regardless of their value ? Currently, the regex
> >>> is strict and will only <= 1.0 values.
> >>>
> >>> -Andreas
> >>>
> >>>
> >>> On Sat, Jul 21, 2018 at 1:18 PM <x3d-public-request at web3d.org> wrote:
> >>>>
> >>>> Send x3d-public mailing list submissions to
> >>>>          x3d-public at web3d.org
> >>>>
> >>>> To subscribe or unsubscribe via the World Wide Web, visit
> >>>>          http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> >>>> or, via email, send a message with subject or body 'help' to
> >>>>          x3d-public-request at web3d.org
> >>>>
> >>>> You can reach the person managing the list at
> >>>>          x3d-public-owner at web3d.org
> >>>>
> >>>> When replying, please edit your Subject line so it is more specific
> >>>> than "Re: Contents of x3d-public digest..."
> >>>>
> >>>>
> >>>> Today's Topics:
> >>>>
> >>>>     1. Re: X3D regexes for colorRGBA, image need work (Don Brutzman)
> >>>>
> >>>>
> >>>> ----------------------------------------------------------------------
> >>>>
> >>>> Message: 1
> >>>> Date: Sat, 21 Jul 2018 10:17:10 -0700
> >>>> From: Don Brutzman <brutzman at nps.edu>
> >>>> To: X3D Graphics public mailing list <x3d-public at web3d.org>
> >>>> Subject: Re: [x3d-public] X3D regexes for colorRGBA, image need work
> >>>> Message-ID: <5f45b810-826f-9182-21b0-8873f0aa5f4a at nps.edu>
> >>>> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> >>>>
> >>>> following Andreas' earlier example, have added some regex tests for SFColorRGBA to regex101.com (it allows login using github account).
> >>>>
> >>>> SFColorRGBA tests for default value pass but illegal values (including empty value) are incorrectly accepted, leading to RED test failure.  screenshot attached, clearly something is awry.
> >>>>
> >>>> https://regex101.com/r/sy4qJ7/1
> >>>>
> >>>> http://www.web3d.org/specifications/X3dRegularExpressions.html#SFColorRGBA
> >>>>
> >>>> On 6/28/2018 8:45 PM, Don Brutzman wrote:
> >>>>> Newly installed X3DJSAIL tests reveal:
> >>>>>
> >>>>> SFColorRGBAObject.initialize() problem: failed to match default value DEFAULT_VALUE [F at 481ba2cf
> >>>>> MFColorRGBAObject.initialize() problem: failed to match default value DEFAULT_VALUE [F at 12d2ce03
> >>>>> SFImageObject.initialize() problem: failed to match default value DEFAULT_VALUE [I at 6239aba6
> >>>>> MFImageObject.initialize() problem: failed to match default value DEFAULT_VALUE [I at 4f3bbf68
> >>>>>
> >>>>> Regexes can be found in schema documentation for each? Simple type at
> >>>>>
> >>>>> http://www.web3d.org/specifications/X3dSchemaDocumentation4.0/x3d-4.0.html
> >>>>>
> >>>>> and further documented at
> >>>>>
> >>>>> http://www.web3d.org/specifications/X3dRegularExpressions.html#SFColorRGBA
> >>>>> http://www.web3d.org/specifications/X3dRegularExpressions.html#SFImage
> >>>>>
> >>>>> Anyone want to tackle these?
> >>>> all the best, Don
> >>>> --
> >>>> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> >>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> >>>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> >>>> -------------- next part --------------
> >>>> A non-text attachment was scrubbed...
> >>>> Name: RegexTestsSFcolorRGBA.png
> >>>> Type: image/png
> >>>> Size: 63284 bytes
> >>>> Desc: not available
> >>>> URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20180721/68fb490f/attachment.png>
> >>>>
> >>>> ------------------------------
> >>>>
> >>>> Subject: Digest Footer
> >>>>
> >>>> _______________________________________________
> >>>> x3d-public mailing list
> >>>> x3d-public at web3d.org
> >>>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
> >>>>
> >>>>
> >>>> ------------------------------
> >>>>
> >>>> End of x3d-public Digest, Vol 112, Issue 43
> >>>> *******************************************
> >>>
> >>>
> >>>
> >>> --
> >>> Andreas Plesch
> >>> Waltham, MA 02453
> >>
> >>
> >>
> >> --
> >> Andreas Plesch
> >> Waltham, MA 02453
> >
> >
> >
>
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman



-- 
Andreas Plesch
Waltham, MA 02453



More information about the x3d-public mailing list