[x3d-public] new feature in X3dTidy stylesheet: numeric significant-digits precision using regular expression (regex)

[Don Brutzman]

BLUF: can now reduce unnecessary numeric bloat in X3D models.

Problem statement: a long-running difficulty in many X3D models is that
floating point numbers are often written into files with an extraordinary
number of digits following the decimal point.

This doesn't make the models incorrect, but it often makes them unwieldy
for editing and Web use.  Few of us need sub-nanometer precision when we
look at a 3D model (cough cough).

Usually this is not a big deal when writing software, since programming
languages typically make it easy to format floating point numbers.  However
various tools and actual practice is widely different, and that can result
in VERY LARGE MODELS when exporting X3D.  There are many such products in
the X3D Examples Archives.  As a case in point, machine tolerances needed
for CAD production is more precise than the human eye can actually discern.

Thus I began pursuing a string-based approach to reducing floating-point
precision, to facilitate the right-sizing of model files in our archives
and elsewhere.

An important feature of X3D is that most models are in meters, unless
indicated otherwise by presence of a UNIT statement.  So that consistency
can be exploited.  For most purposes, am thinking that* 4 significant
digits* is plenty for most visual models (and 3D printed models) since

   - *0.0001 meters*  equals *one-tenth of a millimeter*.
   - *0.0001 radians* equals *0.0057 degrees*.
   - RGB color changes past 2 or 3 digits are very subtle and perhaps
   imperceptible.

Searching for a regular expression (regex) to accomplish this task yielded
minimal results... and so I've developed one.  Am happy to announce it here.

   - *X3D Regular Expressions (regexes)*
   - X3D Regular Expressions (regexes) are used to validate the correctness
   of string and numeric array values in an X3D scene.
   - https://www.web3d.org/specifications/X3dRegularExpressions.html
   - numeric significant-digits precision filtering
   <https://www.web3d.org/specifications/X3dRegularExpressions.html#SignificantDigitsPrecision>

*Numeric significant-digits precision filtering* is often desirable. Many
geometry-export software tools naively emit floating-point numbers of
extra-large length that result in unnecessary file size, download delays,
and wasted computation when parsing. Such inefficiency is often noticeable
and problematic for large model files, reducing performance and inhibiting
scalability.

It is helpful to remember that regexes have general expressive power
(similar to other programming languages) that can search, and optionally
replace, any string pattern that can be unambiguously expressed. Typically
the control of numeric significant-digit precision is controlled by
software formatting of numeric outputs, not by processing the character
strings in an XML document. No suitable regex pattern was found in any of
the references listed. Nevertheless, a Web search reveals a candidate
solution in the StackOverflow post "Reduce float precision using regexp
<https://stackoverflow.com/questions/51947938/reduce-float-precision-using-regexp-in-swift>"
with a corresponding regex demo <https://regex101.com/r/yINM4e/latest> on
Regex 101 website.

Here is a refined pattern for floating-point arrays as string-based X3D
attributes.


   - ignores leading      whitespace       preceding the numeric value,
      - returns necessary (first  group)     including the optional +/-
      sign character and then leading digits,
      - returns optional   (second group)  including the decimal point and
      (in this case) *4* trailing digits,
      - ignores omitted     excess digits     that may follow beyond the
      desired precision,
      - returns optional   (third  group)    including any trailing
      scientific notation, if present (for example *E-04*), and
      - ignores trailing      whitespace       including optional comma,
      which is an allowed X3D whitespace character.

Significant-digits Precision Pattern
Truncate following *4* digits of precision after decimal point, without
rounding
*XML Schema:*   <xs:restriction base="xs:decimal"/> controls type but not
precision or significant digits
*Regex pattern:*  \s*([+-]?\d*)(\.\d{0,*4*})\d*([Ee][+-]?\d+)\s*[,]?
regex101 examples: numeric filter, significant digits precision
<https://regex101.com/r/mjYeZe/latest>

The pattern truncates following the desired digits of precision after
decimal point, without rounding. If present, any trailing zeros within the
significant digits are retained. Thus additional preprocessing might be
necessary for rounding off positive/negative values and trimming trailing
zeros (as accomplished in X3dTidy XSLT stylesheet
<https://www.web3d.org/x3d/stylesheets/X3dTidy.html>).

Since X3D models are typically defined using default units of meters for
length, and radians for angle values, four digits of precision may be
satisfactory for many purposes. The nature and purpose of data in the model
itself matters, so if in doubt use higher precision.


   - *0.0001 meters*  equals *one-tenth of a millimeter*.
      - *0.0001 radians* equals *0.0057 degrees*.

*Warning:* authors are advised to *not* apply this regex if preservation of
full-precision values might be necessary for preserving archival
correctness, or for maintaining verifiable data provenance
<https://en.wiktionary.org/wiki/provenance> of an original 3D dataset.


This capability is now deployed in X3dTidy.xslt stylesheet, and the interim
release of X3D-Edit <https://www.web3d.org/x3d/tools/X3D-Edit/X3D-Edit.html>.
Authors can pick however many significant digits they want to retain, the
offered default is 4 digits.

Thanks to an interesting suggestion by Carol McDonald, the implementation
also includes a related option to retain trailing zeros.  That is sometimes
useful as an indication of expected precision, perhaps even as a contract
requirement.

Here is a screenshot image showing the X3D-Edit panel for X3D Tidy settings
<https://www.web3d.org/x3d/tools/X3D-Edit/images/X3D-EditX3D-TidyPanel.png>
that
authors can use.

I am now going through all of our (nearly 4,000) examples by applying X3D
Tidy during the global build process, reviewing change logs, then visually
confirming results are OK before committing each update.  The next message
describes results seen in the Humanoid Animation (HAnim) example archives.

Onward we go, am looking forward to further X3D model refinements.  Have
fun with X3D and HAnim!  😃 👍
all the best, Don
-- 
X3D Graphics, Maritime Robotics, Distributed Simulation
Relative Motion Consulting  https://RelativeMotion.info
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20260223/799c28b7/attachment.html>