[x3d-public] Wishlist for view3dscene

[John Carlson]

To keep numbers like 0.1, 0.1, use “BigNums” (google bigfloat
pascal)—arbitrary precision numbers, which may be done in an author-time
converter.  This is not the same as runtime, where you need the performance
of binary representations.

I hope this helps.

Solution:  Perhaps a build time preprocessor which filters types and/or
annotations could be used?

Hopefully the author-time visualization is similar as the runtime
visualization.  I’m not sure if Joe does author-time visualization.  I
don’t, I use vim.   We’re probably a dying breed.

I will look into “BigNums” in JavaScript for X3DJSONLD.

Java has BigNums, but I don’t know if Xj3D or X3D-Edit uses them.

Probably 3D graphics libraries do not use BigNums.  I don’t see any reason
why converters from encoding to encoding can’t use BigNums, except sharing
code with the runtime.  See above solution.

Good discussion!

John

On Sun, Feb 12, 2023 at 2:35 PM Joseph D Williams <joedwil at earthlink.net>
wrote:

> I’ll just let this ride with you thinking that if I type one number you
> can go ahead and change it to something else just because it is too much
> work to keep it the same. I don’t want your runtime numbers, I want to see
> it run with my authortime numbers. I will decide what resolution is right
> when I  publish my user code. Maybe that is why I am stuck in Notepad, but
> really, any gui is not going to autocorrect, even something like x3dedit.
>
> True it is bound to change when you are exporting your runtime code to do
> a transcoding like you did for classic to xml, by loading the thing, then
> export with no regard to input. To me, that mangled the thing almost beyond
> recognition and puffed it up by greatly. Process needs improvement, with
> more reference to the input content, Thank You.
>
> True what really happens under the covers may be off by rounding float
> please just recall that human-readable text is the final test of whether it
> is X3D or a blob. As a browser maker, when you choose a solution it should
> be on the side of author convenience and readability and validation rather
> than browser implementer idealism, convenience, or reduced workload.
>
> Thanks Again for your progress with your browser.
>
> Joe
>
>
>
> *From: *Michalis Kamburelis <michalis.kambi at gmail.com>
> *Sent: *Saturday, February 11, 2023 4:47 PM
> *To: *John Carlson <yottzumm at gmail.com>
> *Cc: *Joseph D Williams <joedwil at earthlink.net>; X3D Public Mailing List
> (x3d-public at web3d.org) <x3d-public at web3d.org>
> *Subject: *Re: [x3d-public] Wishlist for view3dscene
>
>
>
> It is true that on output -- we may produce more digits than we saw in
>
> input, because internally the numbers get converted into floats, and
>
> floats cannot represent all the numbers precisely.
>
>
>
> That is, putting
>
>
>
>    0.0001
>
>
>
> into float (4-byte IEEE-754 single) results in a number equal
>
>
>
>   0.00009999...
>
>
>
> When writing, we don't have anymore the information whether you
>
> actually specified literally  "0.0001" or "0.00009999..".
>
>
>
> That said, solution with storing a text representation of every input
>
> float  (including text representation of every array of floats) is
>
> just not feasible. That's a lot of information, and additional
>
> maintenance about what to do when these numbers change (what if code
>
> manipulates those numbers? including X3D scripts / SAI?).
>
>
>
> No X3D browser, no 3D software in general (any game engine, 3D viewer
>
> for any format) that allows to render and manipulate the 3D content,
>
> is storing the original numbers in text form, from what I know. That
>
> would be just a lot of extra work. The software in practice has a
>
> choice of outputting the numbers with max precision (yes, it means you
>
> see more digits than on input) or making an assumption that some
>
> digits are not important for user (and then, users who specify very
>
> small numbers, and count on their precision, are at a loss).
>
>
>
> Unless you have a specific software that only does conversion, never
>
> manipulation, then maybe can afford to just store numbers as text.
>
> Alas, that is not CGE/view3dscene (and I claim that we're in same spot
>
> as all other X3D browsers or 3D viewers; if they output numbers in
>
> decimal, doing float->text conversion, they have to assume some
>
> precision that may not match the input).
>
>
>
> Regards,
>
> Michalis
>
>
>
>
>
> sob., 11 lut 2023 o 22:26 John Carlson <yottzumm at gmail.com> napisał(a):
>
> >
>
> > We should check all of my synthesis tools in X3DJSONLD, i know some of
> them extend the precision.
>
> >
>
> > The best thing to do is create a small file which shows the problem, and
> then run it through our processes.  I’ll do that when I get home.
>
> >
>
> > If you find a bug, please report:
>
> >
>
> > https://github.com/coderextreme/X3DJSONLD/issues
>
> >
>
> > Just look at current data files:
>
> >
>
> > https://github.com/coderextreme/X3DJSONLD/tree/master/src/main/data/*
>
> >
>
> >
>
> >
> https://github.com/coderextreme/X3DJSONLD/tree/master/src/main/*/net/*/data/*
>
> >
>
> > I’m going through a major revision, bringing the  product up to ES6.
>
> >
>
> > Thanks!
>
> >
>
> > John
>
> >
>
> > On Sat, Feb 11, 2023 at 3:02 PM Joseph D Williams <joedwil at earthlink.net>
> wrote:
>
> >>
>
> >> The option is to present the text in text form. What you do inside
> should not bother my right to read and interact with the text. And please
> don’t give me more resolution than I request in my user code. Don’t
> autocorrect how many digits I specify. Hanim doesn’t need anymore than 4
> anywhere. Think of how somebody looks  at the code and sees these endless
> cols of 10 of more digits, not even consistent and not readable  anymore.
>
> >>
>
> >> Joe
>
> >>
>
> >>
>
> >>
>
> >>
>
> >>
>
> >> From: John Carlson
>
> >> Sent: Friday, February 10, 2023 11:20 AM
>
> >> To: Michalis Kamburelis
>
> >> Cc: X3D Public Mailing List (x3d-public at web3d.org)
>
> >> Subject: Re: [x3d-public] Wishlist for view3dscene
>
> >>
>
> >>
>
> >>
>
> >>  There should be an option to convert binary vertex and index data to
> ASCII, if only during authoring.
>
> >>
>
> >>
>
> >>
>
> >> Remember that Joe/designers like to tweak.
>
> >>
>
> >>
>
> >>
>
> >> I find that converting JSON through JavaScript massively increases
> precision, but i haven’t looked at why yet.  Maybe JSON/JavaScript
> numerical libraries should be responsible for keeping precise, accurate
> numbers, perhaps by storing alternate encodings for I/O and computation.
>
> >>
>
> >>
>
> >>
>
> >> One can always convert to EXI and back, to serve ASCII purposes.
>
> >>
>
> >>
>
> >>
>
> >> John
>
> >>
>
> >>
>
> >>
>
> >> On Fri, Feb 10, 2023 at 12:36 PM Michalis Kamburelis <
> michalis.kambi at gmail.com> wrote:
>
> >>
>
> >> We use optional scientific notation (with exponent) when writing
>
> >> numbers in CGE/view3dscene. To be precise, we use a formatting option
>
> >> that outputs either a scientific or decimal notation, whichever is
>
> >> shorter. The scientific notation is not that much shorter to win
>
> >> usually.
>
> >>
>
> >> As for round-trips: you must have maximum precision to have reliable
>
> >> round-trips, I believe :) Otherwise you will always discover use-cases
>
> >> when you loose some digits, and they will be important for someone.
>
> >>
>
> >> glTF does what is obvious, and I'd say X3D should follow, this is
>
> >> something I'd like to work on for X3D 4.1 (
>
> >> https://github.com/michaliskambi/x3d-tests/wiki/Binary-meshes ).
>
> >> Namely, the floats in per-vertex data are always stored as binary.
>
> >> Only the floats like material parameters are stored as text in JSON.
>
> >>
>
> >> X3D binary encoding is one solution here, but it is not wide-spread,
>
> >> e.g. I hope zero hope that Blender exporter would output X3D binary
>
> >> encoding someday (there's too many other critical issues in Blender
>
> >> X3D exporter). And X3D binary encoding doesn't make data interleaved,
>
> >> so it doesn't solve the issue of effectively delivering it to GPU.
>
> >>
>
> >> "keep numbers and arrays as strings" -- no, this really wouldn't help
>
> >> with anything. The X3D browser has to parse the text to numbers at
>
> >> some point, to process the points, upload them to GPU etc. Right now
>
> >> the bottleneck of reading real-life X3D is always, in my experience,
>
> >> "your code effectively has to make million of text-to-float
>
> >> conversions". I believe this is a bottleneck regardless of the
>
> >> programming language, 1000 * 1000 calls to text-to-float is always
>
> >> going to be much slower than reading binary data (where you say "this
>
> >> is my pointer to 1000 * 1000 * 4 bytes, just copy there the contents
>
> >> of this file").
>
> >>
>
> >> Regards,
>
> >> Michalis
>
> >>
>
> >> pt., 10 lut 2023 o 18:57 John Carlson <yottzumm at gmail.com> napisał(a):
>
> >> >
>
> >> > Not sure how feasible this is with Pascal, etc.   Perhaps consider
> scientific or engineering notation with exponents.
>
> >> >
>
> >> > In most cases i see, extending precision significantly increases file
> size, and makes it harder to do round trips.
>
> >> >
>
> >> > I don’t know if there’s a JavaScript or Python solution for this.
>
> >> >
>
> >> > Perhaps we should not implement number types in our synthesized
> programs, and keep numbers and arrays as strings.   Then we have to ask
> what is done in glTF.
>
> >> >
>
> >> > John
>
> >> >
>
> >> > On Fri, Feb 10, 2023 at 7:13 AM Michalis Kamburelis <
> michalis.kambi at gmail.com> wrote:
>
> >> >>
>
> >> >> Thanks, and sorry for a delay in answering!
>
> >> >>
>
> >> >> The "description" field has been added to texture nodes and few more,
>
> >> >> as you can see in another thread on x3d-public :)
>
> >> >>
>
> >> >> As for digits precision when saving: it's not so easy, in general
> it's
>
> >> >> (one of) the problems that come from X3D using text representation
> for
>
> >> >> floats in per-vertex data. Namely, if we cut the saved numbers to 4
>
> >> >> decimal places, I'm 90% sure that I will get a valid bugreport from
>
> >> >> someone:
>
> >> >>
>
> >> >> """
>
> >> >> Hey, my stuff no longer works! My models (in glTF, X3D, whatever)
> have
>
> >> >> vertexes at coordinates like Z = 0.00001, 0.00002, 0.00003. And now,
>
> >> >> after processing and saving to X3D using your tools, they are all
>
> >> >> equal zero, resulting in bugs (things start to fight in Z buffer).
>
> >> >> """
>
> >> >>
>
> >> >> Surely we could add a configuration option for it, but it feels
>
> >> >> counter-productive, when my recommendation to everyone would be still
>
> >> >> "use the maximum precision, to be safe". I would add a feature that I
>
> >> >> explicitly say should not be used.
>
> >> >>
>
> >> >> In the end, the longer text representation for numbers only hurts the
>
> >> >> file size. And I don't believe we can really fix it by outputting
> less
>
> >> >> digits. The right solution is storing per-vertex data in a binary
>
> >> >> blob, when each float is binary 4 bytes, so it's
>
> >> >>
>
> >> >> - efficient storage
>
> >> >> - efficient reading by X3D implementation to a "float[] array", no
>
> >> >> need for text parsing.
>
> >> >>
>
> >> >> Regards,
>
> >> >> Michalis
>
> >> >>
>
> >> >>
>
> >> >>
>
> >> >>
>
> >> >> śr., 28 gru 2022 o 21:00 Brutzman, Donald (Don) (CIV)
>
> >> >> <brutzman at nps.edu> napisał(a):
>
> >> >> >
>
> >> >> > Hi Michalis. Joe and I are really enjoying the many excellent
> features of view3dscene.
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > Here are some suggested refinements and improvements.
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > View3dscene converter: round off significant digits, rather than
> 10+ decimal places.  (For example, suggested values in HAnim spec go to 4
> decimal places.
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > View3dscene: note that we have added ‘description’ field to many
> nodes in X3D4, including ImageTexture.
>
> >> >> >
>
> >> >> > 1 warnings:
>
> >> >> > X3D: Unknown X3D field name (unhandled X3D XML attribute)
> "description" in node "ImageTexture"
>
> >> >> > Scene URL: "C:\x3d-code\www.web3d.org
> \x3d\content\examples\HumanoidAnimation\Skin\JoeSkeletonSkinSiteSaluteWalk.x3d".
>
> >> >> > Use "File->View Warnings" menu to view these warnings again.
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > Coming soon: import test of three models in HAnim Motion
> specification.
>
> >> >> >
>
> >> >> >
> https://www.web3d.org/documents/specifications/19774/V2.0/MotionDataAnimation/MotionDataAnimation.html
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > Again thanks for great work.
>
> >> >> >
>
> >> >> >
>
> >> >> >
>
> >> >> > all the best, Don
>
> >> >> >
>
> >> >> > --
>
> >> >> >
>
> >> >> > Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> >> >> >
>
> >> >> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> >> >> >
>
> >> >> > X3D graphics, virtual worlds, Navy robotics https://
> faculty.nps.edu/brutzman
>
> >> >> >
>
> >> >> >
>
> >> >>
>
> >> >> _______________________________________________
>
> >> >> x3d-public mailing list
>
> >> >> x3d-public at web3d.org
>
> >> >> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
> >>
>
> >>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230212/68be7689/attachment-0001.html>