[x3d-public] Wishlist for view3dscene

Sat Feb 11 13:02:39 PST 2023

The option is to present the text in text form. What you do inside should not bother my right to read and interact with the text. And please don’t give me more resolution than I request in my user code. Don’t autocorrect how many digits I specify. Hanim doesn’t need anymore than 4 anywhere. Think of how somebody looks  at the code and sees these endless cols of 10 of more digits, not even consistent and not readable  anymore. 
Joe

From: John Carlson
Sent: Friday, February 10, 2023 11:20 AM
To: Michalis Kamburelis
Cc: X3D Public Mailing List (x3d-public at web3d.org)
Subject: Re: [x3d-public] Wishlist for view3dscene

 There should be an option to convert binary vertex and index data to ASCII, if only during authoring.

Remember that Joe/designers like to tweak.

I find that converting JSON through JavaScript massively increases precision, but i haven’t looked at why yet.  Maybe JSON/JavaScript numerical libraries should be responsible for keeping precise, accurate numbers, perhaps by storing alternate encodings for I/O and computation.

One can always convert to EXI and back, to serve ASCII purposes.

John 

On Fri, Feb 10, 2023 at 12:36 PM Michalis Kamburelis <michalis.kambi at gmail.com> wrote:
We use optional scientific notation (with exponent) when writing
numbers in CGE/view3dscene. To be precise, we use a formatting option
that outputs either a scientific or decimal notation, whichever is
shorter. The scientific notation is not that much shorter to win
usually.

As for round-trips: you must have maximum precision to have reliable
round-trips, I believe :) Otherwise you will always discover use-cases
when you loose some digits, and they will be important for someone.

glTF does what is obvious, and I'd say X3D should follow, this is
something I'd like to work on for X3D 4.1 (
https://github.com/michaliskambi/x3d-tests/wiki/Binary-meshes ).
Namely, the floats in per-vertex data are always stored as binary.
Only the floats like material parameters are stored as text in JSON.

X3D binary encoding is one solution here, but it is not wide-spread,
e.g. I hope zero hope that Blender exporter would output X3D binary
encoding someday (there's too many other critical issues in Blender
X3D exporter). And X3D binary encoding doesn't make data interleaved,
so it doesn't solve the issue of effectively delivering it to GPU.

"keep numbers and arrays as strings" -- no, this really wouldn't help
with anything. The X3D browser has to parse the text to numbers at
some point, to process the points, upload them to GPU etc. Right now
the bottleneck of reading real-life X3D is always, in my experience,
"your code effectively has to make million of text-to-float
conversions". I believe this is a bottleneck regardless of the
programming language, 1000 * 1000 calls to text-to-float is always
going to be much slower than reading binary data (where you say "this
is my pointer to 1000 * 1000 * 4 bytes, just copy there the contents
of this file").

Regards,
Michalis

pt., 10 lut 2023 o 18:57 John Carlson <yottzumm at gmail.com> napisał(a):
>
> Not sure how feasible this is with Pascal, etc.   Perhaps consider scientific or engineering notation with exponents.
>
> In most cases i see, extending precision significantly increases file size, and makes it harder to do round trips.
>
> I don’t know if there’s a JavaScript or Python solution for this.
>
> Perhaps we should not implement number types in our synthesized programs, and keep numbers and arrays as strings.   Then we have to ask what is done in glTF.
>
> John
>
> On Fri, Feb 10, 2023 at 7:13 AM Michalis Kamburelis <michalis.kambi at gmail.com> wrote:
>>
>> Thanks, and sorry for a delay in answering!
>>
>> The "description" field has been added to texture nodes and few more,
>> as you can see in another thread on x3d-public :)
>>
>> As for digits precision when saving: it's not so easy, in general it's
>> (one of) the problems that come from X3D using text representation for
>> floats in per-vertex data. Namely, if we cut the saved numbers to 4
>> decimal places, I'm 90% sure that I will get a valid bugreport from
>> someone:
>>
>> """
>> Hey, my stuff no longer works! My models (in glTF, X3D, whatever) have
>> vertexes at coordinates like Z = 0.00001, 0.00002, 0.00003. And now,
>> after processing and saving to X3D using your tools, they are all
>> equal zero, resulting in bugs (things start to fight in Z buffer).
>> """
>>
>> Surely we could add a configuration option for it, but it feels
>> counter-productive, when my recommendation to everyone would be still
>> "use the maximum precision, to be safe". I would add a feature that I
>> explicitly say should not be used.
>>
>> In the end, the longer text representation for numbers only hurts the
>> file size. And I don't believe we can really fix it by outputting less
>> digits. The right solution is storing per-vertex data in a binary
>> blob, when each float is binary 4 bytes, so it's
>>
>> - efficient storage
>> - efficient reading by X3D implementation to a "float[] array", no
>> need for text parsing.
>>
>> Regards,
>> Michalis
>>
>>
>>
>>
>> śr., 28 gru 2022 o 21:00 Brutzman, Donald (Don) (CIV)
>> <brutzman at nps.edu> napisał(a):
>> >
>> > Hi Michalis. Joe and I are really enjoying the many excellent features of view3dscene.
>> >
>> >
>> >
>> > Here are some suggested refinements and improvements.
>> >
>> >
>> >
>> > View3dscene converter: round off significant digits, rather than 10+ decimal places.  (For example, suggested values in HAnim spec go to 4 decimal places.
>> >
>> >
>> >
>> > View3dscene: note that we have added ‘description’ field to many nodes in X3D4, including ImageTexture.
>> >
>> > 1 warnings:
>> > X3D: Unknown X3D field name (unhandled X3D XML attribute) "description" in node "ImageTexture"
>> > Scene URL: "C:\x3d-code\www.web3d.org\x3d\content\examples\HumanoidAnimation\Skin\JoeSkeletonSkinSiteSaluteWalk.x3d".
>> > Use "File->View Warnings" menu to view these warnings again.
>> >
>> >
>> >
>> > Coming soon: import test of three models in HAnim Motion specification.
>> >
>> > https://www.web3d.org/documents/specifications/19774/V2.0/MotionDataAnimation/MotionDataAnimation.html
>> >
>> >
>> >
>> > Again thanks for great work.
>> >
>> >
>> >
>> > all the best, Don
>> >
>> > --
>> >
>> > Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
>> >
>> > Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
>> >
>> > X3D graphics, virtual worlds, Navy robotics https:// faculty.nps.edu/brutzman
>> >
>> >
>>
>> _______________________________________________
>> x3d-public mailing list
>> x3d-public at web3d.org
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230211/39ecb088/attachment.html>