[X3D-Public] containerField is should or must ? large attributes; compression

[Don Brutzman]

[summary:  lookouts have sighted large icebergs ahead, port and starboard...]

On 8/6/2010 8:18 AM, Alan Hudson wrote:
> On 8/5/2010 6:59 PM, Tony Parisi wrote:

actually Braden wrote the second-level quote, i believe:

>  >> There's no question that containerField is made necessary by other
>  >> aspects of the XML encoding. Unfortunately, time has shown that those
>  >> aspects are themselves poor design choices.

certainly our knowledge and skills have grown over time, which is good.
however not sure what "poor design choices" means.

>  >> fieldValue wrapper nodes were talked about some months ago as an
>  >> approach to resolve issues with overly large attribute values. These
>  >> wrappers also render containerField unnecessary; and in light of their
>  >> availability, it probably makes sense to deprecate containerField.

well, <fieldValue> element certainly is an interesting alternative possibility.
especially since that functionality has already been implemented as part of
ProtoInstance.  this would seem to minimize "language bloat" issues.

so the idea is appealing.  has anybody yet:
- implemented it in a player, or estimated the work-effort involved?
- written an entry for the X3D v3.3 proposed changes wiki?
- written potential prose for the specification?

http://www.web3d.org/membership/login/memberwiki/index.php?title=X3D_v3.3_Specification_Changes
p.s. discussion topic for next Wednesday's call:  can this wiki go open?

also has anybody yet:
- estimated the impact on X3D scenes?  for Immersive Profile, the
   Collision proxy example is well known.  however there are a
   number of advanced components (shaders, layers/layouts, nurbs etc.)
   where the clutter/obfuscation of parent-child node relationships
   in a scene might become severe.  heck, we're still having trouble
   collecting examples for several of these using any syntax.

those 4 points are paths forward that allow us to implement/evaluate.

any fix needs to be carefully scoped to avoid the "wrapper tag"
morass of a decade ago.  it also helps if a fix has a corresponding
problem statement, so we can measure whether proposed solutions are
useful...  which leads us to the next potential iceberg:

> As it stands now we avoid the XML encoding for any real sized files. I
> am a fan of being able to completely validate the file. So when I teach
> X3D I use the XML encoding to help them catch errors. But for
> production usage we stick with .x3dv or .x3db.

well we have something like 3000 examples online and they feel pretty
real, especially since they are debuggable thanks to XML validation.
there are a _lot_ of production XML applications and parsers available.

the .x3d XML versions are also best for maintaining "master" copies in
version control, since conversion into other forms is easier when you
start from XML.  those conversions from .x3d to .x3dv .x3db .html and
.wrl are also automated.  if someone rewrote the XSLT converters using
SAX or STAX then likely any memory issues associated with conversion
would go away.

for jumbo attributes, it was pointed out that parsing problems are a
problem of the implementation, not X3D or XML design.  wondering if any
bug reports were submitted, wondering if any better parsers have been
found.

another pretty simple solution to jumbo attributes is to split a huge
node into multiple smaller nodes.  of note is that there are maximum
limits to geometry size specified in the specification, these are found
in each of the X3D Profile definitions in the specification itself.
for example
http://www.web3d.org/x3d/specifications/ISO-IEC-19775-1.2-X3D-AbstractSpecification/Part01/immersive.html

table excerpt for IFS:
> IndexedFaceSet 10 vertices per face. 5000 faces. Less than 15,000 indices. 10 vertices per face. 5000 faces. 15,000 indices in any index field.

the ConformanceNist Test Suite has a number of test scenes designed to
check player conformance with specification limits.  it was designed to
VRML97 parameters but most of those required-support values carried over
unchanged to X3D.  Links and 5000-face IndexedFaceSet example  are at:

http://www.web3d.org/x3d/content/examples/ConformanceNist
http://www.web3d.org/x3d/content/examples/ConformanceNist/Geometry/IndexedFaceSet
http://www.web3d.org/x3d/content/examples/ConformanceNist/Geometry/IndexedFaceSet/5000faces.x3d

A quick test of this scene in BS Contact, Instant Reality, Octaga and
Xj3D rendered speedily and without any console complaint.

checking the spec, comparing it to the test suite, and creating more
test scenes is a useful way to contribute, if anyone is interested.

If the question is "how big is required?" then spec has already answered.
If the question is "how big do you want to go?" then that seems to need
further justification.  Anyone can pick an abitrary number, anyone
else can double that, etc. etc. which isn't very helpful.  Working
through a precise rationale for the correct amounts of geometry support
is certainly important, especially if we hope to converge on an
implementable/deployable Mobile Profile.

regarding "what problem are we solving?" i hope it is worth pointing out
that the .x3d XML encoding currently meets all design requirements of
the X3D Specification.

of course any worry about jumbo attributes might also be reduced a bit
by the X3D Compressed Binary Encoding.  perhaps more people could help
with that?  i hope to discuss possible code changes to ParaView with
Kristian Sons this week, and NPS will try to match the changes in Xj3D
if we can.  getting the X3D Specification examples converted into .x3db
equivalently is our final milestone to complete the approval process
for the recent last-minute errata fixes we submitted  to ISO.
http://www.web3d.org/x3d/content/examples/Basic/X3dSpecification

if we look to the XML community to see who else is interested in
performance of really large numeric scenes, then a lot of work
is found in the standards effort for Efficient XML Interchange (EXI).
EXI is a W3C Candidate Recommendation.  http://www.w3.org/XML/EXI


any way, it is good to know there is still interest in grappling with
these and other challenges.  hopefully these dialogs will help our
big group continue to grow in our common understanding of goals and
solutions.

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br      brutzman at nps.edu
Watkins 270   MOVES Institute, Monterey CA 93943-5000 USA  +1.831.656.2149
X3D, virtual worlds, underwater robots     http://faculty.nps.edu/brutzman