[x3d-public] ] V4.0 Opendiscussion/workshopon X3DHTMLintegration

[Philipp Slusallek]

Hi Joe,

Am 01.07.2016 um 22:44 schrieb Joe D Williams:
>>> Doing a full WebComponent implementation for Hanim is left as an
> exercise for the reader :-).
> 
> I like the grin at the end of that:). I think you are just getting an
> idea of the depth of the X3D HAnim. I think HAnim is so fundamental
> that you _should_ help a lot. After all, you are telling me that there
> is no problem in hierarchy definitions, binding for deformable skin
> and other geometry, and synchronization, with some script to prove it,
> but I have not yet studied it or seen details of how I implement my
> favorite x3d hanim:)

I think you overlooked the smiley at the end of my text :-).

> For now, please excuse the following. This is soooo basic to me that I
> must get it out.
> 
> http://xml3d.org/xml3d/specification/latest/#dataflow-graph-xflow
> 
> XML3D;
> <mesh type="triangles">
>  <int name="index">0 1 2 2 3 0 4 ... </int>
>  <float3 name="position">-1.0 -1.0 -1.0 ... </float3>
>  <float3 name="normal">0.0 0.0 -1.0 0.0 0.0 ... </float3>
>  <float2 name="texcoord">1.0 0.0 ... </float2>
> </mesh>
> 
> Wow, I can't even bring myself to think of that XML3D code as any form
> of (x)html. I mean sure I can figure it out, but why should I have to?
> 
> This example could look like this is in X3D style:
> 
> X3D:
> <indexedtriangleset
>  index="0 1 2 2 3 0 4 ..."
>  coord="0 1 2 2 3 0 4 ..."
>  normal="0.0 0.0 -1.0 0.0 0.0 ..."
>  texcoord="1.0 0.0 ..."
> </indexedtriangleset>
> 
> To me, the XML3D form is just way too confusing. When I am working
> with this stuff,
> I expect to have at my disposal a sufficiently advanced interface to
> the user code where I just don't need to know or mostly even care
> about the frigging data typing.

I still suggest you read the papers that I sent links to earlier, or our
spec. Then you would better understand, why we use this syntax and why
its is so much more useful. Anyway here is a very short summary.

X3D allows only a fixed set of predefined attributes for each element.
We have a generic mechanism here that is needed as we also allow the
same mechanism to pass arbitrary attribute values to shaders for each
and every primitive. And somehow the system has to know what the data
types are.

We use the same mechanism to also pass data to scripts, light source
shaders, view configurations, WebComponents, and so on. This is why we
call it "generic"!

BTW, it is the same mechanism that the GPU is using to pass data
efficiently to HW (buffers!). So in many case, can we keep this data in
buffers directly on the GPU, apply the shaders and other compute
operations there (even an entire sequence of them), and then use the
results directly for rendering without going back to the CPU.

Note that because of the built-in reuse of data in the flow-graphs of
Xflow we can also reuse these buffers with (partial/intermediate)
results somewhere within a flow-graph.

> First, I recommend the engaging study of the X3D @containerfield as an
> example of use of an attribute to declare an important parameter of
> the element. This study of containerfield reveals it is like the
> <float2> in that it is about the data in the container, not about the
> container so this vital information can surely be set as an attribute
> of the element rather than as its @name or <container>...
> </container>. I mean we just don't need container fields in X3D XML,
> even though we sometimes need to identify an acceptable container.

Before recommending some "hack" like "@containerfield", please again,
first try to understand what you are targeting. The xflow sytax requires
a bit more text but is much more generic and as a result offers
dramatically more functionality that way. Also see below.

> Then you can call that element by a real functionality, such as:
> 
> <mesh type="triangles">
>  <index vartype="int"> 0 1 2 2 3 0 4 ... </index>
>  <position vartype="float3"> -1.0 -1.0 -1.0 ... </position>
>  <normal vartype="float3"> 0.0 0.0 -1.0 0.0 0.0 ... </normal>
>  <texcoord vartype="float2"> 1.0 0.0 ... </texcoord>
> </mesh>
> 
> Or even extend your vocabulary to call the container what it is
> actually containing such as:
> 
> <triangles>
>  <index vartype="int"> 0 1 2 2 3 0 4 ... </index>
>  <position vartype="float3"> -1.0 -1.0 -1.0 ... </position>
>  <normal vartype="float3"> 0.0 0.0 -1.0 0.0 0.0 ... </normal>
>  <texcoord vartype="float2"> 1.0 0.0 ... </texcoord>
> </triangles>

This assumes that we know beforehand the set of all possible attributes
to geometry (and shaders attached to them). Due to programmable shading
(see our work on shade.js as part of XML3D) this set is not limited at
all. So this is suggestion simply not an option!

> Advantage is then you can use a system of defaults that keeps your
> user code from looking lke a bunch strung-together features hung
> together by data type elements that are in no way optimized for human
> readablility and editing but instead only for the convenience of your
> processing scripts.

Yes, we do not optimize for human readability at this low level. Data
buffers (long arrays of numbers) are not very human readable in the
first place -- beyond trivial models!

Where we do care about readability very much is at the higher layers
where you take these buffers and combine them in flexible way, pass them
to shaders, refernce them from other data elements for reuse, specify
the attributes to Web components. At that level its a simple URL or
fragment name refering to single buffer or a collection thereof. Due to
the ability to flexibly combine data elements with each other, this is
extremely elegant and compact, I believe.

> That leaves us with the topic of should the data be in attributes or
> elements which needs a special topic.

Due to the unlimited set of possible names for shader attributes putting
them in attributes is simply not an option!

There are other arguments as well, for example by looking at the
correspondence between text and 3D data -- but let's not go into this.

> And really, since you are defining an indexed triangle geometry, then
> why not use the current official ISO name for that thing?
> There is No doubt that this world-standard functionality for defining
> this type of geometry is officially named IndexedTriangleSet which
> clearly distinguishes it from the default-indexed device that is
> officially named TriangleSet.

The mesh element is used as a single entry point to interpret generic
data elements (from Xflow) as geometry to be rendered. There is not a
single interpretation such but we are supporting a number of them (not
all in the currently released version yet): points, lines, triangles,
trianglesstripts, volume data, and many more. Please note that -- for
obvious reasons -- this is also very similar to how data is passed to
the GPU via OpenGL Draw calls. See comments above.

One can argue if "mesh" is really the best name for this element,
though. And we are considering changes right now. But it will certainly
not be "IndexedTriangleSet" :-).

> More later,
> Thanks Again for your interest in discussing some more about this. I
> hope we can get past this small rant about basic XML3D syntax and dig
> deeper into hanim stuffs.

I am happy to keep answering your questions. But may I again ask you to
first read our papers as this will answer many of your questions already
and save all of us time. It will also allow us to go into more of the
core ideas that at least I consider more interesting.


Best,

	Philipp


> All Best,
> Joe
> 
> ----- Original Message ----- From: "Philipp Slusallek"
> <philipp.slusallek at dfki.de>
> To: "Joe D Williams" <joedwil at earthlink.net>; "'X3D Graphics public
> mailing list'" <x3d-public at web3d.org>
> Sent: Thursday, June 16, 2016 9:54 PM
> Subject: Re: [x3d-public] ] V4.0 Opendiscussion/workshopon
> X3DHTMLintegration
> 
> 
>> Hi Joe, all,
>>
>> FYI: It seemes that my email to the x3d-public list were dropped by
>> the
>> server. Because I did not get a error message, I only found out now
>> (thanks for your help Don, Leonard!). It seems that Leonard will be
>> able
>> to resend them to the list.
>>
>> I think this was/is a good discussion that is worth having on the
>> list
>> archives.
>>
>> Am 16.06.2016 um 22:20 schrieb Joe D Williams:
>>>> but to have a direct
>>>> mapping from the data of the game
>>>
>>> Yes, well then that is not the scope of X3D, at least in this case.
>>> If
>>> you see the kick example, then the 'standard' X3D hanim model
>>> actually
>>> exposes more of what you need in better interactive model than this
>>> example.
>>
>> Maybe it is now. But I would argue that it should be possible to
>> more
>> easily work with data in various formats and not be forced into the
>> one
>> and only format.
>>
>> Not only may you have todo extra work, there might even be features
>> that
>> are not covered (skinning for tangent vectors are such) and extra
>> work
>> done that is not even needed in some situations.
>>
>> I think this aspect of the design is worth reconsidering for a new
>> version. Especially, since an spec/implementation is available that
>> would support that and could be integrated into the new version.
>>
>>> I think what we are seeing in the .heavy animation (discalimer:
>>> haven't
>>> seen the code) is not the actions of a connected hierarcal
>>> skeleton, but
>>> more like just moving the segments (bones) around without exposed
>>> concepts such as a joint center. That is ok, the skeleton may
>>> expose
>>> itself in the anim code. As near as I can tell from the data
>>> without
>>
>> It is a full hierarchical skeleton. We do propagate the
>> transformation
>> of the upper joints downwards as necessary.
>>
>> All the code is in the xml3d.js repository on github, if you want to
>> look at it. The link in the file goes to the minified version where
>> its
>> hardly readable.
>>
>>> seeing the animation user code then this is basically like early
>>> simple
>>> motion capture data dealing with motion captured then edited but
>>> basically built to be played back like a video at some more or less
>>> 'fixed frame rates, but not much processing.
>>
>> It is definitely more than that! In a project with Daimler we are
>> using
>> it for example for doing fully automatic motion synthesis for
>> virtual
>> workes in a factory model to execute some high-level tasks given
>> some
>> written instructions ("pick up part X, use the skrew driver to fix
>> it to
>> the car Y and location Z"). All this is based on semanic annotations
>> of
>> the XML3D model and the shown character animation engine.
>>
>>>> simply create their
>>>> own high-level elements in a modular way using WebComponents and
>>>> publish
>>>> that for others to use.
>>>
>>> more later I hope, but that is the purpose of X3D. To establish
>>> these
>>> practical, bound, data sets that can be animated in a realistic
>>> way.
>>> There are several players that use this model and is the basic
>>> world
>>> standard' simulation humanoidj for detailed simulation and
>>> interaction.
>>>
>>> So, for high level elements, so far I don't see any baetterthan X3D
>>> Hanim. I see that way that the runtime interpolator data is stored
>>> in
>>> you code and I think I can compare is with the way it is presented
>>> in X3D.
>>
>> I still think you are misunderstanding me: The goal of our work os
>> NOT
>> to develop a "better Hanim". It is to have a declarative interface
>> to
>> specify all sorts of geometry and other processingin an easy and
>> high-level way.
>>
>> This can then be used by an web developer to define their own object
>> representations and how they get processed for display, interaction,
>> etc. You are not limited to the way someone defined at some point
>> (as
>> good as that may be, it will never be perfectly suited for anyone).
>>
>> This is something that X3D does not really offer today, and which we
>> think would be useful to add. Just a suggestion.
>>
>> What I am saying in this thread is that our concept is powerful
>> enogh to
>> also implement Hanim.
>>
>>> Anyway, I think I will be away for a week or so but since in hanim,
>>> for
>>> people playing in free and open locations, the high-level elements
>>> of
>>> X3D HAnim must be considered as a tried an true way since the
>>> essentials
>>> of the humanoid, what you need to build and take control are all
>>> mostly
>>> built-in, For example,X3D used to call the thing a Transform but
>>> added
>>> some features and called it a joint. For early armatures, the oint
>>> angle
>>> was not an issue, all they needed to care about where the bones or
>>> segments. Then advancements in creation of a realistc model, Joints
>>> were
>>> exposed. Anyway, I hope you are interested enough to comment on the
>>> data
>>> structures in the example I sent.
>>
>> See above.
>>
>>>> I am sure large parts of X3D should fit this
>>>> model quite nicely.
>>>
>>> Fine, I start with the existing X3D hierachies and names of
>>> interfaces.
>>> More later, I hope.
>>
>> You can do exactly this with Xflow. Anyone can implement Hanim or
>> any
>> other animation system. All it takes is a bit of WebComponents to
>> defined the representation and and some small Xflow processing
>> pipeline.
>>
>> You want to add something to Hanim (e.g. tangents)? Be my friend and
>> just add a single line to the processing pipeline. and it works. No
>> need
>> to wait for a revised Hanim standard.
>>
>> Xflow gives you the flexibility of being able to define your own
>> processing pipeline and allows it to be mapped to hardware and
>> execute
>> highly efficienly. It iss not limited to a single predefined way of
>> doing things.
>>
>>
>> Best,
>>
>> Philipp
>>
>>> Thanks and Best,
>>> Joe
>>>
> 
> snipped

-- 

-------------------------------------------------------------------------
Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH
Trippstadter Strasse 122, D-67663 Kaiserslautern

Geschäftsführung:
  Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
  Dr. Walter Olthoff
Vorsitzender des Aufsichtsrats:
  Prof. Dr. h.c. Hans A. Aukes

Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
VAT/USt-Id.Nr.: DE 148 646 973, Steuernummer:  19/673/0060/3
---------------------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: philipp_slusallek.vcf
Type: text/x-vcard
Size: 441 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20160703/768b3d95/attachment.vcf>