[x3d-public] ] V4.0 Opendiscussion/workshopon X3DHTMLintegration

[Joe D Williams]

Hi Philipp, just getting back to this, my main topic is to compare the 
data structures you are using with those in X3D HAnim. I was looking 
at the example robots and wondering exactly how you were doing it. 
Since I was thinking about basic skeletal animation, i wondered how 
you were storing skeleton structure,skin geometry, skeleton to skin 
bindings, keys and key values, texture coordinates, and metadata about 
the structure parts, etc. Then, unfortunately, to see that part of the 
user code, I could not get there.

So I wanted (1) to understand relative data structures for xflow and 
X3D HAnim for this application;
(2) with my current favorite X3D HAnim skeletal animation example 
tester, start transcoding this from X3D to xflow; (3) see this example 
running in xflow encoding.

If this is the same data in different structures with different names, 
then the transformation might be real simple style sheet transform. 
That would be a great asset.

If the data is different then maybe those animations are using a 
different method than the X3D 'standard' form of realtime skeletal 
animation. If so, then I would like to see your animation style 
running in X3D. I am pretty sure that no matter what style is being 
used there is a data type and animation method for it in X3D.

So, I hope to renew this conversation by looking to the next depth of 
your user code for this important animation method.

Please have a look at the attached. This represents plain old 
hierarchal skeleton with a structure of Joints and a simple continious 
mesh skin with a simple texture mapping. The skin is deformed to 
follow skeleton animations. Each vertex is bound to one or more 
skeleton components (Joints) along with a weight value. Each skin 
vertex is displaced according to sum of weighted rotation of the bound 
Joint(s).
[For this model, the concept and the data of bone or segment 
orientation, and of joint rotation, are considered to be equivalent 
ways to express the desired animation result. Just that X3D HAnim puts 
the binding on Joint(s).]

Anyway, I hope you and all are doing well, I'm just looking for 
comparing stuff as it relates to realtime realistic interactive 
interfaces. I think the X3D HAnim structure we have now, is a very 
competent baseline for this style of skeletal animation and general 
mesh animation, no matter what the pieces are named.

In all of this everything is important and the animations are the 
stuff that needs saving and leveraging the most, so that is what HAnim 
is aimed at: a 'standard' skeleton, a 'standard' skin, and 'standard" 
animations. The term 'standard' means different things in each 
context, A 'standard' skeleton has a known target Joint hierarchy. The 
'standard' skin has a known number of points with known skeleton 
bindings. The 'standard' animations have known interfaces with the 
model.

Thanks Again and Best Regards,
Joe


----- Original Message ----- 
From: "Joe D Williams" <joedwil at earthlink.net>
To: "'X3D Graphics public mailing list'" <x3d-public at web3d.org>; 
"Philipp Slusallek" <philipp.slusallek at dfki.de>
Sent: Monday, July 04, 2016 2:13 AM
Subject: Re: [x3d-public] ] V4.0 Opendiscussion/workshopon 
X3DHTMLintegration


> One can argue if "mesh" is really the best name for this element, 
> though. And we are considering changes right now.

>> XML3D;
>> <mesh type="triangles">
>>  <int name="index">0 1 2 2 3 0 4 ... </int>
>>  <float3 name="position">-1.0 -1.0 -1.0 ... </float3>
>>  <float3 name="normal">0.0 0.0 -1.0 0.0 0.0 ... </float3>
>>  <float2 name="texcoord">1.0 0.0 ... </float2>
>> </mesh>
>>

So, I see, the 'mesh' is an element name then you give it a real name
'triangles' in an attribute. One can also argue that @triangles is not
really the best name for your geometry. The @type attribute serves
what purpose? And you called @containerField a hack?

> But it will certainly not be "IndexedTriangleSet" :-).

Why? maybe someday someone will ask why use a different name for the
same thing? What about if you are dealing with geometry same as an
indexedfaceset? Is that also off your list? What about other stuffs,
like lights?

Oohh, I am guessing you are the first one to ever deal with with this
so you get to pick names for things. When you consider changes to
naming items, then you should bother to see what those items have been
called in earlier history. The way you are thinking about this is just
to give a general name then leave it up to the user to discover what
is actually there. For example, the term indexed in the name indicates
that an index is supplied. Another 'standard' way of dealing with this
is when the term 'indexed' does not appear, then there is no index
supplied. This is just one of many small details the a real
specification addresses.

I don't know where you could come up with the idea that names for
common features are up for grabs. I know your work is in progress but
If you are thinking that you can come up with an HAnim model that does
not use the names Joint, Segment, and SIte in a fully defined
realistic hierarchy that gets animated by joint rotations then, once
again, like history repeats(?) this will be a long grind.

> And somehow the system has to know what the data types are.

And the only way the system can know is to spell it out in the user
code? If you are considering using other XML tools, then there are
other options to putting these repetitive and boring keystrokes in the
user code. Please take another look at 'somehow' because there must be
other ways to do it.

> I think you overlooked the smiley at the end of my text :-).

Nor really, I still think you are just getting an idea of the depth of
the X3D HAnim.

> > Note that because of the built-in reuse of data in the flow-graphs 
> > of Xflow we can also reuse these buffers with 
> > (partial/intermediate) results somewhere within a flow-graph.

I still do not understand why you mix the runtime requirements and
advantages of your Xflow flow-graphs with the authortime syntax. All
players must take the data  they are given and arrange the data for
their optimum runtime.

Sorry, my error in my example. the @coord attr should be @point
instead.

 >> X3D:
>> <indexedtriangleset
>>  index="0 1 2 2 3 0 4 ..."
>>  coord="0 1 2 2 3 0 4 ..."
>>  normal="0.0 0.0 -1.0 0.0 0.0 ..."
>>  texcoord="1.0 0.0 ..."
>> </indexedtriangleset>
>>

I see your XML3D for a 'mesh' subtitled 'triangles' with the same data
as for an X3D Indexedtriangleset but not only do you insist that the
keystrokes for data type elements <float2 ... > are necessary and a
reasonable element name but you use another different name (xflow
location = x3d point) for the data. So why does this Xflow require
different names for the same thing. To me, when you repeat the
suggestion to read your paper, I must ask you to read some history and
see that people have been here before you and me. So, there is some
history before X3D and before Xflow so the reality is that xflow does
not need to rename existing concepts. But, I guess location rather
than point was picked a long, long, long time ago?

Anyway,for this 'mesh' called 'triangles' you managed to use tha
'standard' name for the same things 3 out of 4 times. normal is
normal, texcoord is texcoord, index is index but oops, location should
be point, Why not take that extra step and make it 100percent?
(serioussmiley)

Note that the process of generating a standard means picking the best
from multiple running and proven implementations. Since there are
multiple implementations and only one standard then one of the steps
is to pick names for things. So, when you are suggesting a
standardized interface for defining 'mesh' geometry, why not use the
historical terms rather than substitutes you pick out of the air. This
helps everyone later on when study shows that the term location is
used for another purpose in X3D and is always SF. Of course big
problems come when the implementation creator says I don't care i'm
not going to change any of my code just because X3D says so. So, just
as you say read my paper to understand, I must say read the existing
specs to understand. I don't know, maybe somewhere you may find some
tool that uses the term location instead of point for vertex
coordinates?

> BTW, it is the same mechanism that the GPU is using to pass data 
> efficiently to HW (buffers!).

In this process of creating a 'standard' nobody cares about "mechanism
that the GPU is using to pass data ". How you do your runtime is up to
you. All a conforming system has to do is accept X3D user code then
function like a 'standard' X3D scene should function in the runtime.
If your data is organized to take advantage of some hardware or
software or whatever better than others then your application may
produce better results than another implementation and you will win.
However, to say I need a head start by having the users put the data
in the form I demand to optimize my processing then..., well, maybe
you have a better name for that sort of coercion than I do.

But clearly, the idea that your runtime data is better optimized for
some configuration has Zero to do with the user code. You are in
effect saying that the user code must be organized to suit your data
processing style. Again, since the X3D is implementation independent,
if somebody says "Hey, Xflow is so great you all gotta change all your
user code because of Xflow" I don't think that will work. Do you?

Thanks and Best,
Joe

http://www.hypermultimedia.com/x3d/Quick/NodesIDLIndex.htm




----- Original Message ----- 
From: "Philipp Slusallek" <philipp.slusallek at dfki.de>
To: "Joe D Williams" <joedwil at earthlink.net>; "'X3D Graphics public
mailing list'" <x3d-public at web3d.org>
Sent: Sunday, July 03, 2016 4:55 AM
Subject: Re: [x3d-public] ] V4.0 Opendiscussion/workshopon
X3DHTMLintegration


> Hi Joe,
>
> Am 01.07.2016 um 22:44 schrieb Joe D Williams:
>>>> Doing a full WebComponent implementation for Hanim is left as an
>> exercise for the reader :-).
>>
>> I like the grin at the end of that:). I think you are just getting 
>> an
>> idea of the depth of the X3D HAnim. I think HAnim is so fundamental
>> that you _should_ help a lot. After all, you are telling me that 
>> there
>> is no problem in hierarchy definitions, binding for deformable skin
>> and other geometry, and synchronization, with some script to prove 
>> it,
>> but I have not yet studied it or seen details of how I implement my
>> favorite x3d hanim:)
>
> I think you overlooked the smiley at the end of my text :-).
>
>> For now, please excuse the following. This is soooo basic to me 
>> that I
>> must get it out.
>>
>> http://xml3d.org/xml3d/specification/latest/#dataflow-graph-xflow
>>
>> XML3D;
>> <mesh type="triangles">
>>  <int name="index">0 1 2 2 3 0 4 ... </int>
>>  <float3 name="position">-1.0 -1.0 -1.0 ... </float3>
>>  <float3 name="normal">0.0 0.0 -1.0 0.0 0.0 ... </float3>
>>  <float2 name="texcoord">1.0 0.0 ... </float2>
>> </mesh>
>>
>> Wow, I can't even bring myself to think of that XML3D code as any 
>> form
>> of (x)html. I mean sure I can figure it out, but why should I have 
>> to?
>>
>> This example could look like this is in X3D style:
>>
>> X3D:
>> <indexedtriangleset
>>  index="0 1 2 2 3 0 4 ..."
>>  coord="0 1 2 2 3 0 4 ..."
>>  normal="0.0 0.0 -1.0 0.0 0.0 ..."
>>  texcoord="1.0 0.0 ..."
>> </indexedtriangleset>
>>
>> To me, the XML3D form is just way too confusing. When I am working
>> with this stuff,
>> I expect to have at my disposal a sufficiently advanced interface 
>> to
>> the user code where I just don't need to know or mostly even care
>> about the frigging data typing.
>
> I still suggest you read the papers that I sent links to earlier, or 
> our
> spec. Then you would better understand, why we use this syntax and 
> why
> its is so much more useful. Anyway here is a very short summary.
>
> X3D allows only a fixed set of predefined attributes for each 
> element.
> We have a generic mechanism here that is needed as we also allow the
> same mechanism to pass arbitrary attribute values to shaders for 
> each
> and every primitive. And somehow the system has to know what the 
> data
> types are.
>
> We use the same mechanism to also pass data to scripts, light source
> shaders, view configurations, WebComponents, and so on. This is why 
> we
> call it "generic"!
>
> BTW, it is the same mechanism that the GPU is using to pass data
> efficiently to HW (buffers!). So in many case, can we keep this data 
> in
> buffers directly on the GPU, apply the shaders and other compute
> operations there (even an entire sequence of them), and then use the
> results directly for rendering without going back to the CPU.
>
> Note that because of the built-in reuse of data in the flow-graphs 
> of
> Xflow we can also reuse these buffers with (partial/intermediate)
> results somewhere within a flow-graph.
>
>> First, I recommend the engaging study of the X3D @containerfield as 
>> an
>> example of use of an attribute to declare an important parameter of
>> the element. This study of containerfield reveals it is like the
>> <float2> in that it is about the data in the container, not about 
>> the
>> container so this vital information can surely be set as an 
>> attribute
>> of the element rather than as its @name or <container>...
>> </container>. I mean we just don't need container fields in X3D 
>> XML,
>> even though we sometimes need to identify an acceptable container.
>
> Before recommending some "hack" like "@containerfield", please 
> again,
> first try to understand what you are targeting. The xflow sytax 
> requires
> a bit more text but is much more generic and as a result offers
> dramatically more functionality that way. Also see below.
>
>> Then you can call that element by a real functionality, such as:
>>
>> <mesh type="triangles">
>>  <index vartype="int"> 0 1 2 2 3 0 4 ... </index>
>>  <position vartype="float3"> -1.0 -1.0 -1.0 ... </position>
>>  <normal vartype="float3"> 0.0 0.0 -1.0 0.0 0.0 ... </normal>
>>  <texcoord vartype="float2"> 1.0 0.0 ... </texcoord>
>> </mesh>
>>
>> Or even extend your vocabulary to call the container what it is
>> actually containing such as:
>>
>> <triangles>
>>  <index vartype="int"> 0 1 2 2 3 0 4 ... </index>
>>  <position vartype="float3"> -1.0 -1.0 -1.0 ... </position>
>>  <normal vartype="float3"> 0.0 0.0 -1.0 0.0 0.0 ... </normal>
>>  <texcoord vartype="float2"> 1.0 0.0 ... </texcoord>
>> </triangles>
>
> This assumes that we know beforehand the set of all possible 
> attributes
> to geometry (and shaders attached to them). Due to programmable 
> shading
> (see our work on shade.js as part of XML3D) this set is not limited 
> at
> all. So this is suggestion simply not an option!
>
>> Advantage is then you can use a system of defaults that keeps your
>> user code from looking lke a bunch strung-together features hung
>> together by data type elements that are in no way optimized for 
>> human
>> readablility and editing but instead only for the convenience of 
>> your
>> processing scripts.
>
> Yes, we do not optimize for human readability at this low level. 
> Data
> buffers (long arrays of numbers) are not very human readable in the
> first place -- beyond trivial models!
>
> Where we do care about readability very much is at the higher layers
> where you take these buffers and combine them in flexible way, pass 
> them
> to shaders, refernce them from other data elements for reuse, 
> specify
> the attributes to Web components. At that level its a simple URL or
> fragment name refering to single buffer or a collection thereof. Due 
> to
> the ability to flexibly combine data elements with each other, this 
> is
> extremely elegant and compact, I believe.
>
>> That leaves us with the topic of should the data be in attributes 
>> or
>> elements which needs a special topic.
>
> Due to the unlimited set of possible names for shader attributes 
> putting
> them in attributes is simply not an option!
>
> There are other arguments as well, for example by looking at the
> correspondence between text and 3D data -- but let's not go into 
> this.
>
>> And really, since you are defining an indexed triangle geometry, 
>> then
>> why not use the current official ISO name for that thing?
>> There is No doubt that this world-standard functionality for 
>> defining
>> this type of geometry is officially named IndexedTriangleSet which
>> clearly distinguishes it from the default-indexed device that is
>> officially named TriangleSet.
>
> The mesh element is used as a single entry point to interpret 
> generic
> data elements (from Xflow) as geometry to be rendered. There is not 
> a
> single interpretation such but we are supporting a number of them 
> (not
> all in the currently released version yet): points, lines, 
> triangles,
> trianglesstripts, volume data, and many more. Please note that --  
> for
> obvious reasons -- this is also very similar to how data is passed 
> to
> the GPU via OpenGL Draw calls. See comments above.
>
> One can argue if "mesh" is really the best name for this element,
> though. And we are considering changes right now. But it will 
> certainly
> not be "IndexedTriangleSet" :-).
>
>> More later,
>> Thanks Again for your interest in discussing some more about this. 
>> I
>> hope we can get past this small rant about basic XML3D syntax and 
>> dig
>> deeper into hanim stuffs.
>
> I am happy to keep answering your questions. But may I again ask you 
> to
> first read our papers as this will answer many of your questions 
> already
> and save all of us time. It will also allow us to go into more of 
> the
> core ideas that at least I consider more interesting.
>
>
> Best,
>
> Philipp
>
>
>> All Best,
>> Joe
>>
>> ----- Original Message ----- From: "Philipp Slusallek"
>> <philipp.slusallek at dfki.de>
>> To: "Joe D Williams" <joedwil at earthlink.net>; "'X3D Graphics public
>> mailing list'" <x3d-public at web3d.org>
>> Sent: Thursday, June 16, 2016 9:54 PM
>> Subject: Re: [x3d-public] ] V4.0 Opendiscussion/workshopon
>> X3DHTMLintegration
>>
>>
>>> Hi Joe, all,
>>>
>>> FYI: It seemes that my email to the x3d-public list were dropped 
>>> by
>>> the
>>> server. Because I did not get a error message, I only found out 
>>> now
>>> (thanks for your help Don, Leonard!). It seems that Leonard will 
>>> be
>>> able
>>> to resend them to the list.
>>>
>>> I think this was/is a good discussion that is worth having on the
>>> list
>>> archives.
>>>
>>> Am 16.06.2016 um 22:20 schrieb Joe D Williams:
>>>>> but to have a direct
>>>>> mapping from the data of the game
>>>>
>>>> Yes, well then that is not the scope of X3D, at least in this 
>>>> case.
>>>> If
>>>> you see the kick example, then the 'standard' X3D hanim model
>>>> actually
>>>> exposes more of what you need in better interactive model than 
>>>> this
>>>> example.
>>>
>>> Maybe it is now. But I would argue that it should be possible to
>>> more
>>> easily work with data in various formats and not be forced into 
>>> the
>>> one
>>> and only format.
>>>
>>> Not only may you have todo extra work, there might even be 
>>> features
>>> that
>>> are not covered (skinning for tangent vectors are such) and extra
>>> work
>>> done that is not even needed in some situations.
>>>
>>> I think this aspect of the design is worth reconsidering for a new
>>> version. Especially, since an spec/implementation is available 
>>> that
>>> would support that and could be integrated into the new version.
>>>
>>>> I think what we are seeing in the .heavy animation (discalimer:
>>>> haven't
>>>> seen the code) is not the actions of a connected hierarcal
>>>> skeleton, but
>>>> more like just moving the segments (bones) around without exposed
>>>> concepts such as a joint center. That is ok, the skeleton may
>>>> expose
>>>> itself in the anim code. As near as I can tell from the data
>>>> without
>>>
>>> It is a full hierarchical skeleton. We do propagate the
>>> transformation
>>> of the upper joints downwards as necessary.
>>>
>>> All the code is in the xml3d.js repository on github, if you want 
>>> to
>>> look at it. The link in the file goes to the minified version 
>>> where
>>> its
>>> hardly readable.
>>>
>>>> seeing the animation user code then this is basically like early
>>>> simple
>>>> motion capture data dealing with motion captured then edited but
>>>> basically built to be played back like a video at some more or 
>>>> less
>>>> 'fixed frame rates, but not much processing.
>>>
>>> It is definitely more than that! In a project with Daimler we are
>>> using
>>> it for example for doing fully automatic motion synthesis for
>>> virtual
>>> workes in a factory model to execute some high-level tasks given
>>> some
>>> written instructions ("pick up part X, use the skrew driver to fix
>>> it to
>>> the car Y and location Z"). All this is based on semanic 
>>> annotations
>>> of
>>> the XML3D model and the shown character animation engine.
>>>
>>>>> simply create their
>>>>> own high-level elements in a modular way using WebComponents and
>>>>> publish
>>>>> that for others to use.
>>>>
>>>> more later I hope, but that is the purpose of X3D. To establish
>>>> these
>>>> practical, bound, data sets that can be animated in a realistic
>>>> way.
>>>> There are several players that use this model and is the basic
>>>> world
>>>> standard' simulation humanoidj for detailed simulation and
>>>> interaction.
>>>>
>>>> So, for high level elements, so far I don't see any baetterthan 
>>>> X3D
>>>> Hanim. I see that way that the runtime interpolator data is 
>>>> stored
>>>> in
>>>> you code and I think I can compare is with the way it is 
>>>> presented
>>>> in X3D.
>>>
>>> I still think you are misunderstanding me: The goal of our work os
>>> NOT
>>> to develop a "better Hanim". It is to have a declarative interface
>>> to
>>> specify all sorts of geometry and other processingin an easy and
>>> high-level way.
>>>
>>> This can then be used by an web developer to define their own 
>>> object
>>> representations and how they get processed for display, 
>>> interaction,
>>> etc. You are not limited to the way someone defined at some point
>>> (as
>>> good as that may be, it will never be perfectly suited for 
>>> anyone).
>>>
>>> This is something that X3D does not really offer today, and which 
>>> we
>>> think would be useful to add. Just a suggestion.
>>>
>>> What I am saying in this thread is that our concept is powerful
>>> enogh to
>>> also implement Hanim.
>>>
>>>> Anyway, I think I will be away for a week or so but since in 
>>>> hanim,
>>>> for
>>>> people playing in free and open locations, the high-level 
>>>> elements
>>>> of
>>>> X3D HAnim must be considered as a tried an true way since the
>>>> essentials
>>>> of the humanoid, what you need to build and take control are all
>>>> mostly
>>>> built-in, For example,X3D used to call the thing a Transform but
>>>> added
>>>> some features and called it a joint. For early armatures, the 
>>>> oint
>>>> angle
>>>> was not an issue, all they needed to care about where the bones 
>>>> or
>>>> segments. Then advancements in creation of a realistc model, 
>>>> Joints
>>>> were
>>>> exposed. Anyway, I hope you are interested enough to comment on 
>>>> the
>>>> data
>>>> structures in the example I sent.
>>>
>>> See above.
>>>
>>>>> I am sure large parts of X3D should fit this
>>>>> model quite nicely.
>>>>
>>>> Fine, I start with the existing X3D hierachies and names of
>>>> interfaces.
>>>> More later, I hope.
>>>
>>> You can do exactly this with Xflow. Anyone can implement Hanim or
>>> any
>>> other animation system. All it takes is a bit of WebComponents to
>>> defined the representation and and some small Xflow processing
>>> pipeline.
>>>
>>> You want to add something to Hanim (e.g. tangents)? Be my friend 
>>> and
>>> just add a single line to the processing pipeline. and it works. 
>>> No
>>> need
>>> to wait for a revised Hanim standard.
>>>
>>> Xflow gives you the flexibility of being able to define your own
>>> processing pipeline and allows it to be mapped to hardware and
>>> execute
>>> highly efficienly. It iss not limited to a single predefined way 
>>> of
>>> doing things.
>>>
>>>
>>> Best,
>>>
>>> Philipp
>>>
>>>> Thanks and Best,
>>>> Joe
>>>>
>>
>> snipped
>
> -- 
>
> -------------------------------------------------------------------------
> Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) GmbH
> Trippstadter Strasse 122, D-67663 Kaiserslautern
>
> Geschäftsführung:
>  Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
>  Dr. Walter Olthoff
> Vorsitzender des Aufsichtsrats:
>  Prof. Dr. h.c. Hans A. Aukes
>
> Sitz der Gesellschaft: Kaiserslautern (HRB 2313)
> VAT/USt-Id.Nr.: DE 148 646 973, Steuernummer:  19/673/0060/3
> ---------------------------------------------------------------------------
>


_______________________________________________
x3d-public mailing list
x3d-public at web3d.org
http://web3d.org/mailman/listinfo/x3d-public_web3d.org 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Kick1Ex.zip
Type: application/x-zip-compressed
Size: 16959 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20161102/1463e531/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: hanimkickb.jpg
Type: image/jpeg
Size: 15623 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20161102/1463e531/attachment-0001.jpg>