[x3d-public] gltf metadata: X3Dv4 MetadataSet design issue
Brutzman, Donald (Don) (CIV)
brutzman at nps.edu
Sat Feb 2 22:35:42 PST 2019
[summary: simpler Metadata refinement proposed for X3Dv4]
Andreas and John: your analysis sounds good. I share this same pain with you both.
I talked just a little bit with Dick about this open issue while we were in Korea recently, while viewing someone else's carefully prepared scene-metadata slide that was valid but nearly unreadable because of all the endlessly verbose containerField="value" entries.
For thoroughness here, reference can be found in X3D Core Component. Further writeups can be found in X3D Tooltips and online book chapter. Ouch ouch ouch.
X3D Example Archives: X3D for Web Authors, Chapter 15 Metadata
https://x3dgraphics.com/examples/X3dForWebAuthors/Chapter15Metadata/index.html
X3D for Web Authors, Chapter 15 Metadata Information
https://x3dgraphics.com/examples/X3dForWebAuthors/Chapter15Metadata/Chapter15MetadataInformation.html
The entire complexity (which also complicates X3DUOM) derives from the reasonable design goal of having every node in X3D include a /metadata/ field. The ambiguity arises with the MetadataSet node, which has an SFNode /metadata/ field, plus an MFNode /value/ field for other nodes. Further documented at
X3D Scene Authoring Hints: Variations from containerField defaults
http://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#containerFieldVariations
"11. MetadataSet can contain multiple other child X3DMetadataNode nodes by setting their containerField='value' (rather than default containerField='metadata'). Similar to the other metadata nodes, MetadataSet can also contain a single X3DMetadataNode node describing itself (with default containerField='metadata')."
Roy and I got pretty far on this issue during X3DUOM design but didn't have a chance to finish.
Since MetadataSet can only contain other metadata nodes, this has a similar affect on all of them. Circular, self-referential issue in our data design. Metaphorically, note illustrative Ouroboros image at
https://en.wikipedia.org/wiki/Self-reference
https://en.wikipedia.org/wiki/Self-reference#/media/File:Ouroboros.png
Our current X3D solution works unambiguously but (as you've seen) confusingly. Pretty frustrating. Over the years have been searching for a simpler alternative. Hmmmm.
I think that our best strategy forward seems best illustrated by following "doctor doctor" joke:
- Patient: "Doctor doctor, my arm hurts when i do this..." (raising arm and grimacing)
- Doctor: "Um, don't do that."
- Patient: (lowers arm)
- Nurse: (a short time later) "That will be $100 please."
So... how do we not do that? 8)
Without implementing it is hard to be 100% sure, but one example might be in our X3D interpolators. Look at X3D type requirements there for keyValue and value_changed:
============================================================
19.3.1 X3DInterpolatorNode
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/interp.html#X3DInterpolatorNode
X3DInterpolatorNode : X3DChildNode {
SFFloat [in] set_fraction (-∞,∞)
MFFloat [in,out] key [] (-∞,∞)
MF<type> [in,out] keyValue []
SFNode [in,out] metadata NULL [X3DMetadataObject]
[S|M]F<type> [out] value_changed
}
============================================================
What if we follow that example, this time for metadata within X3D nodes?
Currently the corresponding abstract node type definition is
============================================================
7.3.5 X3DNode
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/core.html#X3DNode
X3DNode {
SFNode [in,out] metadata NULL [X3DMetadataObject]
}
This abstract node type is the base type for all nodes in the X3D system.
============================================================
What if this most-central specification definition instead allowed both SFNode/MFNode and instead said
X3DNode {
[S|M]FNode [in,out] metadata NULL [X3DMetadataObject]
}
... with MetadataSet as the only X3D node using MFNode type for the field.
... then all children of MetadataSet (namely all of the Metadata* nodes) are only allowed in the /metadata/ field.
... then corresponding MetadataSet interface gets really simplified. Current version:
============================================================
7.4.5 MetadataSet
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/core.html#MetadataSet
MetadataSet : X3DNode, X3DMetadataObject {
SFNode [in,out] metadata NULL [X3DMetadataObject]
SFString [in,out] name ""
SFString [in,out] reference ""
MFNode [in,out] value [] [X3DMetadataObject]
}
The metadata provided by this node is contained in the metadata nodes of the value field.
============================================================
... might then become:
============================================================
MetadataSet : X3DNode, X3DMetadataObject {
MFNode [in,out] metadata NULL [X3DMetadataObject]
SFString [in,out] name ""
SFString [in,out] reference ""
}
The metadata provided by (and for) this node is contained in the metadata nodes of the metadata field.
============================================================
... then all Metadata* nodes have only a single containerField value, same as current default containerField="metadata".
... then all MetadataSet content is much simpler to read and much less error prone.
... worth repeating: much less error prone.
... then any metadata affecting the MetadataSet itself (rather than being a value) must be self evident (which is more logical and probably is the case anyway) rather than dependent on what field name it is assigned to.
... any existing metadata in current X3D content that is outside of MetadataSet (perhaps the majority of legacy content) is completely unaffected.
... "conversion" for legacy content simply means metadata nodes always go in the /metadata/ field.
... we add appropriate writeups noting this simplification so that legacy X3Dv3.0 thru X3Dv3.3 content is converted legitimately without flaw.
Does all this sound OK? Reactions welcome. Wear boots, stomp squarely and hard on it please, this is central and historically has been very slippery.
Hopeful analogy is that maybe this half-twist changes us to a more-elegant Moebius strip than an ourobouros.
https://en.wikipedia.org/wiki/Möbius_strip
A legitimate case can be made that we MUST fix this representation difficulty if we expect to keep growing and properly handle metadata structures for CAD, medical, ontologies, etc. etc.
I'll add it to next X3D Working Group agenda for discussion. Please send comments and initial reactions now prior to any attempted pursuit.
In case your eyes crossed at any point (mine sure did) then here is message summary again: this proposed approach seems simpler.
p.s. if it actually works and the pain stops, someone can send their $100 to Anita for Web3D Consortium membership. your future medical record thanks you... 8)
http://www.web3d.org/join
On 2/1/2019 1:02 PM, Andreas Plesch wrote:
> Yeah, I had to look up how containerfield works for metadata. It is
> one of the few cases where the default containerfield value
> ('metadata') often does not apply but still makes most sense as
> default.
>
> There is only 'metadata' or 'value' available as containerfield
> values. 'children' is not a possible value, since Metadata nodes
> cannot be children.
>
> containerfield='value' only applies to content of MetadataSet nodes.
> However, MetadataSet may be the most common Metadata node describing
> metadata for something in the scene.
>
> <Something>
> <MetadataSet name='authoringRecord' value='metadata' containerField='metadata'>
> <MetadataString name='recording' value='scanned' containerField='value'/>
> ..
> </MetadataSet>
>
> This makes the MetadataString part of the MetadataSet record.
>
> Somewhat confusingly, it is also possible to define metadata for metadata (!):
>
> <Something>
> <MetadataSet name='authoringRecord' value='metadata' containerField='metadata'>
> <MetadataString name='metaSystem' value='ABMDatabase'
> containerField='metadata'/>
> <MetadataString name='recording' value='scanned' containerField='value'/>
> ..
> </MetadataSet>
>
> Here, the 'ABMDatabase' would refer to a definition of metadata system
> where authoringRecord has a meaning. I suspect that this usage is very
> rare.
>
> There could be additional levels of metadata embeddings. [ regions of
> metaSystems or some such ].
>
> I hope I got this right as it gets pretty abstract,
>
> -Andreas
>
> On Fri, Feb 1, 2019 at 11:26 AM John Carlson <yottzumm at gmail.com> wrote:
>>
>> JSON Schema has value. OK. I must have misunderstood.
>>
>>
>>
>> John
>>
>>
>>
>> Sent from Mail for Windows 10
>>
>>
>>
>> From: John Carlson
>> Sent: Friday, February 1, 2019 10:04 AM
>> To: Andreas Plesch; X3D Graphics public mailing list; Don Brutzman
>> Subject: RE: [x3d-public] gltf metadata
>>
>>
>>
>> Re: containerField of value (oops)
>>
>>
>>
>> You might check the JSON schema, but I’m not sure “value” is the preferred name for a containerField. “children” may be preferred. That is, convert the X3D to JSON, and then check against the JSON 3.3 schema. Generally “value” => 1 value and “children” => >1 value
>>
>>
>>
>> Sent from Mail for Windows 10
>>
>>
>>
>> From: Andreas Plesch
>> Sent: Friday, February 1, 2019 9:43 AM
>> To: X3D Graphics public mailing list
>> Subject: [x3d-public] gltf metadata
>>
>>
>>
>> Here is how x3dom currently translates gltf metadata into x3d. Since
>>
>> x3dom translates the non-binary gltf data into dom and makes it
>>
>> (optionally) available as a dom node tree under the inline, translated
>>
>> metadata can then be accessed as well, to show somewhere or do
>>
>> whatever.
>>
>>
>>
>> gltf has the ability to provide unstructured or optionally structured
>>
>> metadata for the global asset and for each resource. This is similar
>>
>> to the Metadata nodes in x3d.
>>
>>
>>
>> glTF metadata can be provided in an 'extras' property which is
>>
>> available to each kind of node. In addition, the toplevel asset node
>>
>> contains metadata in defined properties.
>>
>>
>>
>> The asset node data is translated into a WorldInfo node in the following way.
>>
>>
>>
>> The copyright, generator, version, and minversion properties of the
>>
>> toplevel asset node go into WorldInfo as a MFString info field: '
>>
>> "copyright:(c)2020 sb" "version:2.0" ..' .The property name is joined
>>
>> together with its value by a colon. [ There are alternatives, such as
>>
>> property name followed by property value. ]
>>
>>
>>
>> extras of the toplevel asset node go into the WorldInfo metadata field
>>
>> packaged inside a MetadataSet holding node with name 'global':
>>
>>
>>
>> <WorldInfo info= ' "copyright:(c)2020 sb" "version:2.0" '
>>
>> title='asset.extras.title value if given'>
>>
>> <MetadataSet name='global' containerfield='metadata'>
>>
>> <MetadataSet name='asset-extras' containerfield='value'>
>>
>> extras (see below)
>>
>>
>>
>> The asset extras metadata are named 'asset-extras'.
>>
>>
>>
>> There is no defined 'title' property in gltf but if there is an
>>
>> asset.extras.title string property it goes in the title field of
>>
>> WorldInfo.
>>
>>
>>
>> [ I am starting to think that it would be better to not use the info
>>
>> field and just have an additional entry into the global MetadataSet,
>>
>> an 'asset-properties' entry ].
>>
>>
>>
>> The 'global' holding node also contains the extras of the gltf scene node.
>>
>>
>>
>> <WorldInfo>
>>
>> <MetadataSet name='global' containerfield='metadata'>
>>
>> <MetadataSet name='asset-extras' containerfield='value'>
>>
>> extras (see below)
>>
>> <MetadataSet name='scene-extras' containerfield='value'>
>>
>> extras (see below)
>>
>>
>>
>> The scene extras metadata are name 'scene-extras'.
>>
>>
>>
>> The scene metadata are put into WorldInfo metadata because X3D Scene
>>
>> itself cannot have a Metadata node (X3D Metadata nodes are not child
>>
>> nodes) and because using the META statements in the X3D header would
>>
>> then require managing the X3D header. Also, it seems logical to group
>>
>> all scene-wide data together.
>>
>>
>>
>> All other extras properties in nodes go into metadata nodes as
>>
>> required in the following way. The gltf extras json property can have
>>
>> any json value, including arrays and objects. It is recommended to use
>>
>> a json object as name:value map but any json value is allowed.
>>
>> A json string becomes a MetadataString.
>>
>> A json number becomes a MetadataFloat.
>>
>> A json boolean becomes a MetadataBoolean.
>>
>> A json array or object becomes a MetadataSet. An array is actually an
>>
>> object with numerical, ordered keys: {'0':v0, '1':v1 ..}. It can be
>>
>> treated as an object.
>>
>> The name field for each metadata node is 'extras' by default or the
>>
>> key name in case of values for MetadataSet.
>>
>> Here are examples:
>>
>> <Transform>
>>
>> <MetadataString name='extras' value='extras_value_string'
>>
>> containerfield='metadata'/>
>>
>> ----
>>
>> <Transform>
>>
>> <MetadataSet name='extras' containerfield='metadata'>
>>
>> <MetadataString name='component' value='rotor' containerfield='value'/>
>>
>> <MetadataString name='warehouse' value='StarlingCity'
>>
>> containerfield='value'/>
>>
>> ----
>>
>> <WorldInfo>
>>
>> <MetadataSet name='global' containerfield='metadata'>
>>
>> <MetadataSet name='asset-extras' containerfield='value'>
>>
>> <MetadataString name='created' value='2/3/2345' containerfield='value'/>
>>
>> <MetadataString name='author' value='art artist' containerfield='value'/>
>>
>> <MetadataString name='license' value='CC 4.0,
>>
>> http://license.org/CC' containerfield='value'/>
>>
>> -----
>>
>>
>>
>> This description is probably longer than the lines of code it takes to
>>
>> implement this kind of translation.
>>
>>
>>
>> Any input or feedback much appreciated while this effort is underway.
>>
>>
>>
>> -Andreas
>>
>>
>>
>>
>>
>> --
>>
>> Andreas Plesch
>>
>> Waltham, MA 02453
>>
>>
>>
>> _______________________________________________
>>
>> x3d-public mailing list
>>
>> x3d-public at web3d.org
>>
>> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>>
>>
>>
>>
>
>
>
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list