[x3d-public] Topics for Discussion at JSON meeting: SFNode/MFNode field/fieldValue content, ProtoBody, regexes
Roy Walmsley
roy.walmsley at ntlworld.com
Sat Feb 6 02:39:19 PST 2016
Don,
Thanks for your in-depth considerations.
The only one I would like to comment further on is that regarding nodes in ProtoBody. This is my topic 3).
I can see your argument, and, in some ways, agree. However, how would that be validated? I have a ProtoBody element, to which I need to assign properties. What properties should I assign. Using your method I would have to assign every possible containerField name, and assign the nodes that can go there. Another disadvantage is that comments would be confined to the -children property, losing some of the ability to be located with most of the nodes they refer to.
Using a single -children field means it is the only property I have to assign. Yes, I have to add every concrete node as a possible child. It also has the advantage that comments can be interspersed.
At the moment there is no easy way to validate that a ProtoInstance is being used in the correct place. Validation would need to be able to find the first child of the relevant ProtoBody and check that it could be used in the field that calls up the ProtoInstance.
Let's take a small example excerpt. It's not meant to be meaningful, only illustrative of principles.
<ProtoDeclare>
<ProtoInterface/>
<ProtoBody>
<MetadataBoolean/>
</ProtoBody>
</ProtoDeclare>
....
<MetadataSet>
<ProtoInstance name="myProto" containerField="value"/>
</MetadataSet>
....
<MetadataSet>
<ProtoInstance name="myProto" containerField="metadata"/>
</MetadataSet>
How would the JSON encoding handle the ProtoBody? Will you assign it to a -metadata or a -value field?
The JSON encoding of the MetadataSet and the ProtoInstance would correctly assign the ProtoInstance to the appropriate field in each case. So there is no loss of information if the ProtoBody assigns the MetadataBoolean to a generic -children field.
Now, consider a variation.
<ProtoDeclare>
<ProtoInterface/>
<ProtoBody>
<MetadataBoolean/>
<MetadataBoolean/>
</ProtoBody>
</ProtoDeclare>
Now, how would you encode the ProtoBody? According to the strict wording of the specification is It illegal to use this prototype in a -metadata field? The first node is still legal! So the wording of the specification is probably insufficient.
Has that muddied the waters?
Roy
-----Original Message-----
From: Don Brutzman [mailto:brutzman at nps.edu]
Sent: 06 February 2016 03:04
To: Roy Walmsley; John Carlson
Cc: x3d-public at web3d.org
Subject: Re: [x3d-public] Topics for Discussion at JSON meeting: SFNode/MFNode field/fieldValue content, ProtoBody, regexes
Summary: Once again the X3D JSON design is looking pretty solid, and our tool suites are pretty powerful at detecting edge-case issues.
On 2/5/2016 10:37 AM, Don Brutzman wrote:
> [correction: shifted to x3d-public]
> [...]
> On 2/4/2016 10:56 AM, Roy Walmsley wrote:
>> Topics for Discussion for JSON encoding meeting February 5^th 2016 at 1000 PST, 1800 GMT.
>>
>> 1) *Review of progress on JSON Schema development and validation
>> testing on example archive.*
As the screen shot only began to indicate, this is really rocking! as discussed it will be great to
- autogenerate the X3D JSON Schema from the X3D Object Model
- include it as informative annex in future spec
- Document the various validation features possible/supported
- Generate X3D JSON Schema documentation using XML Spy (feature request submitted to Altova) or another tool
It was also totally impressive to see that John is applying X3D JSON with other JavaScript libraries to good effect.
Tweet:
"d3 layout orbit x3d" by John Carlson visualizes multiple example scenes using new #X3D JSON encoding
https://twitter.com/Web3DConsortium/status/695687691308896256
https://www.youtube.com/watch?v=p0fMd92mu5s
Data-Driven Documents @d3js_org https://d3js.org
Great visualization!
Presumably the availability of an X3D JSON Schema will not only improve scene quality but further simplify the reliable integration of X3D into a variety of JavaScript frameworks.
uh... "X3D Anywhere" anybody?!
Further deep-dive issues follow.
>> 2) *Encoding of node fields in ‘field’ and ‘fieldValue’ elements
>> within Script and Prototypes.*
>> [...snip/shuffle...]
>> Illustration of 2) above is in X3D Example Archives: Basic, CAD, Cad
>> Geometry Extern Prototypes (see
>> http://www.web3d.org/x3d/content/examples/Basic/CAD/_pages/page02.htm
>> l)
>>
>> Find the ProtoInstance named ‘IndexedQuadSet’. It has two fieldValue children. The second has the name ‘coord’, and a child Coordinate node. When converted into JSON this Coordinate node is allocated to a field /-coord/, which is the default containerField name for this node. This is incorrect. It should be allocated to the /-value/ field. This is because the element fieldValue has attributes of ‘name’ and ‘value’.
>>
>> A similar issue arises with default values for node fields within the field element.
OK this change is worth close scrutiny to avoid confusion later.
For <field> and <fieldValue>, when the "@value" attribute is used for simple types, everything works OK.
For <field> and <fieldValue>, with contained SFNode/MFNode content or contained comments or contained ROUTEs, the key is now "-value" (hyphen instead of ampersand).
I have applied this correction to field/fieldValue SFNode/MFNode handling in X3dToJson.xslt conversion stylesheet and will deploy updates to the example archives.
Example excerpts follow, check out the differences for ("index","@value") and ("coord","-value") below:
http://www.web3d.org/x3d/content/examples/Basic/CAD/CadGeometryExternPrototypes.x3d
http://www.web3d.org/x3d/content/examples/Basic/CAD/CadGeometryExternPrototypes.json (copy attached) =========================================
<ProtoInstance containerField='geometry' name='IndexedQuadSet'>
<fieldValue name='index' value='0 3 2 1 4 5 6 7 2 7 9 8 2 3 4 7'/>
<fieldValue name='coord'>
<Coordinate point='-1.5 0 0 -1.5 1 -1 -.5 1 -1 -.5 0 0 0.5 0 0 1.5 0 0 1.5 1 -1 0.5 1 -1 -0.5 2 -1 0.5 2 -1'/>
</fieldValue>
</ProtoInstance>
=========================================
{ "ProtoInstance":
{
"@name":"IndexedQuadSet",
"fieldValue": [
{
"@name":"index",
"@value":[0,3,2,1,4,5,6,7,2,7,9,8,2,3,4,7]
},
{
"@name":"coord",
"-value":[
{ "Coordinate":
{
"@point":[-1.5,0,0,-1.5,1,-1,-0.5,1,-1,-0.5,0,0,0.5,0,0,1.5,0,0,1.5,1,-1,0.5,1,-1,-0.5,2,-1,0.5,2,-1]
}
}
]
}
]
}
}
=========================================
Next issue:
>> 3) *Encoding of node children in ProtoBody*
>> [...snip/shuffle...]
>> Taking 3) above: ProtoBody can take any nodes as children. The current converter assigns them to a field named as per the default containerField. I propose that all nodes, irrespective of type, be added to a /-children/ field.
Let's think this one through a bit more, please.
The first node in a ProtoBody has special significance, it indicates the node type of the prototype being declared. This language feature lets X3D authors define a prototype alternative for any node in X3D - a huge extensibility feature. From the spec:
7.2.5.8 PROTO statement
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/core.html#PROTOStatement
(last paragraph)
"The <protoDefinition> consists of a list of nodes the first of which is used to specify the node type for the prototype. This list may instantiate other prototypes provided that the definitions of the referenced prototypes precede this PROTO statement. See 4.4.4.3 PROTO definition semantics for details."
Of note is that this is the only place in the scene graph where nodes with dissimilar node types can be side by side as peers.
In the XML encoding, the node type for the first ProtoBody child is defined by it's default (or overridden) containerField value.
In the current JSON encoding, the first node is similarly exposed, and the type is the key. For example ProtoBody "-geometry" in JSON encoding example below. Subsequent nodes/ROUTEs/comments all fall under the "-children" key.
I've provided abridged (but otherwise unmodified) source excerpts for the examples we discussed during today's teleconference.
[Emotional warning: this pair of examples are wonderfully or horribly detailed, depending on your linguistic rheostat. Our 3D imaginations are even bigger than the current Web, so here we go...]
Excerpts follow.
http://www.web3d.org/x3d/content/examples/Basic/CAD/CadGeometryPrototypes.x3d
http://www.web3d.org/x3d/content/examples/Basic/CAD/CadGeometryPrototypes.json (copy attached) =========================================
<ProtoDeclare appinfo='The IndexedQuadSet node represents a 3D shape composed of a collection of individual quadrilaterals (quads). IndexedQuadSet uses the indices in its index field to specify the vertices of each quad from the coord field. Each quad is formed from a set of four vertices of the Coordinate node identified by four consecutive indices from the index field If the index field does not contain a multiple of four coordinate values the remaining vertices shall be ignored.' documentation='http://www.web3d.org/x3d/specifications/ISO-IEC-19775-Amendment1-PDAM-X3DAbstractSpecification/Part01/components/CADGeometry.html#IndexedQuadSet' name='IndexedQuadSet'>
<ProtoInterface>
<field accessType='inputOnly' appinfo='range [0..∞) or -1' name='set_index' type='MFInt32'>
<!-- No specific initialization value -->
</field>
<field accessType='inputOutput' appinfo='[X3DVertexAttributeNode]' name='attrib' type='MFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='inputOutput' appinfo='[X3DColorNode]' name='color' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='inputOutput' appinfo='[X3DCoordinateNode]' name='coord' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='inputOutput' appinfo='[FogCoordinate]' name='fogCoord' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='inputOutput' appinfo='[X3DNormalNode]' name='normal' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='inputOutput' appinfo='[X3DTextureCoordinateNode]' name='texCoord' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
<field accessType='initializeOnly' name='ccw' type='SFBool' value='true'/>
<field accessType='initializeOnly' appinfo='colorPerVertex ignored in IndexedQuadSet, and always treated as true' name='colorPerVertex' type='SFBool' value='true'/>
<field accessType='initializeOnly' name='normalPerVertex' type='SFBool' value='true'/>
<field accessType='initializeOnly' name='solid' type='SFBool' value='true'/>
<field accessType='initializeOnly' appinfo='range [0..∞) or -1' name='index' type='MFInt32'>
<!-- No specific initialization value -->
</field>
<field accessType='inputOutput' appinfo='[X3DMetadataObject]' name='metadata' type='SFNode'>
<!-- Specification initialization: NULL node -->
</field>
</ProtoInterface>
<ProtoBody>
<IndexedFaceSet DEF='RenderedIQS'>
<IS>
<connect nodeField='attrib' protoField='attrib'/>
<connect nodeField='color' protoField='color'/>
<connect nodeField='colorPerVertex' protoField='colorPerVertex'/>
<connect nodeField='coord' protoField='coord'/>
<connect nodeField='fogCoord' protoField='fogCoord'/>
<connect nodeField='normal' protoField='normal'/>
<connect nodeField='texCoord' protoField='texCoord'/>
<connect nodeField='ccw' protoField='ccw'/>
<connect nodeField='normalPerVertex' protoField='normalPerVertex'/>
<connect nodeField='solid' protoField='solid'/>
</IS>
</IndexedFaceSet>
<!-- Initial node in the PROTO body is actual node type, and the only node rendered. Remaining ProtoBody nodes not rendered -->
<Group DEF='UnrenderedIQS'>
<Script DEF='IndexedQuadSetToIndexedFaceSet' directOutput='true'>
<field accessType='initializeOnly' name='index' type='MFInt32'/>
<field accessType='inputOnly' name='set_index' type='MFInt32'/>
<field accessType='initializeOnly' name='renderedIQS' type='SFNode'>
<IndexedFaceSet USE='RenderedIQS'/>
</field>
<field accessType='initializeOnly' name='localTraceEnabled' type='SFBool' value='true'/>
<field accessType='initializeOnly' name='coordIndexNew' type='MFInt32'>
<!-- constructed during initialization -->
</field>
<IS>
<connect nodeField='index' protoField='index'/>
<connect nodeField='set_index' protoField='set_index'/>
</IS>
<![CDATA[
ecmascript:
// [...abridged...]
]]>
</Script>
<Group>
<MetadataString name='metadataHolder'>
<IS>
<connect nodeField='metadata' protoField='metadata'/>
</IS>
</MetadataString>
</Group>
</Group>
</ProtoBody>
</ProtoDeclare>
=========================================
{ "ProtoDeclare":
{
"@name":"IndexedQuadSet",
"@appinfo":"The IndexedQuadSet node represents a 3D shape composed of a collection of individual quadrilaterals (quads). IndexedQuadSet uses the indices in its index field to specify the vertices of each quad from the coord field. Each quad is formed from a set of four vertices of the Coordinate node identified by four consecutive indices from the index field If the index field does not contain a multiple of four coordinate values the remaining vertices shall be ignored.",
"@documentation":"http://www.web3d.org/x3d/specifications/ISO-IEC-19775-Amendment1-PDAM-X3DAbstractSpecification/Part01/components/CADGeometry.html#IndexedQuadSet",
"ProtoInterface": {
"field": [
{
"@name":"set_index",
"@accessType":"inputOnly",
"@appinfo":"range [0..∞) or -1",
"@type":"MFInt32",
"-children":[
{ "#comment":"No specific initialization value"
}
]
},
{
"@name":"attrib",
"@accessType":"inputOutput",
"@appinfo":"[X3DVertexAttributeNode]",
"@type":"MFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"color",
"@accessType":"inputOutput",
"@appinfo":"[X3DColorNode]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"coord",
"@accessType":"inputOutput",
"@appinfo":"[X3DCoordinateNode]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"fogCoord",
"@accessType":"inputOutput",
"@appinfo":"[FogCoordinate]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"normal",
"@accessType":"inputOutput",
"@appinfo":"[X3DNormalNode]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"texCoord",
"@accessType":"inputOutput",
"@appinfo":"[X3DTextureCoordinateNode]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
},
{
"@name":"ccw",
"@accessType":"initializeOnly",
"@type":"SFBool",
"@value":true
},
{
"@name":"colorPerVertex",
"@accessType":"initializeOnly",
"@appinfo":"colorPerVertex ignored in IndexedQuadSet, and always treated as true",
"@type":"SFBool",
"@value":true
},
{
"@name":"normalPerVertex",
"@accessType":"initializeOnly",
"@type":"SFBool",
"@value":true
},
{
"@name":"solid",
"@accessType":"initializeOnly",
"@type":"SFBool",
"@value":true
},
{
"@name":"index",
"@accessType":"initializeOnly",
"@appinfo":"range [0..∞) or -1",
"@type":"MFInt32",
"-children":[
{ "#comment":"No specific initialization value"
}
]
},
{
"@name":"metadata",
"@accessType":"inputOutput",
"@appinfo":"[X3DMetadataObject]",
"@type":"SFNode",
"-children":[
{ "#comment":"Specification initialization: NULL node"
}
]
}
]
},
"ProtoBody": {
"-geometry":[
{ "IndexedFaceSet":
{
"@DEF":"RenderedIQS",
"IS": {
"connect": [
{
"@nodeField":"attrib",
"@protoField":"attrib"
},
{
"@nodeField":"color",
"@protoField":"color"
},
{
"@nodeField":"colorPerVertex",
"@protoField":"colorPerVertex"
},
{
"@nodeField":"coord",
"@protoField":"coord"
},
{
"@nodeField":"fogCoord",
"@protoField":"fogCoord"
},
{
"@nodeField":"normal",
"@protoField":"normal"
},
{
"@nodeField":"texCoord",
"@protoField":"texCoord"
},
{
"@nodeField":"ccw",
"@protoField":"ccw"
},
{
"@nodeField":"normalPerVertex",
"@protoField":"normalPerVertex"
},
{
"@nodeField":"solid",
"@protoField":"solid"
}
]
}
}
}
],
"-children":[
{ "#comment":"Initial node in the PROTO body is actual node type, and the only node rendered. Remaining ProtoBody nodes not rendered"
},
{ "Group":
{
"@DEF":"UnrenderedIQS",
"-children":[
{ "Script":
{
"@DEF":"IndexedQuadSetToIndexedFaceSet",
"@directOutput":true,
"field": [
{
"@name":"index",
"@accessType":"initializeOnly",
"@type":"MFInt32"
},
{
"@name":"set_index",
"@accessType":"inputOnly",
"@type":"MFInt32"
},
{
"@name":"renderedIQS",
"@accessType":"initializeOnly",
"@type":"SFNode",
"-value":[
{ "IndexedFaceSet":
{
"@USE":"RenderedIQS"
}
}
]
},
{
"@name":"localTraceEnabled",
"@accessType":"initializeOnly",
"@type":"SFBool",
"@value":true
},
{
"@name":"coordIndexNew",
"@accessType":"initializeOnly",
"@type":"MFInt32",
"-children":[
{ "#comment":"constructed during initialization"
}
]
}
],
"IS": {
"connect": [
{
"@nodeField":"index",
"@protoField":"index"
},
{
"@nodeField":"set_index",
"@protoField":"set_index"
}
]
},
"#sourceText":[ "ecmascript:","[...abridged...]" ]
}
},
{ "Group":
{
"-metadata":[
{ "MetadataString":
{
"@name":"metadataHolder",
"IS": {
"connect": [
{
"@nodeField":"metadata",
"@protoField":"metadata"
}
]
}
}
}
]
}
}
]
}
}
]
}
}
},
=========================================
Assessment: syntax for these ProtoDeclare encodings are already closely aligned with each other, and express the semantics of the specification accurately.
Recommendation: no change, enjoy! 8)
YMMV, further insights and improvements welcome. Please advise.
>> 4) *Encoding of single value for an MFxxxxx field (non-node) - array
>> or single value*
As discussed, in previous emails and on today's call:
- strict typing of arrays has expressive power and matches the object-oriented nature of JSON.
- creation of an X3D JSON Schema has the expressive power to isolate problems and enforce correct typing.
- The appropriate rules exist already within X3dToJson.xslt conversion stylesheet.
So at this point, I simply have to go through the invalid JSON scenes identified by John's JSON validation suite and fix either offending .x3d content or incorrect translation details inside the stylesheet.
>> 5) *Validation of /url/ fields.*
We discussed how JSON/javascript diagnostics (probably regex i.e. regular expression based) are hiccuping on the multiple url entries in X3D url MFString arrays, especially when # character is included in each ProtoDeclare url.
These are legal constructs for X3D.
Future work: when we finish current X3D XML Schema, X3D Object Model and X3D JSON work, we will return to the X3D Regular Expressions effort and document correctly working regexes for X3D urls.
Skeleton page is at
http://www.web3d.org/specifications/X3dRegularExpressions.html
Meanwhile: continuing bugfix attention to any invalid url content detected in the X3D Examples Archives.
>> 6)The numeric value -0.
Further investigation in next message. This topic might seem pedantic (ok ok it is very pedantic) but it is important to get right from a specification perspective.
Summary. Once again the X3D JSON design is looking pretty solid, and our tool suites are pretty powerful at detecting edge-case issues.
Thanks again for steady sustained progress. Have fun with X3D JSON!
all the best, Don
--
Don Brutzman Naval Postgraduate School, Code USW/Br brutzman at nps.edu
Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
More information about the x3d-public
mailing list