[x3d-public] announce: X3D XML schema updates - regression testing, validation issues

Sat Sep 24 10:44:54 PDT 2016

On 9/21/2016 10:11 PM, Don Brutzman wrote:
> Summary: an excellent X3D XML schema update has been deployed supporting the X3D Object Model.  Note that the following schema changes should not affect validation of XML-encoded scenes.  Regression testing of scenes in the X3D Example Archives will occur next.  All feedback welcome.

Performed X3D XML schema validation testing of 3864 total X3D scenes in the X3D Example Archives.

	http://www.web3d.org/x3d/content/examples/X3dResources.html#Examples

No major new errors were found, so the schemas appear to remain stable.  8)

Aside from scenes with known errors or intentional deviations, found the following problems.  Representative excerpts follow.

===================================================================================
a. GeoViewpoint validation errors for navType and headlight attributes, even though they don't appear in the .x3d source scene.  Example excerpt:

http://www.web3d.org/x3d/content/examples/Basic/Geospatial/CaliforniaCampuses.x3d
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\Geospatial\CaliforniaCampuses.x3d:35:141: cvc-complex-type.3.2.2: Attribute 'navType' is not allowed to appear in element 'GeoViewpoint'.
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\Geospatial\CaliforniaCampuses.x3d:35:141: cvc-complex-type.3.2.2: Attribute 'headlight' is not allowed to appear in element 'GeoViewpoint'.

These attributes were removed from GeoViewpoint in X3D version 3.3 since corresponding support already existed in NavigationInfo node.
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/versionContent.html#GeoViewpoint

Not exactly sure why this error was triggered, but found that navType and headlight were mistakenly omitted from v3.2 schema (other schema versions OK).  Corrected.

===================================================================================
b. HAnimJoint not allowing child Transform node.

http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/DiamondManLOA-1.x3d
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\HumanoidAnimation\DiamondManLOA-1.x3d:63:64: cvc-complex-type.2.4.a: Invalid content was found starting with element 'Transform'. One of '{IS, MetadataDouble, MetadataFloat, MetadataInteger, MetadataSet, MetadataString, HAnimJoint, HAnimSegment, HAnimDisplacer}' is expected.
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\HumanoidAnimation\DiamondManLOA-1.x3d:80:28: The content of element type "HAnimJoint" must match "(IS?,(MetadataDouble|MetadataFloat|MetadataInteger|MetadataSet|MetadataString|ProtoInstance)?,(HAnimJoint|HAnimSegment|HAnimDisplacer|ProtoInstance)*)".

Specification comment submitted separately, providing detail why such content is needed.  If allowed, will amend schema to validate such content.

===================================================================================
c. H-Anim enumeration name additions for high-fidelity feet and hands in X3D version 4 have some mismatches.  Saved for another day.

http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/HAnimModelFootLeft.x3d
http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/HAnimModelFootRight.x3d
http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/HAnimModelHandLeft.x3d
http://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/HAnimModelHandRight.x3d

===================================================================================
d. ShaderPart node is not allowing CDATA plain-text content (similar to X3D Script node).

http://www.web3d.org/x3d/content/examples/Basic/Shaders/SimpleShader.x3d
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\Shaders\SimpleShader.x3d:52:30: The content of element type "ShaderPart" must match "(IS?,(MetadataBoolean|MetadataDouble|MetadataFloat|MetadataInteger|MetadataSet|MetadataString|ProtoInstance)?)".

Script, ShaderPart and ShaderProgram are the only nodes with a "mixed" content model, i.e. allowing child text (usually protected within a CDATA block).

	<xs:complexType mixed="true">

The schema constructs look OK and consistent in v3.2 also.

The example does validate OK using XML Spy and online X3D Validator.  So the schema appears correct, perhaps this was a false positive.

However, looking at the DTD, it appears that #PCDATA needs to be added to ShaderPart and ShaderProgram.

	<!ELEMENT Script ( #PCDATA|field|IS|%MetadataNodes; )* >

	<!ELEMENT ShaderPart ( (IS?), (%MetadataNodes;)? ) >
	<!ELEMENT ShaderProgram ((field)*, (IS?), (%MetadataNodes;)? ) >

Also note how the Script content model in the DTD is unordered - I believe this is because whitespace character data can appear before or after the other elements.

Am aligning DTD definitions for ShaderPart/ShaderProgram to match Script.  Of note is that the XML Schema should still detect ordering of child elements OK.

===================================================================================
e. <unit> (UNIT) statement not handled properly within <head> section.

http://www.web3d.org/x3d/content/examples/Basic/Units/UnitBacteria.x3d
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\Units\UnitBacteria.x3d:6:52: cvc-complex-type.2.4.a: Invalid content was found starting with element 'meta'. One of '{unit}' is expected.

   <head>
     <unit category='length' conversionFactor='0.000001' name='micro'/>
     <meta content='UnitBacteria.x3d' name='title'/>

This is apparently flagged because the schema is looking for statements in the following order: <component> <meta> or <unit>.

However the X3D specification is looking for a different order:

7.2.5.1 Organization
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/core.html#Organization
"The PROFILE statement may be optionally followed by one or more COMPONENT, UNIT and/or META statements in that order. There may be multiple of each of the COMPONENT, UNIT, and/or META statements."

Modified schema to match.  No problem in DTD.  TODO added to Schematron checks to provide a less mysterious diagnostic.

===================================================================================
f. ImageTexture3D not validating within IsoSurfaceVolumeData.

http://www.web3d.org/x3d/content/examples/Basic/VolumeRendering/IsoSurfaceSkull.x3d
> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\VolumeRendering\IsoSurfaceSkull.x3d:30:235: cvc-complex-type.2.4.a: Invalid content was found starting with element 'ImageTexture3D'. One of '{ProtoInstance}' is expected.

         <IsoSurfaceVolumeData dimensions='1.28 1.28 1.28' surfaceValues='.15'>
           <CartoonVolumeStyle colorSteps='32'/>
           <ImageTexture3D containerField='voxels' url='"datasets/nrrd/skull.nrrd" "https://snoid.sv.vt.edu/medical/X3DVolumes/examples/datasets/nrrd/skull.nrd" "https://savage.nps.edu/Savage/VolumeRendering/datasets/nrrd/skull.nrd"'/>
         </IsoSurfaceVolumeData>

X3D specification:
41.4.6 IsoSurfaceVolumeData
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/volume.html#IsoSurfaceVolumeData

IsoSurfaceVolumeData : X3DVolumeDataNode {
   SFFloat [in,out] contourStepSize  0        (-∞,∞)
   SFVec3f [in,out] dimensions       1 1 1    (0,∞)
   SFNode  [in,out] gradients        NULL     [X3DTexture3DNode]
   SFNode  [in,out] metadata         NULL     [X3DMetadataObject]
   MFNode  [in,out] renderStyle      []       [X3DVolumeRenderStyleNode]
   SFFloat [in,out] surfaceTolerance 0        [0,∞)
   MFFloat [in,out] surfaceValues    []       (-∞,∞)
   SFNode  [in,out] voxels           NULL     [X3DTexture3DNode]
   SFVec3f []       bboxCenter       0 0 0    (-∞,∞)
   SFVec3f []       bboxSize         -1 -1 -1 [0,∞) or -1 -1 -1
}

33  Texturing3D Component, 33.3.1 X3DTexture3DNode
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/texture3D.html#X3DTexture3DNode

The following nodes implement the X3DTexture3DNode interface: ComposedTexture3D, ImageTexture3D, PixelTexture3D.  These nodes are aligned in the schema as Texture3DContentModel.  Note that they can serve in to child fields: gradients and voxels.

The schema content-model problem appears to be use of <xs:sequence> instead of <xs:choice> (even though unbounded already).

Relaxed to <xs:choice> which is consistent with specification and less restrictive.  No problem with ordering seen in DTD.

===================================================================================
g. MultiTexture not validating within Appearance.

> C:\x3d-code\www.web3d.org\x3d\content\examples\Basic\development\ProfileExampleInterchange.x3d:70:22: The content of element type "Appearance" must match "(IS?,(MetadataDouble|MetadataFloat|MetadataInteger|MetadataSet|MetadataString|ProtoInstance)?,(LineProperties|FillProperties|Material|ImageTexture|MovieTexture|PixelTexture|TextureTransform|MultiTextureTransform|ProtoInstance)*)".

       <Shape>
         <LineSet/>
         <Appearance>
           <MultiTexture/>
           <MultiTextureTransform/>
         </Appearance>
       </Shape>

18.4.3 MultiTexture
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/texturing.html#MultiTexture

	MultiTexture : X3DTextureNode {

12.4.1 Appearance
http://www.web3d.org/documents/specifications/19775-1/V3.3/Part01/components/shape.html#Appearance

Appearance : X3DAppearanceNode {
   SFNode [in,out] fillProperties   NULL [FillProperties]
   SFNode [in,out] lineProperties   NULL [LineProperties]
   SFNode [in,out] material         NULL [X3DMaterialNode]
   SFNode [in,out] metadata         NULL [X3DMetadataObject]
   MFNode [in,out] shaders          []   [X3DShaderNode]
   SFNode [in,out] texture          NULL [X3DTextureNode]
   SFNode [in,out] textureTransform NULL [X3DTextureTransformNode]
}

In schema, Appearance extends X3DAppearanceNode, which contains AppearanceChildContentModel,
which extends (AppearanceChildContentModelNoProtoInstance + ProtoInstance).

AppearanceChildContentModelNoProtoInstance can contain MultiTexture and MultiTextureTransform nodes.

Apparently the problem was that MultiTexture was missing from v3.0 DTD, other DTDs and schemas look OK.  Fixed.

===================================================================================

Summary of changes follow.

X3D DTD update activity:

24 September 2016, brutzman
- (v3.1+) allow CDATA content for script source code within ShaderPart, ShaderProgram
- (v3.0)  allow MultiTexture in AppearanceChildNodes

X3D Schema update activity:

24 September 2016, brutzman
- (v3.2)  Restored GeoViewpoint attributes for navType and headlight
- (v3.3+) Corrected order of contained statements within <head>: component, unit, meta
- (v3.3+) Relaxed ordering of IsoSurfaceVolumeData content.

===================================================================================

Tough set of fixes but useful. These appear to be lingering issues that were not previously diagnosed/corrected in previous regression tests.

No problems appear to have occurred from the X3D Object Model changes, thankfully.  Everything listed here now checked into version control.

Questions and corrections welcome, of course - that's why these updates are posted!

Will deploy these schema, DTD changes (plus corresponding X3D Object Model and Java SAI Library refinements) to the website tonight.

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman