[x3d-public] x3d.py error; fixed and deployed updates to X3D Schema, X3DUOM, X3DJSAIL, X3D Ontology and x3d.py X3DPSAIL

Don Brutzman brutzman at nps.edu
Thu Apr 30 21:34:12 PDT 2020


Thanks for tracking this down Andreas, very useful.  Also agreed that it is a good idea to put parameters in the .XML() function signature.

The syntax.html restrictions you quote are all part of Chapter 8, HTML syntax.

XML rules appear in Chapter 9, XML Syntax.  Once you subtract warnings about Processing Instructions and DOCTYPES in DOM Document, which are never used within X3D scene graphs anyway, this chapter seems very short and just like... plain old XML.

There used to be a Quirks mode in HTML5 that allowed mismatched tags, matching tags, etc.  Looks like it was removed it as browsers matured.  Seems like HTML parser writers instead went out of their way to enforce void tags and no singleton tags in "HTML syntax", effectively guaranteeing XML incompatibility.

So our first python .XML() parameter would seem to be (default) syntax="HTML" with "XML" alternative, each with corresponding strict compliance to all requirements in HTML5.2 Recommendation.

Next parameter: hmmm, the precision requirements for different data in a scene can vary.  For example, significant digits for points and normals and colors and angles and transparency each have varying perceptual sensitivity.  One thing for sure is that default precision=None should have no effect since digital signature or round-trip testing might be involved (First Do No Harm).  Not sure how we should express this parameter.

Another conceivable parameter is verbose=False though not sure why anyone would want default attribute values as an output... perhaps as a learning or authoring assist.  Not hard to include if desired...

Are there any other .XML() parameter options that a python programmer (or Jupyter notebook author) might want?


On 4/29/2020 1:03 PM, Andreas Plesch wrote:
> I suppose the example notebooks are getting sufficiently useful to
> start to clean them up and then better organize in a more permanent
> place.
> 
> .XML() as a function does not seem unusual and would allow for
> parameters. In addition to a short_empty_elements=False option,
> controlling the precision (and perhaps format) of output of numerical
> fields could be also very useful.
> 
> With regards to singletons my understanding is that HTML5 generally
> does not allow them. The exception is a well defined list of 14
> elements which are called void or empty elements, for example the
> <area> element. For those 14 elements closing tags are actually
> invalid (although most browsers do not mind them). In addition, the
> preferred syntax for empty elements is without the solidus(/) : <area>
> . However, <area /> is also allowed.
> Since all of the X3D tag names are not on this list of empty elements,
> they all need the closing tag. Note, that all this is only true for
> the HTML encoding of HTML5 which is not XML. XHTML is xml and all the
> XML rules apply.
> 
> Ok. I looked up relevant sections in the HTML5 spec.:
> 
> https://www.w3.org/TR/html52/syntax.html#void-elements : lists the 14 elements
> 
> "Void elements only have a start tag; end tags must not be specified
> for void elements."
> 
> Further, X3D tag names are (currently) considered "Unknown Elements"
> since they are not part of HTML. Unknown elements are classified as
> "Normal Elements:
> https://www.w3.org/TR/html52/single-page.html#elements-in-the-dom.
> Normal elements have a closing tag since they are not known void
> elements and may have text or other content:
> https://www.w3.org/TR/html52/syntax.html#writing-html-documents-elements
> "  normal elements have a start tag to indicate where they begin, and
> an end tag to indicate where they end."
> 
> There are also Foreign Elements which are elements from the MathML
> namespace and the SVG namespace. These are defined elsewhere and
> include singletons. HTML recognizes this and allows the short
> self-closing syntax </> for those: " Foreign elements must either have
> a start tag and an end tag, or a start tag that is marked as
> self-closing, in which case they must not have an end tag."
> 
> https://www.w3.org/TR/html52/syntax.html#start-tags
> 
> item 6. explains the self-closing syntax:
> 
> " Start tags must have the following format:
> ...
> 6. Then, if the element is one of the void elements, or if the element
> is a foreign element, then there may be a single U+002F SOLIDUS
> character (/). This character has no effect on void elements, but on
> foreign elements it marks the start tag as self-closing.
> 
> 7.  Finally, start tags must be closed by a U+003E GREATER-THAN SIGN
> character (>)."
> 
> There are also Custom Elements. Since custom elements are not in the
> list of 14 void elements, they also have an end tag.
> 
> -Andreas
> 
> 
> On Wed, Apr 29, 2020 at 2:28 AM Don Brutzman <brutzman at nps.edu> wrote:
>>
>> Thanks for confirm Andreas.  Your example keeps getting better!  (Suggested backing out viewpoints to more distant perspective if possible; also typo "baseed").
>>
>> Given the state of browser support, yes that makes sense to add option for adding closing tags rather than singletons.  My understanding is that this is a limitation of some HTML browsers... any idea why, have you found any rationale or documentation about it?  Wondering if there is a reason (or maybe they are just unreconstructed HTML4 quirk worshippers).
>>
>> I will work on a property and also persistent configuration settings, if possible.  Further can ensure consistent capabilities in X3dToX3dom/X3dToX_ITE stylesheet switches and also X3DJSAIL.
>>
>> Also hoping to omit the () function suffix from .XML() method.  Response will be less rapid this time (actually have a day job too).
>>
>> Am planning to get full regression testing of export working before adding similar capabilty for .VRML encoding so that any latent errors are detected and fully ironed out.  Of note is that x3d.py binary is only 277K, much less than source (which is over 1MB now).  As a result I won't worry about trying to refactor output serializations in an object-oriented way, that can only cause errors and grief when compared to the easily inspected serialization code already in place.  Efficiency already seems great so we'll reserve effort for what actually adds value.
>>
>> On 4/27/2020 1:15 PM, Andreas Plesch wrote:
>>> Hi Don,
>>>
>>> thanks for the quick response. I can confirm that 0.0.28 now properly
>>> generates XML for MF values (arrays), and SF vectors (sequences).
>>>
>>> https://nbviewer.jupyter.org/github/andreasplesch/OCCToX3D/blob/conda2/notebooks/stp_x3d_with_vp4.ipynb
>>>
>>> has an example at the end where I used x3d.py for generating Text and
>>> Inline nodes.
>>>
>>> For HTML purposes it would be useful to have an option for .XML() to
>>> generate a full end tag. Inline().XML(short_empty_elements=False)
>>> would then generate
>>>
>>> <Inline></Inline>
>>>
>>> I know this is a bit out of scope of x3d.py but perhaps possible to
>>> add to the autogeneration.
>>>
>>> Currently, I use the standard, python included ElementTree module to
>>> do this but including it in x3d.py would save parsing and reexporting
>>> the XML.
>>>
>>> thanks again, Andreas
>>>
>>> On Mon, Apr 27, 2020 at 2:32 AM Don Brutzman <brutzman at nps.edu> wrote:
>>>>
>>>> Did a lot of work this weekend, all interrelated and cross-checked.  Uploads in progress.
>>>>
>>>> Test logs are themselves maintained in version control so that (a) output changes are evident during development, and (b) other developers can see what is expected.
>>>>
>>>> a. X3D Schema and X3DUOM
>>>>      http://web3d.org/specifications
>>>> - annotation appinfo added to a number of String enumeration values, which can be either strict choices or optional values in X3D spec.
>>>> - X3DUOM updated to indicate simpleType as well as baseType for each field when appropriate, giving fuller expression to object model.
>>>> - X3DUOM name attributes now appear first for readability.
>>>> - These changes supported the follow-on library improvements, additional work also applied.
>>>>
>>>> -------
>>>>
>>>> b. X3DJSAIL
>>>>      X3D Java Scene Access Interface Library (X3DJSAIL)
>>>>      https://www.web3d.org/specifications/java/X3DJSAIL.html
>>>> - Corresponding update applied to X3DJSAIL with only minor tweaks.
>>>>
>>>> -------
>>>>
>>>> c. X3D Ontology
>>>>      https://www.web3d.org/x3d/content/semantics/semantics.html
>>>> - simple-type enumeration lists added, excerpt follows
>>>> - typing of fields now reflects these simple types
>>>> - name attribute typing is also stricter, e.g. some names are NMTOKEN
>>>> - found feature in protege for diagnosing error lines in turtle .ttl
>>>> - ready for further work on accessType representations, continuing prior efforts
>>>>
>>>> :closureTypeChoices rdf:type rdfs:Datatype ;
>>>>      rdfs:label "closureTypeChoices" ;
>>>>      dc:description "closureTypeChoices are strictly allowed enumeration values for ArcClose2D closureType field" ;
>>>>      rdfs:range xs:NMTOKEN ;
>>>>      rdfs:domain [ owl:unionOf ( 'PIE' 'CHORD' ) ] .
>>>> # PIE "Connects arc endpoints to center, forming a pie wedge"
>>>> # CHORD "Connects arc endpoints directly to each other, as in chord on a circle"
>>>>
>>>> -------
>>>>
>>>> d. x3d.py X3DPSAIL package
>>>>      https://www.web3d.org/x3d/stylesheets/python/python.html
>>>> - renamed toXML() as XML(), added XML() export methods to all X3D types
>>>> - used these for export output for all field values for consistency
>>>> - added unit tests for all types, tests successful
>>>>      https://www.web3d.org/x3d/stylesheets/python/examples/PythonX3dSmokeTests.py
>>>>      https://www.web3d.org/x3d/stylesheets/python/build.examples.log.txt
>>>>
>>>> example excerpts:
>>>> - - - - - - - - - -
>>>> SFString test       = 'test setter'
>>>> SFString test.value = test setter
>>>> SFString test.XML() = test setter
>>>> SFString     isValidSFString(test)=True
>>>> SFString assertValidSFString(test)
>>>> MFString test       = [test, setters and getters]
>>>> MFString test.value = ['test', 'setters and getters']
>>>> MFString test.XML() = "test" "setters and getters"
>>>> MFString     isValidMFString(test)=True
>>>> MFString assertValidMFString(test)
>>>> downcasting by dereferencing simple-list value=[one], type=<class 'x3d.MFString'> as one
>>>> SFString test downcast legal singleton: SFString(MFString("one"))='one', type=<class 'x3d.SFString'>
>>>> MFString test   upcast legal SF value:  MFString(SFString("two"))=['two'], type=<class 'x3d.MFString'>
>>>> - - - - - - - - - -
>>>> MFNode test.XML()             =
>>>> <Group DEF='B'/>
>>>> <Shape DEF='C'>
>>>>      <Appearance DEF='D'>
>>>>        <Material DEF='E'/>
>>>>      </Appearance>
>>>> </Shape>
>>>> <WorldInfo DEF='F'/>
>>>> - - - - - - - - - -
>>>> - simple-type enumeration lists added, excerpt follows
>>>> - added assertion methods to check enumeration values valid, throws exception otherwise
>>>>
>>>> CLOSURETYPECHOICES = (
>>>>        # strict set of allowed values follow, no other values are valid
>>>>        'PIE', # Connects arc endpoints to center, forming a pie wedge
>>>>        'CHORD' # Connects arc endpoints directly to each other, as in chord on a circle
>>>> )
>>>> def assertValidClosureType(fieldName, value):
>>>>        """
>>>>        Utility function to assert type validity of closureTypeChoices value, otherwise raise X3DTypeError with diagnostic message.
>>>>        Note MFString enumeration values are provided in XML syntax, so check accordingly.
>>>>        """
>>>>        if  not value:
>>>>            return True # no failure on empty defaults
>>>>        if  isinstance(value,SFString) or isinstance(value,str):
>>>>            if str(value) in CLOSURETYPECHOICES:
>>>>                return True
>>>>            raise X3DTypeError(fieldName + ' value=' + value + ' does not match allowed enumerations in CLOSURETYPECHOICES=' + str(CLOSURETYPECHOICES))
>>>>        if  isinstance(value,MFString):
>>>>            if MFString(value).XML() in CLOSURETYPECHOICES:
>>>>                return True
>>>>            raise X3DTypeError(fieldName + ' value=' + MFString(value).XML() + ' does not match allowed enumerations in CLOSURETYPECHOICES=' + str(CLOSURETYPECHOICES))
>>>>
>>>> class ArcClose2D(_X3DGeometryNode):
>>>> # ...
>>>>        @property # getter - - - - - - - - - -
>>>>        def closureType(self):
>>>>            """Defines whether pair of line segments connect to center (PIE), or single line-segment chord connects arc endpoints (CHORD)."""
>>>>            return self.__closureType
>>>>        @closureType.setter
>>>>        def closureType(self, closureType=None):
>>>>            if  closureType is None:
>>>>                closureType = SFString.DEFAULT_VALUE
>>>>            assertValidSFString(closureType)
>>>>            assertValidClosureType('closureType', closureType)
>>>>            self.__closureType = closureType
>>>>
>>>> -------
>>>>
>>>> So Andreas hopefully this should give you everything you might need and expect for .XML() export.
>>>>
>>>> Am hoping Loren can help me with incantation so that all we need syntactically is .XML export.
>>>>
>>>> It will be straightforward to add .VRML export (.X3DV and .WRL methods perhaps) once we are satisfied with .XML for .x3d files/strings.
>>>>
>>>> I like how Python usually fails bitterly until you get things exactly right.
>>>>
>>>> Didn't have time to check anything with python packaging yet.
>>>>
>>>> Feedback welcome.
>>>>
>>>> On 4/23/2020 12:14 PM, Don Brutzman wrote:
>>>>> Thanks for trouble report.  This was similar to error a few days ago, common cause was incorrect handling of MFString fields by toXML() method.
>>>>>
>>>>> Now fixed and deployed.  I've performed regression tests on about 4000 models, have ~30 Traceback errors to continue diagnosis on.  So... toXML() is pretty much implemented now!
>>>>>
>>>>> * Python X3D Package x3d.py
>>>>>      X3D Python Scene Access Interface Library (X3DPSAIL)
>>>>>      https://www.web3d.org/x3d/stylesheets/python/python.html
>>>>>
>>>>> * pip install x3d; Python package x3d
>>>>>      https://pypi.org/project/x3d
>>>>>
>>>>> Further improvement suggestions and bug reports welcome.  Hoping to complement other work, today's Design Printing Scanning demo was great - thanks very much.
>>>>>
>>>>> Have fun with X3D Python!
>>>>>
>>>>> On 4/23/2020 3:55 AM, Andreas Plesch wrote:
>>>>>> When I try this:
>>>>>>
>>>>>> inline=Inline(url=["Vent.x3d"])
>>>>>>
>>>>>> toXML() generates an error:
>>>>>>
>>>>>> inline.toXML()
>>>>>>
>>>>>> ---------------------------------------------------------------------------
>>>>>> TypeError                                 Traceback (most recent call last)
>>>>>> <ipython-input-26-2e0e2ed2e4db> in <module>
>>>>>> ----> 1 inline.toXML()
>>>>>>
>>>>>> ~/x3d.py in toXML(self, indentLevel)
>>>>>>      22341             result += " load='" + str(self.load) + "'"
>>>>>>      22342         if self.url != list():
>>>>>>> 22343             result += " url='" + self.url + "'"
>>>>>>      22344         if self.visible != True:
>>>>>>      22345             result += " visible='" + str(self.visible) + "'"
>>>>>>
>>>>>> TypeError: can only concatenate str (not "list") to str
>>>>>>
>>>>>> This is the x3d.py linked from the web3d web page.
>>>>>>
>>>>>> Do I need to do something different?
>>>>>>
>>>>>> -Andreas
>>>> all the best, Don
>>>> --
>>>> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
>>>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
>>>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
>>>
>>>
>>>
>>
>> all the best, Don
>> --
>> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
> 
> 
> 

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman



More information about the x3d-public mailing list