[x3d-public] X3D DOCTYPE problem; X3D Validator has workaround, necessary path forward confirmed

Sun Sep 13 16:34:00 PDT 2020

Again thanks for helpful diagnosis.

I have been able to modify the X3D Validator so that if http:// is used for XML DOCTYPE or XML Schema, they get replaced with https:// instead before working on the full model for validation.  Seems to work OK.

[1] X3D Validator
     https://savage.nps.edu/X3dValidator/validate
     "The X3D Validator performs comprehensive Quality Assurance (QA) testing to ensure the validity of X3D3 and X3D4 models."

Meanwhile despite work with XML Catalog and other techniques, am unable to get SAX-based Java tools to work with the redirect.  This breaks the majority of our regression tests and conversion stylesheets.

So one tool is fixed, and XML Spy was adept, but many others are now blocked from using X3D models.

Therefore the immediate path forward is:

a. Restore direct access to standardized-url DOCTYPE and schema.  Fileset patterns are

	http://www.web3d.org/specifications/x3d-*.dtd
	http://www.web3d.org/specifications/x3d-*.xsd

and the long-term path forward is:

b. X3D Working Group consider specification change to all relevant X3D XML Encoding DOCTYPE and Schema references to support https:// as well.  Encouraging strong security seems like a good idea for all versions of X3D.

Further trouble reports and feedback welcome.  Again apologies for all lost development time, hopefully the server configuration for doctype and schema will be restored soon.

On 9/12/2020 7:34 PM, vmarchetti at kshell.com wrote:
> 
> 
> As observed at 10:20 PM EDT on 12 Sept 2020
> 
> A HTTP request to https://www.web3d.org/specifications/x3d-3.3.dtd made with the curl tool (on Mac OS) returns the 142k DTD file, appears well formed
> 
> The request to the http (unencrypted, discouraged by modern browsers) http://www.web3d.org/specifications/x3d-3.3.dtd returns a redirect HTTP response, with headers:
> 
> HTTP/1.1 302 Found
> Date: Sun, 13 Sep 2020 02:15:33 GMT
> Server: Apache
> X-Content-Type-Options: nosniff
> Location: https://www.web3d.org/specifications/x3d-3.3.dtd
> Cache-Control: max-age=1209600
> Expires: Sun, 27 Sep 2020 02:15:33 GMT
> Content-Length: 232
> Content-Type: text/html; charset=iso-8859-1
> 
> 
> and the body of the HTTP response is an HTML-markup version of this information, so not well formed DTD.
> 
> It appears that some of the validation tools will follow the redirect and some will not.
> 
> I do not know if the server configuration has been changed recently to redirect requests to URLS  in http://www.web3d.org/specifications/* space.
> 
> Request for our webpages i.e. http://www.web3d.org/ have been redirected for some time.
> 
> Solutions to this problem would be
> 
> -- allow server to directly server requests to http://www.web3d.org/specifications/* without redirection
> -- update or configure tools to follow HTTP 302 redirections
> 
> Vince Marchetti
> 
>> On Sep 12, 2020, at 7:55 PM, Don Brutzman <brutzman at nps.edu> wrote:
>>
>> We noticed an emergent problem with X3D Validator about a week ago and, with difficulty, have been trying to sleuth what the root cause might be.
>>
>> After multiple diagnostic efforts, we now think that the problem is on the web3d.org server, there is some issue within the DTD files themselves.  Unfortunately there is also a problem with my account which currently prevents troubleshooting further and (hopefully) fixing it.
>>
>> This problem has blocked proper DTD validation, which might in turn block validation of numerous tools loading/validating X3D models.  Have confirmed that some tools work (probably due to prior caching) and other tools fail. Example failure output:
>>
>>> check well-formedness
>>> http://www.web3d.org/specifications/x3d-3.3.dtd:1:3: The markup declarations contained or pointed to by the document type declaration must be well-formed.
>>> Could not validate document C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\HelloWorld.x3d: org.xml.sax.SAXParseExceptionpublicId: ISO//Web3D//DTD X3D 3.3//EN; systemId: http://www.web3d.org/specifications/x3d-3.3.dtd; lineNumber: 1; columnNumber: 3; The markup declarations contained or pointed to by the document type declaration must be well-formed.
>>> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\HelloWorld.x3d is not a valid XML document
>>
>> Meanwhile XML Spy can validate the same model successfully.  It is both exceptionally strict and also added native X3D support a few months back.  So this approach confirms that X3D models we are testing remain unchanged and correct.
>>
>> Web3D Communications Team and webmaster have been notified about the access problem preventing further troubleshooting.
>>
>> The DOCTYPE problem does not appear to prevent successful X_ITE and X3DOM display of X3D models.
>>
>> * https://x3dgraphics.com/examples//X3dForWebAuthors/Chapter02GeometryPrimitives/BoxExampleX_ITE.html
>> * https://x3dgraphics.com/examples//X3dForWebAuthors/Chapter02GeometryPrimitives/BoxExampleX3dom.xhtml
>>
>> Once this is also sorted out, will post regarding resumption of capability.  Apologies for any difficulties this problem causes tool developers and scene modelers.
>>
>> all the best, Don
>> --
>> Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
>> X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman
>>
>> _______________________________________________
>> communications mailing list
>> communications at web3d.org
>> http://web3d.org/mailman/listinfo/communications_web3d.org
> 

all the best, Don
-- 
Don Brutzman  Naval Postgraduate School, Code USW/Br       brutzman at nps.edu
Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
X3D graphics, virtual worlds, navy robotics http://faculty.nps.edu/brutzman