[x3d-public] X3D DOCTYPE problem; X3D Validator has workaround, necessary path forward confirmed; fixed, upgrade path proposed

John Carlson yottzumm at gmail.com
Sat Sep 19 21:11:33 PDT 2020


Changing everything to https:// (2 fields per file, I think) sounds like a
lot of work.  I have only changed DOCTYPE in my examples, and I did it all
by hand.  I cannot imagine doing it on 3000 or more examples, unless you
can do it with X3DTidy or similar.

I suggest you create a small script that will run on Windows Terminal (WSL,
Ubuntu), Linux Bash, and MacOSX zsh for all those who have ANY http: marked
content.  This is Huge, and may cause significant downtime for many people.

I am surprised you are not shouting LOUDER!

John

On Sat, Sep 19, 2020 at 7:45 PM Don Brutzman <brutzman at nps.edu> wrote:

> Special thanks to Vince Marchetti who tracked down and fixed the
> unintended redirects from http to https that thwarted operation of
> regression testing and other validation tools.
>
> [1] X3D Specifications: Schema and DOCTYPE Validation
>      https://www.web3d.org/specifications
>
> Full regression testing in progress for all models in X3D Example
> Archives, with all tests so far are passing as expected.
>
> [2] X3D Resources, Examples: Scene Archives for X3D
>      https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>
> Looking ahead: subject to X3D Working Group discussion and confirmation, I
> expect that all of our recommended "http" addresses for DOCTYPE and XML
> Schema should be upgraded to https.
>
> [3] X3D Scene Authoring Hints: Validation of X3D Scenes using DTD and XML
> Schema
>
> https://www.web3d.org/x3d/content/examples/X3dSceneAuthoringHints.html#Validation
>
> Please advise if anyone has concerns with this approach.
>
> Considering pros and cons:  additional security for X3D4 models is no
> problem.  Backwards compatibility for X3D3 XML models seems fully workable
> since both http-served and https-served copies of these files remain
> available.  Improved file security (avoiding any possibility of Man In The
> Middle attacks on our URIs) is good.
>
> [4] Wikipedia: Man-in-the-middle attack
>      https://en.wikipedia.org/wiki/Man-in-the-middle_attack
>
> Absent objections, I hope to accomplish this change from http to https for
> DOCTYPE and XML Schema to all published content this month.
>
>
> On 9/13/2020 4:34 PM, Don Brutzman wrote:
> > Again thanks for helpful diagnosis.
> >
> > I have been able to modify the X3D Validator so that if http:// is used
> for XML DOCTYPE or XML Schema, they get replaced with https:// instead
> before working on the full model for validation.  Seems to work OK.
> >
> > [1] X3D Validator
> >      https://savage.nps.edu/X3dValidator/validate
> >      "The X3D Validator performs comprehensive Quality Assurance (QA)
> testing to ensure the validity of X3D3 and X3D4 models."
> >
> > Meanwhile despite work with XML Catalog and other techniques, am unable
> to get SAX-based Java tools to work with the redirect.  This breaks the
> majority of our regression tests and conversion stylesheets.
> >
> > So one tool is fixed, and XML Spy was adept, but many others are now
> blocked from using X3D models.
> >
> > Therefore the immediate path forward is:
> >
> > a. Restore direct access to standardized-url DOCTYPE and schema.
> Fileset patterns are
> >
> >      http://www.web3d.org/specifications/x3d-*.dtd
> >      http://www.web3d.org/specifications/x3d-*.xsd
> >
> > and the long-term path forward is:
> >
> > b. X3D Working Group consider specification change to all relevant X3D
> XML Encoding DOCTYPE and Schema references to support https:// as well.
> Encouraging strong security seems like a good idea for all versions of X3D.
> >
> > Further trouble reports and feedback welcome.  Again apologies for all
> lost development time, hopefully the server configuration for doctype and
> schema will be restored soon.
> >
> >
> > On 9/12/2020 7:34 PM, vmarchetti at kshell.com wrote:
> >>
> >>
> >> As observed at 10:20 PM EDT on 12 Sept 2020
> >>
> >> A HTTP request to https://www.web3d.org/specifications/x3d-3.3.dtd
> made with the curl tool (on Mac OS) returns the 142k DTD file, appears well
> formed
> >>
> >> The request to the http (unencrypted, discouraged by modern browsers)
> http://www.web3d.org/specifications/x3d-3.3.dtd returns a redirect HTTP
> response, with headers:
> >>
> >> HTTP/1.1 302 Found
> >> Date: Sun, 13 Sep 2020 02:15:33 GMT
> >> Server: Apache
> >> X-Content-Type-Options: nosniff
> >> Location: https://www.web3d.org/specifications/x3d-3.3.dtd
> >> Cache-Control: max-age=1209600
> >> Expires: Sun, 27 Sep 2020 02:15:33 GMT
> >> Content-Length: 232
> >> Content-Type: text/html; charset=iso-8859-1
> >>
> >>
> >> and the body of the HTTP response is an HTML-markup version of this
> information, so not well formed DTD.
> >>
> >> It appears that some of the validation tools will follow the redirect
> and some will not.
> >>
> >> I do not know if the server configuration has been changed recently to
> redirect requests to URLS  in http://www.web3d.org/specifications/* space.
> >>
> >> Request for our webpages i.e. http://www.web3d.org/ have been
> redirected for some time.
> >>
> >> Solutions to this problem would be
> >>
> >> -- allow server to directly server requests to
> http://www.web3d.org/specifications/* without redirection
> >> -- update or configure tools to follow HTTP 302 redirections
> >>
> >> Vince Marchetti
> >>
> >>> On Sep 12, 2020, at 7:55 PM, Don Brutzman <brutzman at nps.edu> wrote:
> >>>
> >>> We noticed an emergent problem with X3D Validator about a week ago
> and, with difficulty, have been trying to sleuth what the root cause might
> be.
> >>>
> >>> After multiple diagnostic efforts, we now think that the problem is on
> the web3d.org server, there is some issue within the DTD files
> themselves.  Unfortunately there is also a problem with my account which
> currently prevents troubleshooting further and (hopefully) fixing it.
> >>>
> >>> This problem has blocked proper DTD validation, which might in turn
> block validation of numerous tools loading/validating X3D models.  Have
> confirmed that some tools work (probably due to prior caching) and other
> tools fail. Example failure output:
> >>>
> >>>> check well-formedness
> >>>> http://www.web3d.org/specifications/x3d-3.3.dtd:1:3: The markup
> declarations contained or pointed to by the document type declaration must
> be well-formed.
> >>>> Could not validate document C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\HelloWorld.x3d:
> org.xml.sax.SAXParseExceptionpublicId: ISO//Web3D//DTD X3D 3.3//EN;
> systemId: http://www.web3d.org/specifications/x3d-3.3.dtd; lineNumber: 1;
> columnNumber: 3; The markup declarations contained or pointed to by the
> document type declaration must be well-formed.
> >>>> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\HelloWorld.x3d
> is not a valid XML document
> >>>
> >>> Meanwhile XML Spy can validate the same model successfully.  It is
> both exceptionally strict and also added native X3D support a few months
> back.  So this approach confirms that X3D models we are testing remain
> unchanged and correct.
> >>>
> >>> Web3D Communications Team and webmaster have been notified about the
> access problem preventing further troubleshooting.
> >>>
> >>> The DOCTYPE problem does not appear to prevent successful X_ITE and
> X3DOM display of X3D models.
> >>>
> >>> *
> https://x3dgraphics.com/examples//X3dForWebAuthors/Chapter02GeometryPrimitives/BoxExampleX_ITE.html
> >>> *
> https://x3dgraphics.com/examples//X3dForWebAuthors/Chapter02GeometryPrimitives/BoxExampleX3dom.xhtml
> >>>
> >>> Once this is also sorted out, will post regarding resumption of
> capability.  Apologies for any difficulties this problem causes tool
> developers and scene modelers.
> >>>
> >>> all the best, Don
> >>> --
> >>> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
> >>> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
> >>> X3D graphics, virtual worlds, navy robotics
> http://faculty.nps.edu/brutzman
> >>>
> >>> _______________________________________________
> >>> communications mailing list
> >>> communications at web3d.org
> >>> http://web3d.org/mailman/listinfo/communications_web3d.org
> >>
> >
> > all the best, Don
>
> all the best, Don
> --
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA   +1.831.656.2149
> X3D graphics, virtual worlds, navy robotics
> http://faculty.nps.edu/brutzman
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20200919/d3c5910c/attachment.html>


More information about the x3d-public mailing list