[x3d-public] VRML97 and ClassicVRML model validation from command line using view3dscene

Wed Aug 23 07:08:37 PDT 2023

Don,

1. As for some missing X3D 4.0 nodes: Yes, this can happen, as in the
past I didn't carefully check whether all X3D 4.0 nodes have been
added to CGE/view3dscene, and indeed I was fixing later some missing
ones (like H-Anim nodes from X3D 4.0).

    If you can extract the list of missing X3D 4.0 nodes from above
logs, I'll be grateful -- I didn't find relevant messages now from the
logs you linked, but admittedly I only looked quickly.

2. To avoid the "Error EDownloadError when loading inline file from
URL ...: Downloading network resources (from "http" or "https"
protocols) is not enabled. Set global EnableBlockingDownloads:=true to
enable." pass the additional command-line option

    --enable-downloads

    to tovrmlx3d. Note that it will definitely increase the validation
time. The tovrmlx3d will download the linked resources (Inline X3D
content, X3D content referred by EXTERNPROTOs etc.) from the Internet
at validation.

Regards,
Michalis

śr., 23 sie 2023 o 08:18 Brutzman, Donald (Don) (CIV)
<brutzman at nps.edu> napisał(a):
>
> Michalis, appreciate your adding this feature as part of tovrmlx3d executable.  I’ve integrated into the Ant build scripts and tested operation satisfactorily for all .wrl and .x3dv conversions in the X3D Examples Archives.
>
>
>
> Example command-line invocation on windows, a little terser works out OK for these logs:
>
>
>
> tovrmlx3d.exe C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForWebAuthors\Chapter04ViewingNavigation/CollisionExample.x3dv --validate
>
>
>
> Regression-test results online (and in version control to track progress) as follows.  Most look clean.
>
>
>
> https://www.web3d.org/x3d/content/examples/build.validate.VRML.all.log.txt
>
> https://www.web3d.org/x3d/content/examples/build.validate.x3dv.all.log.txt
>
>
>
> https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/content/examples/build.validate.VRML.all.log.txt
> https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/content/examples/build.validate.x3dv.all.log.txt
>
>
>
> Execution is fast!  Execution took around a half hour for over 4000 examples, most of that time was likely writing test-result outputs to console.
>
>
>
> I haven’t tried fixing any model errors or X3dToVrml97.xslt stylesheet conversion errors yet (e.g. true false vice TRUE FALSE etc.) – that will happen.
>
>
>
> You might see some issues in there that relate to your side, saw a few such as
>
> new X3D4 nodes that might not yet be included in view3dscene,
> Backslash and \n error reports didn’t include line numbers (might be hard to isolate),
>
>
>
> Saw this error message, but not sure how to implement setting the property via command line, please advise:
>
>
>
> tovrmlx3d: Warning: X3D: Error EDownloadError when loading inline file from URL https://www.web3d.org/x3d/content/examples/Basic/HumanoidAnimation/NancyDiving.x3d: Downloading network resources (from "http" or "https"  protocols) is not enabled. Set global EnableBlockingDownloads:=true to enable.
>
>
>
> Happy to now have these post-conversion tests available, enabling continued deliberate improvements in quality assurance (QA).  Again thanks for a tremendous capability, very helpful.
>
>
>
> Have fun with VRML!  8)
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br        brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA    +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics https://faculty.nps.edu/brutzman
>
>
>
>
>
> -----Original Message-----
> From: Michalis Kamburelis <ichalis.kambi at gmail.com>
> Sent: Monday, August 14, 2023 5:40 AM
> To: Brutzman, Donald (Don) (CIV) <brutzman at nps.edu>
> Cc: X3D Public Mailing List (x3d-public at web3d.org) <x3d-public at web3d.org>
> Subject: Re: VRML97 and ClassicVRML model validation from command line using view3dscene
>
>
>
> Don,
>
>
>
> In short: I added now –validate option to tovrmlx3d 😊
>
>
>
> Example usage on a valid model:
>
>
>
> ```
>
> $ ./tovrmlx3d ../demo-models/x3d/axis.x3d –validate $ echo $?
>
> 0
>
> ```
>
>
>
> So empty output, and exit status 0, as expected from most tools. You can use this with Ant, Make or anything else.
>
>
>
> Example usage on a model with problems:
>
>
>
> ```
>
> $ ./tovrmlx3d ../demo-models/x3d/warnings/invalid_statement_omitted_1.x3dv
>
> --validate
>
> tovrmlx3d: Warning: VRML/X3D: Error when reading, will skip the rest of X3D file: Error at line 15 column 2: Expected node type or DEF or USE, got “}”
>
> Exception “Exception”:
>
> Validation failed (consult the warnings above), exiting with non-zero status
>
>
>
> $ echo $?
>
> 1
>
> ```
>
>
>
> So you get output on stderr, and non-zero exit status.
>
>
>
> Please give ~1 hour for Jenkins to process it, and this new version of tovrmlx3d will be part of “snapshot" version on
>
> https://castle-engine.io/view3dscene.php .
>
>
>
> I used above test models from https://github.com/castle-engine/demo-models .
>
>
>
> Details:
>
>
>
> So far, we did validation just by converting and piping output to /dev/null, so "tovrmlx3d xxxx.x3d > /dev/null". But I agree this was
>
> unoptimal:
>
>
>
> - Explicit "--validate" means we don't needlessly waste time converting to /dev/null if you actually just want to validate. We also don't produce eventual warnings when writing, that would sometimes appear and duplicate warnings we already did when reading.
>
>
>
> - In the future, maybe, explicit "--validate" may allow to run some additional tasks dedicated to validation (maybe because they are time-consuming and not necessary for just usual opening). We don't have such tasks right now, but it would be nice to have such option available.
>
>
>
> - With explicit "--validate", any warning or error causes non-zero exit status. Makes sense, since a warning (even if we can continue
>
> working) still means validation failed.
>
>
>
> So ... I added "--validate" option to tovrmlx3d.
>
>
>
> Of note: A rename is on the horizon, for a few reasons (glTF export, consistency with other CGE tools). Most likely:
>
> - tovrmlx3d -> caste-scene-process
>
> - view3dscene -> castle-scene-view
>
> I haven't made up my mind 100% yet, if rename happens I will of course announce it prominently everywhere :)
>
>
>
> Regards,
>
> Michalis
>
>
>
> niedz., 13 sie 2023 o 17:53 Brutzman, Donald (Don) (CIV) <brutzman at nps.edu> napisał(a):
>
> >
>
> > Hi Michalis.  Thanks for all of your immense, continuing work.
>
> >
>
> > I am hoping to add validation of converted .wrl and .x3dv files to the regression test suite for X3D Example Archives.  This will check the quality of output from the X3dToVrml.xslt stylesheets, and improve the quality assurance (QA) of all VRML models offered there – over 4000 in each case. In each case I’ll create and commit a build.log text file so that we can keep track of compliance and progress.
>
> >
>
> > X3D Resources, Examples: Scene Archives for X3D
>
> > https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>
> >
>
> > To do so, am thinking that command-line invocation (CLI) of your view3dscene or tovrmlx3d executable tools can be accomplished through addition to the existing Ant build.xml scripts.  Am keen to proceed with that task.
>
> >
>
> > Am looking through the view3dscene CLI documentation, but am not exactly sure how to accomplish this.
>
> >
>
> > view3dscene, 4. Command-line options
>
> > https://castle-engine.io/view3dscene.php#section_command_line_options
>
> >
>
> > view3dscene, 4.2. Converting to X3D
>
> > https://castle-engine.io/view3dscene.php#section_converting
>
> >
>
> > Additional options needed, or not seen:
>
> >
>
> > Validation report only, plain text via console.  (Perhaps a -validate
>
> > switch is a variation on your ---write switches.) Avoid creation of
>
> > additional files, if possible.  (I might send each output to a single
>
> > temp file, but all that file writing will likely slow down the
>
> > already-long overall process.)
>
> >
>
> > Please advise if there is a way to invoke one of your tools to accomplish this.  CLI example will be helpful.  Thanks in advance for all guidance.
>
> >
>
> > Have fun with VRML!   8)
>
> >
>
> > all the best, Don
>
>