[x3d-public] VRML97 and ClassicVRML model validation from command line using view3dscene
John Carlson
yottzumm at gmail.com
Mon Aug 14 11:25:47 PDT 2023
"The overall json build log looks pretty clean to me. Link and excerpt
follow" Why not actually test the JSON output from x3d.py with standard
ECMAScript or even better, standard python packages?
The Emperor has no clothes! The x3d.py output JSON fails at jshint,
node.js, jsonlint and JSON Schema, if you would use them. But the JSON is
not right, it's WIP, as I said, not even passing JSON standards, ignoring
any JSON Schema stuff! I'm going to keep on promoting x3d-tidy. I'm
guessing you're pretty afraid to run the python generated from 4000
examples. Why not just try HelloWorldProgramOutput*.py? Yes, you have to
change it to emit JSON, of course! Let me give you a hint...use JSON from
inside python for a quick smoke test!
No, print(prependLineNumbers(newModelVRML, err.lineno)) isn't testing JSON
validation.
Here's how to test JSON (not JSON schema, that's way too slow) in python:
https://www.w3schools.com/python/python_json.asp
I will update my NetBeans to :
https://github.com/apache/netbeans/releases/tag/19-rc4 after I send this
message.
Details below:
Comparing:
https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/java/examples/HelloWorldProgramOutput.py
to:
https://github.com/coderextreme/X3DJSONLD/blob/master/src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py
(using
my version of Don's stylesheet).
I see a difference in indentation, which means the stylesheet change,
probably within the last month, will check.
~/X3DJSONLD/src/main/lib/stylesheets
$ ls -ltr !*
ls -ltr X3dToPython.xslt /c/x3d-code/
www.web3d.org/x3d/stylesheets/X3dToPython.xslt
-rw-r--r-- 1 john 197121 215215 Jul 18 10:25 /c/x3d-code/
www.web3d.org/x3d/stylesheets/X3dToPython.xslt
-rw-r--r-- 1 john 197121 215215 Jul 29 05:35 X3dToPython.xslt
(no differences either). So what's up? Here's where I get my stylesheet
from AFAIK:
https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X3dToPython.xslt
(Is there a better repository? Seems hidden? I see X3D-Edit on Web3d
Consortium's github, let me try that--not seeing any significant
difference, but I'm not using Python inside NetBeans yet.
How do I see if I have an up to date X3D-Edit? Will try to download. What
is Apache NetBeans 19???? Ah, a release candidate... Please update your
Apache 19 links here, https://savage.nps.edu/x3d-edit/ to point here:
https://github.com/apache/netbeans/releases (plus all mention of NetBeans
18 on the X3D-Edit site), not here:
https://savage.nps.edu/Savage/developers.html#NetBeans
[image: image.png]
Here's the X3D-Edit about:
[image: image.png]
Here's where I found the link:
https://www.web3d.org/x3d/stylesheets/python/python.html
$ cp /c/x3d-code/
www.web3d.org/x3d/stylesheets/java/examples/HelloWorldProgramOutput.x3d ../d
ata/
~/X3DJSONLD/src/main/shell (master)
$ bash don.sh ../data/HelloWorldProgramOutput.x3d
Investigating output code:
$ git diff HelloWorldProgramOutput.py
warning: in the working copy of
'src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py', LF will be
replaced by CRLF the next time Git touches it
diff --git a/src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py
b/src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py
index 2d6ce6199..f3e166e20 100644
--- a/src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py
+++ b/src/main/python/net/x3djsonld/data/HelloWorldProgramOutput.py
@@ -1,6 +1,10 @@
####################################################################################################
#
-# Now available: developmental python x3d.py package on PyPI for import.
+# Invoking X3D model self-test:
+#
+# $ python HelloWorldProgramOutput.py
+#
+# Python package x3d.py package is available on PyPI for import.
# This approach simplifies Python X3D deployment and use.
# https://pypi.org/project/x3d
#
@@ -9,7 +13,7 @@
# or
# python -m pip install x3d
#
-# Developer options for loading x3d package:
+# Developer options for loading x3d package in other Python programs:
#
# from x3d import * # preferred approach, terser source that avoids
x3d.* class prefixes
#
@@ -46,7 +50,7 @@
newModel=X3D(class_='x3dModel.class',id_='x3dModel.id',profile='Full',style_='x3
meta(content='https://www.web3d.org/specifications/java/X3DJSAIL.html
',name='reference'),
meta(content='HelloWorldProgramOutput.java',name='generator'),
meta(content='6 September 2016',name='created'),
- meta(content='3 July 2023',name='modified'),
+ meta(content='30 July 2023',name='modified'),
meta(content='X3D Java Scene Access Interface Library
(X3DJSAIL)',name='generator'),
meta(content='
https://www.web3d.org/specifications/java/examples/HelloWorldProgram.java
',name='generator'),
meta(content='Netbeans https://www.netbeans.org',name='generator'),
No significant changes.
So let's look at the code:
try:
# print('check newModel.JSON() serialization...')
newModelJSON=newModel.JSON() # test export method JSON() for exceptions
during export
# print(prependLineNumbers(newModelJSON)) # debug
print("Python-to-JSON export of JSON output successful (under
development)")
except Exception as err: # usually SyntaxError
print("*** Python-to-JSON export of JSON output failed:",
type(err).__name__, err)
if newModelJSON: # may have failed to generate
print(prependLineNumbers(newModelJSON,err.lineno))
So let's try running it:
py HelloWorldProgramOutput.py
x3d.py package 4.0.64.4 loaded, have fun with X3D Graphics!
Self-test diagnostics for HelloWorldProgramOutput.py:
meta information, info: continued development and testing in progress
*** Python-to-XML well-formed XML document test failed.
x3dSchemaUrl=https://www.web3d.org/specifications/x3d-4.0.xsd
XMLSchemaParseError cannot import namespace '
http://www.w3.org/2000/09/xmldsig#': Entities are forbidden
(entity_name='dsig')
Python-to-VRML export of VRML output successful
Python-to-JSON export of JSON output successful (under development)
python HelloWorldProgramOutput.py load and self-test diagnostics complete.
So yes, a WIP as I said. I don't even think Don's tests actually produce
JSON output that can be validated with jshint, node.js, jsonlint, JSON
Schema, or any other JSON Parser or ECMAScript/JavaScript environment! So
let's dig and investigate this.
diff hey.py HelloWorldProgramOutput.py
414d413
< print(newModelJSON)
$ py hey.py >hey.json
$ vi hey.json # to take off header x3d.py doesn't write to standard error.
$ jsonlint hey.json
Error: Parse error on line 2:
"e,
------------^
Expecting 'STRING', 'NUMBER', 'NULL', 'TRUE', 'FALSE', '{', '[', got ','
at Object.parseError
(C:\Users\john\AppData\Roaming\npm\node_modules\jsonlint\lib\jsonlint.js:55:11)
at Object.parse
(C:\Users\john\AppData\Roaming\npm\node_modules\jsonlint\lib\jsonlint.js:132:22)
at parse
(C:\Users\john\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:82:14)
at main
(C:\Users\john\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:135:14)
at Object.<anonymous>
(C:\Users\john\AppData\Roaming\npm\node_modules\jsonlint\lib\cli.js:179:1)
at Module._compile (node:internal/modules/cjs/loader:1233:14)
at Module._extensions..js (node:internal/modules/cjs/loader:1287:10)
at Module.load (node:internal/modules/cjs/loader:1091:32)
at Module._load (node:internal/modules/cjs/loader:938:12)
at Function.executeUserEntryPoint [as runMain]
(node:internal/modules/run_main:83:12)
Also, make these changes to you python generated programs for actually
testing JSON:
$ diff HelloWorldProgramOutput.py hey.py
0a1,3
> import json
>
>
413a417
> json.loads(newModelJSON)
john at DESKTOP-8OPL3IE MINGW64 ~/X3DJSONLD/src/main/python/net/x3djsonld/data
(master)
$ py hey.py
x3d.py package 4.0.64.4 loaded, have fun with X3D Graphics!
Self-test diagnostics for HelloWorldProgramOutput.py:
meta information, info: continued development and testing in progress
*** Python-to-XML well-formed XML document test failed.
x3dSchemaUrl=https://www.web3d.org/specifications/x3d-4.0.xsd
XMLSchemaParseError cannot import namespace '
http://www.w3.org/2000/09/xmldsig#': Entities are forbidden
(entity_name='dsig')
Python-to-VRML export of VRML output successful
*** Python-to-JSON export of JSON output failed: JSONDecodeError Expecting
value: line 2 column 11 (char 12)
*** issue in line 2 of 1226 ***
[ 1] {
[ 2] "X3D":,
[ 3] {
[ 4] "encoding":"UTF-8",
[ 5] "$id": "
https://www.web3d.org/specifications/x3d-4.0-JSONSchema.json",
python HelloWorldProgramOutput.py load and self-test diagnostics complete.
I know you don't want any imports in x3d.py, but at least have them there
for testing! My word! At least try them, to see if they work without any
downloads using pip.
$ cp hey.json ~/x3dvalidate/examples/hey.json
$ cd ~/x3dvalidate/
$ node examples/hey.json
node:internal/modules/cjs/loader:1305
throw err;
^
SyntaxError: C:\Users\john\x3dvalidate\examples\hey.json: Unexpected token
',', ..." "X3D"
:,
{
"... is not valid JSON
at parse (<anonymous>)
at Module._extensions..json (node:internal/modules/cjs/loader:1302:39)
at Module.load (node:internal/modules/cjs/loader:1091:32)
at Module._load (node:internal/modules/cjs/loader:938:12)
at Function.executeUserEntryPoint [as runMain]
(node:internal/modules/run_main:83:12)
at node:internal/main/run_main_module:23:47
Node.js v20.4.0
$ node x3dvalidate.js examples/hey.json
(similar output)
So let's actually edit the code..
$ cp examples/hey.json examples/heynew.json
I'm confused!
I see:
"meta":
{
{
"@content":"continued development and testing in progress",
"@name":"info"
}
},
X3dToJson.xslt produces:
"meta": [
{
"@name":"title",
"@content":"HelloWorldProgramOutput.x3d"
},
{
"@name":"info",
"@content":"continued development and testing in progress"
},
Whereas hey.json contains:
"meta":
{
{
"@content":"HelloWorldProgramOutput.x3d",
"@name":"title"
}
},
"meta":
{
{
"@content":"continued development and testing in progress",
"@name":"info"
}
},
Why don't we have a JSON standard yet? Anyone can claim they have X3D
JSON, even if their output isn't valid JSON! Their own tools don't even
have to agree!
I've already cleaned up the JSON header in X3dToPython.xslt. I haven't
gotten farther than that!
See zip updates X3duomToX3dPythonPackage.xslt previously sent. That at
least fixes the meta statement issues, I'm fairly sure.
Excellent work on X3DJSAIL, Don. Please don't go around claiming x3d.py
creates valid, verifiable JSON. I know you didn't. I will check the new
release version of NetBeans and see if I'm wrong and there's some secret,
new version of x3d.py. Note that
https://www.web3d.org/x3d/stylesheets/python/python.html points to NetBeans
18!
John
<https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/stylesheets/X3dToPython.xslt>
On Sun, Aug 13, 2023 at 9:29 PM Brutzman, Donald (Don) (CIV) <
brutzman at nps.edu> wrote:
> x3d.py methods for JSON match the patterns listed in our documentation for
> this suggested encoding.
>
>
>
> - X3D to JSON Stylesheet Converter
> - The X3D to JSON stylesheet converts .x3d XML to .json, supporting
> the forthcoming JSON Encoding for X3D Graphics.
> - https://www.web3d.org/x3d/stylesheets/X3dToJson.html
>
>
>
> So do all of the 4,000 autogenerated examples in our X3D Example Archives.
>
>
>
> - X3D Resources, Examples: Scene Archives for X3D
> - https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>
>
>
> The overall json build log looks pretty clean to me. Link and excerpt
> follow
>
>
>
> - https://www.web3d.org/x3d/content/examples/build.json.all.log.txt
> -
> https://sourceforge.net/p/x3d/code/HEAD/tree/www.web3d.org/x3d/content/examples/build.json.all.log.txt
>
>
>
> X3D examples: X3dForAdvancedModeling.processScenes.json:
>
> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForAdvancedModeling/AdditiveManufacturing//CaffeinePubChem2519SticksColor.x3d
> processing with X3dToJson stylesheet
>
> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForAdvancedModeling/AdditiveManufacturing//CaffeinePubChem2519SticksColor.json
> validation checking with jslint
>
> C:\x3d-code\www.web3d.org\x3d\content\examples\X3dForAdvancedModeling/AdditiveManufacturing//CaffeinePubChem2519SticksColor.json
> validation checking with json-schema-validator
>
> ... does not support latest JSON Schema, this test is not yet usable
>
>
>
> By the way, your mention of “x3d-tidy” still seems misnamed since we
> already have one, for a few years now…
>
>
>
> - X3D Tidy for Scene Cleanup, Corrections and Modifications
> - X3D Tidy is an XSLT stylesheet that checks for simple authoring
> difficulties in X3D scenes and fixes them.
> - https://www.web3d.org/x3d/stylesheets/X3dTidy.html
>
>
>
> Have fun with X3D using Python and JSON! 8)
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics
> https://faculty.nps.edu/brutzman
>
>
>
> *From:* x3d-public <x3d-public-bounces at web3d.org> *On Behalf Of *John
> Carlson
> *Sent:* Sunday, August 13, 2023 2:37 PM
> *To:* GPU Group <gpugroup at gmail.com>
> *Cc:* X3D Public Mailing List (x3d-public at web3d.org) <x3d-public at web3d.org
> >
> *Subject:* Re: [x3d-public] VRML97 and ClassicVRML model validation from
> command line using view3dscene
>
>
>
> AFAIK, JSON output with x3d.py is still a work in progress.
>
>
>
> x3d-tidy should have .x3dj (JSON) I/O, see:
>
>
>
> https://www.npmjs.com/package/x3d-
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.npmjs.com%2Fpackage%2Fx3d-&data=05%7C01%7Cbrutzman%40nps.edu%7Cf9f531f61f1146f6d8b008db9c4592c1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638275594856167951%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=1bc06Bv4xzzI7qjNya6viyy5NP9vBLNcrvzY%2By5n8VM%3D&reserved=0>
>
>
>
> you use it with npx,
>
>
>
> like
>
>
>
> $ npx x3d-tidy -i file.x3dv -o file.x3dj
>
>
>
> I’m trying to figure out why x3dvalidate doesn’t publish properly (maybe
> it does). Anyway, when used with npx, parameters are not accepted. If you
> want to have a copy in your project, it’s available here:
>
>
>
> https://github.com/coderextreme/x3dvalidate
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcoderextreme%2Fx3dvalidate&data=05%7C01%7Cbrutzman%40nps.edu%7Cf9f531f61f1146f6d8b008db9c4592c1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638275594856167951%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=S8Ms6xLnZ8FE4EEcm0PGz3229OjC2vJHpSpCMY5Gao0%3D&reserved=0>
>
>
>
> That should work ok
>
>
>
> On Sun, Aug 13, 2023 at 12:48 PM GPU Group <gpugroup at gmail.com> wrote:
>
> Related: if/once import functions are made for X3D.py, then exports to
> x3dv, x3d, html, json can be done from there.
>
>
>
>
>
> On Sun, Aug 13, 2023 at 9:54 AM Brutzman, Donald (Don) (CIV) <
> brutzman at nps.edu> wrote:
>
> Hi Michalis. Thanks for all of your immense, continuing work.
>
>
>
> I am hoping to add validation of converted .wrl and .x3dv files to the
> regression test suite for X3D Example Archives. This will check the
> quality of output from the X3dToVrml.xslt stylesheets, and improve the
> quality assurance (QA) of all VRML models offered there – over 4000 in each
> case. In each case I’ll create and commit a build.log text file so that we
> can keep track of compliance and progress.
>
>
>
> - X3D Resources, Examples: Scene Archives for X3D
> - https://www.web3d.org/x3d/content/examples/X3dResources.html#Examples
>
>
>
> To do so, am thinking that command-line invocation (CLI) of your
> *view3dscene* or *tovrmlx3d* executable tools can be accomplished through
> addition to the existing Ant build.xml scripts. Am keen to proceed with
> that task.
>
>
>
> Am looking through the view3dscene CLI documentation, but am not exactly
> sure how to accomplish this.
>
>
>
> - view3dscene, 4. Command-line options
> - https://castle-engine.io/view3dscene.php#section_command_line_options
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcastle-engine.io%2Fview3dscene.php%23section_command_line_options&data=05%7C01%7Cbrutzman%40nps.edu%7Cf9f531f61f1146f6d8b008db9c4592c1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638275594856167951%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=LijWonezwvWZ39NSfrz9Zggq6k56pjQeImgcRkGMUxs%3D&reserved=0>
>
>
>
> - view3dscene, 4.2. Converting to X3D
> - https://castle-engine.io/view3dscene.php#section_converting
> <https://nam10.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcastle-engine.io%2Fview3dscene.php%23section_converting&data=05%7C01%7Cbrutzman%40nps.edu%7Cf9f531f61f1146f6d8b008db9c4592c1%7C6d936231a51740ea9199f7578963378e%7C0%7C0%7C638275594856167951%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=zWbWuv%2FbTqJvsCIlO%2BVMCEr2d%2FvDdMsDG%2Bz7isNlC%2BE%3D&reserved=0>
>
>
>
> Additional options needed, or not seen:
>
>
>
> 1. Validation report only, plain text via console. (Perhaps a
> -validate switch is a variation on your ---write switches.)
> 2. Avoid creation of additional files, if possible. (I might send
> each output to a single temp file, but all that file writing will likely
> slow down the already-long overall process.)
>
>
>
> Please advise if there is a way to invoke one of your tools to accomplish
> this. CLI example will be helpful. Thanks in advance for all guidance.
>
>
>
> Have fun with VRML! 8)
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270, MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, navy robotics
> https://faculty.nps.edu/brutzman
>
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230814/4d0ff0ed/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 42593 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230814/4d0ff0ed/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 19490 bytes
Desc: not available
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230814/4d0ff0ed/attachment-0003.png>
More information about the x3d-public
mailing list