[x3d-public] "Bayesian Networks" for X3D Test Coverage -- use case for X3DJSAIL aspect oriented transient extension -- collecting statistics on X3D examples

[John Carlson]

Jeff, this mail is primarily asking for your expertise in testing models with AOP.  In particular, we are trying to get test coverage by counting class (# instances) and method (# relationships) usage in Java.  Where the count == 0 we know we have to add a test case. Feel free to forward this to Benoit Baudry or Sudipto Ghosh, if they have experience with Generating test coverage with bayesian networks, AOP and models (Introduction please)—perhaps they can get us jump started on the necessary AOP code to insert into our API or  a reference.  I am trying to make this *really practical* and *easy*, this is not a “out there” subject (I think I only use the “Bayesian” because of the natural hierarchical style of X3D API, and some tests are dependent on others).  I do not want to “reinvent the wheel”, but IPR for X3D may require a liberal licensing of any code.   We need code we can share between partners for free, with no legal entanglements (Open Source).

I am getting this sorted.   I will use 1) test cases combined with 2) object model (X3DUOM or X3DJSAIL) to build a Bayesian network which will discover the non-occurrence of test cases (node, relationship or field appearances = none) within our X3D test suite (has NIST or anyone else done this, and can I see the report?).  Then using 0% covered cases and the object model, along with other low percentage cases, I will generate rare test cases which extend our test coverage.

See: https://link.springer.com/content/pdf/10.1007/978-3-540-71289-3_22.pdf

I would rather work on this than working on standards documents or medical AI or even x3dv4, thanks, and it fits better within my “career” goals.  I will probably write a specification for the AOP part of this we write.

I am hoping this seems valuable to do. Yes? Would it be possible to use X3DJSAIL (an API to build test models)  to collect and report the statistics?  Or maybe subclasses of X3DJSAIL? It would seem like we would want to store most of the information in X3D classes, rather than generate a bunch of X3D instances in memory—those will be transient.  Perhaps we could use aspect-oriented programming (AOP) and have a great cross-cutting concern (collection of # of relationships between instances and instances; and instances and attributes, I am thinking I can only do this on class leaves, but maybe someone has an idea to “non-override” or construct the appropriate statistics variable names), and we can implement without changing the X3DJSAIL source code. Hmm.

Jeff, do you have or know of any AOP software that collects statistics on API calls I can use to do the previous?   It seems like it would be fairly generic—something one could write in a hour or so, not including testing. Would you like to contribute something to X3D or Open Source?  Thanks!

Seems like a great use of our current products and “old” technology (yeah, I’m an old geezer) to achieve total test coverage.  Don? Vince? Others?  Recommendations for current, well used testing products to generate coverage and report?

It would also be a great way to test for memory leaks in X3DJSAIL and HAnim 2.0.

Should I test all unique attribute values found in X3DUOM for enumerations? I think not. Maybe for a rendering test?

Can we do this in NetBeans or Eclipse? Let me know. I use vim 😊

Revealing my secret: Once I generate the statistics, I get to do a lot of cool test case visualization for a slide show (we can extend our visualization (or visualized) suite to APIs (read ontologies), Alex and Michael)! Can I do the visualizations in X3D, do you think?

On this page, https://www.itl.nist.gov/div897/ctg/vrml/why-nist.html it says

More specifically, we have the following interests in VRML 2.0:
• Develop a formal specification and abstract test suite (ATS) for VRML 2.0. The work will be coordinated and where possible accomplished as part of the ISO editing of the VRML 2.0 specification. This formal specification is an experiment for NIST, and it is hoped that this type of specification will aid in moving towards the automatic generation of tests. 
As a member, can I know any details on this formal specification? Is it X3DJSAIL documentation? Thanks! (we will see if Bayes offers a better way to generate tests).

I’ll try to record the progress of the statistics as we generate tests.   I did not do that last time ☹.  Well, email.

Is there a X3D conformance testing mailing list I should join?

If I get this all together, I may present at Web3D conference.

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190209/99af1177/attachment-0001.html>