[x3d-public] Semantics (specifically Analogical Predication) using predicate sets or relationship types in processes, systems and worlds

[John Carlson]

This message is very long.  Brian, this is 3 different “views” of the world and an attempted mapping using semantics (specifically, analogic predication in the form of predicate sets).

Abstract
======================================================================================================
The predicates we have focused on in the structure of X3DUOM and XML Schema have primarily been the Unified Modeling Language (UML) relationships.  However, there are also predicates for X3D scenes found in the abstract specification which I will subsume by talking about physics predicates. The realization of X3D scenes takes place in the world of the raster display.  

The primary predicates for X3DUOM and XML schema have been what I call sign, semiotic, signing system or sign world oriented, whose nouns are elements in sets and maps, whereas the physics world has predicates have measurements in units of length, time, and mass, deltas and higher dimensions, mass over time, length over time--scalars in vectors and matrices. A third world is the world of the raster display and desktop, or the photographic/movie world, with pixels or fragments in windows, images, textures, menus and pointers.   

In physics, predicates might be called logistics.  Below these predicates relating physics matrices are “converted” from the quantitative functions to qualitative maps by using words or signs.  I have no good way of converting sign predicates to physics predicates, except through the use of functions, scripts, recipes, plans and programs (reflexively associated in UML), from adhoc instruction by a human, or from templatized and inferenced logs of using a program in which physical manipulation of the prototypical model in the computer is used (Model Transformation By Demonstration) to guide the transformation of other models. This is a pattern, and  the strategy method may be employ with these to meet goals and are executed by the interpreter pattern.  However, things like Force Directed Graphs may be considered for ungainly amounts of data.  Often the number of classes exceed capable display space however.  In the photographic and movie world you capture actions in the frame of the image.  So a frame is the single element in a log of actions.  You can infer future action (such as driving a car) by looking at an inferenced log of frames.

These also affect the material or appearance of shapes in X3D documents.  Both the sign and x3d scenes map , project or render to the display.
This might be called the projection of the physical world onto a raster via a viewpoint  Perhaps the display world can map signs to raster to physical through the use of annotations and other metaphors.   The keys and the mouse are primary predicate or event generators on the desktop, whereas in VR/AR, hand controllers might be used for mobility.

There are other worlds in X3D.  For example, one might consider actions which convert system symptoms or diagnoses found in predicate logs (looking at history of a system) and interactively through inspections (looking a live system) to plans of actions.   This is flooding in a whole new set of predications, great.

These three worlds reflect the 3 types of nouns found in them, namely, pixel, symbol and curve.  There are generators for these to create virtual worlds, loops (numbers), procedures (structures), fractals (structures, images), chaos (weather, smoke), statistics (odds), quantum (atoms), spatiotemporal (vectors and matrices), text (grammars), tables(schemas) .. or everything (ontologies) logs (records of predicates), inspections (looking at a live predicate)

More?

UML and X3DUOM predicates (signs)
Can we document all predicate sets and top level predicate tuples (relationships) used in X3D in the X3D ontology and/or X3DUOM? The top level predicate is association, below that, predicate sets bifurcate into tuples or dualities made up of 2 or possibly more predicates.  Predicate sets are predicates related with IS_A or tuple/relationship (“has subclass”,  “has superclass”). A predicate is the same thing as one way of a relationship, without specified cardinality, assuming relationships are 2 way.  There is a rumor that relationships are 3 way, for example, a Publisher/Subscriber relationship may have a Mediator (ROUTE), but perhaps the predicate is the 3rd object involved, and the Mediator (the predicate) has another predicate to the Publisher [out] and another predicate with the Subscriber [in], thereby splitting the relationship among three objects (this is common in relational algebra I believe).  Mediate(out, in), is_a (sub, super), has_a (piece, whole), and realization(has class, has instance) are 4 of the largest set of relationships in X3D.  You can find similar relationships between classes in UML https://creately.com/blog/diagrams/understanding-the-relationships-between-classes/ the major headings are the relationships.  There’s 16 predicates in relationships in UML+2 for Mediator=18 predicates, but only 9 top level predicate sets (sub, super, piece, whole, class, instance, in, out, associated with—assuming associations are directional) I’ve only named 8 predicate sets made up of 4 relationships in X3DUOM.

X3D  and world predicates (length, mass, time and deltas in multiple dimensions)
===================================================================================================
Are there other UML relationships found in X3D related to associations? What are the predicates, predicate sets and predicate tuples, can we create some subsets of associate that are interesting besides those already found?  If association is the top level predict (above is_a) , what are some associations between predicates?   I saw some spatiotemporal relationships in a document. What are the predicate sets and predicate tuples for those? In Time(come Before, come After), In Width(Left, Right), In Depth(In Front, Behind), In Height(Above, Below)  Time and Depth (Follow, Lead) in Size and Time(Fatter, Taller, Longer, Thinner, Smaller, Shorter) in In Depth and Time(Zoom in, Zoom Out) Rotation, Space and Time (roll or lean left and right, yaw or turn left and right, and pitch or face up and down, face left, face right) Time and Height (Raise, Up, Lower, Down). Time and Width( Pan Left, Pan Right).  In general, there are three fundamental things that are measured and they study in physics, namely Length a relationship between two points in space in multiple dimensions, called Size (Width, Depth and Height), Time (see above) in multiple dimensions called Time Zones and Mass. Mass(More, Less or different Atoms).  What is mass in multiple dimensions, Weight?  Length has a relative origin, time has a relative origin and mass has a relative origin (note that you can’t have negative length, time or mass values, except in relationships, so there are arrows in length and mass as well as time).  When you include velocity you can have negative position, when you consider relative position on the globe you can have relative time, and you can have negative weight (when diving). Do we want to consider quaternions for Time and Mass? Motion along a length is called velocity.  change in time is called the passing of seasons.   Loss or addition of atoms is called losing or gaining mass.   We may also be able to find some more predicate sets we can document in X3DUOM in conceptual dependency theory.  See: umich link (or below the link).   There are also relationships between components and nodes.  What is this predicate set?

Publish/Subscribe predicate set, Human Development/Work, and Sport predicates
======================================================================================================
The crux of this is the rapid growth of the predicate tuple set {Out/In, Relate/Understand, Describe/Derive, Offense/Defense, Generate/Recognize Talk/Listen, Produce/Consume, Teach/Learn, Create/Destroy, Write/Read, Send/Receive, Output/Input, Buy/Sell, Load/Extract, Memorize/Recall, Develop/Use, Request/Reply, Query/Response, Broadcast/Poll} and maybe {Spread/Gather, Join/Split, Multiplex/Demultiplex, Publish/Subscribe} for multiple producers or consumers or their analogical predication in sign worlds, virtual worlds, real world, systems, communication, economy, math, military, patterns, education and computers.  If you want to help me, you can contribute your own distinct predicate unity sets or predicate tuple/duality/trinity/… sets. In an effort to talk about things different from Input, Processing, and Output, I also talk about objects in worlds and other predicate sets such as Prototype/Design/Storyboard, Act/Perform/Operate, Build/Script/Develop, Research and Brainstorm, Prepare/Plan/Train, Direct/Desire/Specify/Select/Goal Set/Target, and Strategize (what are their dualities?) see: https://coderextreme.net/chord.html [ Brian L. note the use of the word chord here. How can one develop and plan at the same time, for example using top-down decomposition in a programming language, structure charts, data flow diagrams? How can I develop and specify at the same time for example, literate programming, formal specification language?  I can develop and use (unit test) with similar languages.  Why does natural language seem to cover all the bases for most things, and instead of one language for computers we have hundreds if not thousands to choose from? Is it because of its basis on math, which has domain specific languages inside it? Or is it merely a jargon issue where everyone invents their own names for things and that’s why is it called a net in fishing, a graph in math, a rete in humans, a doily in crocheting, and a network in botany and socializing?   We’ve developed some conventions. We have the Input/Output/Process convention down cold and have come up with many derivations.  Can we develop more conventions with this email thread, perhaps those related to the SDLC?] I am more interested in further analogical predication with the other predicate sets in the various worlds like the military, movie making and product development.  I am less interested in mathematical predicates (translate/scale/rotate) at this time, but these are important.

Conceptual Dependency Theory (Schank) predicates
======================================================================================================
I think I need to step back before I dig too deep into analogical predication and look at conceptual dependency theory: https://en.wikipedia.org/wiki/Conceptual_dependency_theory  from: https://deepblue.lib.umich.edu/bitstream/handle/2027.42/30278/0000679.pdf;jsessionid=B8039541E82D2A2EB49A7E14548F33E7?sequence=1

Representations of English sentences could be constructed by piecing together these building blocks to form a conceptual dependency graph. PTRANS: The transfer of location of an object ATRANS: The transfer of ownership, possession, or control of an object MTRANS: The transfer of mental information between agents MBUILD: The construction of a thought or of new information by an agent ATTEND: The act of focusing attention of a sense organ toward an object GRASP: The grasping of an object by an actor so that it may be manipulated PROPEL: The application of a physical force to an object MOVE: The movement of a bodypart of an agent by that agent INGEST:The taking in of an object (food, air, water, etc.) by an animal EXPEL: The expulsion of an object by an animal SPEAK: The act of producing sound, including non-communicative sounds Figure 1. The conceptual dependency primitives.

The set of primitives varied somewhat during the course of conceptual dependency theory, but it consisted of approximately 10-12 predicates, each of which represented a type of action. The primitives are shown in Figure 1.

Summary of table relating Conceptual Dependency, Sign and World predicates (extend?)
======================================================================================================

PTRANS: Width, Depth and Height and Time
ATRANS:  Grant, Revoke control of Objects (Grant and Revoke Invoking Predicates on Object to the Subject over Time)
MTRANS: Access Control of Signs over Time
MBUILD: Publisher, Produce Signs over Time
ATTEND: Subscriber, Consume Signs over Time
GRASP: Has_A and Time
PROPEL: Mass Times Acceleration (change in Motion) Wind Physics, Rigid Body Physics
MOVE: Time plus Subject piece/whole Motion
INGEST:  Consume Mass over Time
EXPEL: Produce Mass over Time
SPEAK: Produce Sound Signs over Space-Time
+ Sign Predicates from UML and X3DUOM

What this basically means is we only need about 10-12 predicate sets to describe *everything*, and then by analogical predication, convert all other predicates to those 10-12.  So remember, only 10-12 predicates in X3D semantics folks.

Leftover Predicates (what’s this?)
======================================================================================================
So what are these predicate sets, Grant, Revoke control of Objects (Grant and Revoke Invoking Predicates on Object to the Subject over Time)?  These things control who has access to apply predicates to objects.  Security essentially.   (and we’re not just talking about objects, we’re talking predicates).   Ever try to empty the trash without the empty the trash or del or rm command?  What if you had the empty, del and rm command, and you couldn’t hide your trash can due to ambient authority?  How many times would you try the command? Try to design a security system for every predicate and controlled object out there.  Can you?  A unforgeable key or reference to the combination of the predicate and the object (think opening a door to a room where you can manipulate an object) which cannot be duplicated.  Access may be revoked by creating a second key and lock (add as many different keys and locks as you like to the door), but leaving that lock unlocked until access is denied, then locking it (if your having difficulty, think of a doctors office, with two doors owned by the doctor providing access to the patient (you have to be a patient (key1) and have an appointment (key2) to get through both doors) and one door providing public access, where keys are granted to people by staff.  Locked doors are pretty cheap in a virtual environment, and security guards who check badges are not.   Which do you think does a better job?   These keys are something I’d like to see added to virtual environments more.  They have been added to EMonkeyEngine.   The only security we have is essentially all or nothing in a virtual world right now (do you have the session identifier?).   Keys could be used with the NetworkSensor (yes, unforgeable due to HTTPS sessions) to grant access to the semantic triples in X3D.  If you didn’t want to use keys, you could grant all the keys to your database administrator.

The questions journalists answer and how they are related to predicates.
A question is a query, and the questions journalists (publishers) answer are who?, what? When? Where? Why? and how?  
How?  is a templatized script of actions, or predicates.  So your how queries (how do I do this?) will probably come as a script template.   You can either fill in the script template where predicate sets are tables a where clause (logic), and specify placeholders or variable names (SELECT) to return.   This is a lot like Prolog, where the predicates are part of the query, and parameters are passed to the predicates joined with fuzzy (at what state should I make a change) quantum (binary plus undecided), spatial-temporal (where?  when?), statistical (what is the probability?), predicate, or chaotic operations (what is your forecast?).  For example, you might want to enumerate all the ways to bake an apple pie that will win first prize at the state fair.  Out will pop all matching recipes or scripts.   Thus the computer doesn’t know how to do complex predicate, it just matches a database of scripts against templated predicates, variables, and logic and fills in all the possible variables in as many ways as it can, and produces stored predicates. It takes real live programmers to insert all those scripts into the database (now you know why Microsoft bought GitHub), and why American and Japanese AI didn’t work.   Scripts are programs are recipes are plans of action.  If you goal is answering the question how?   Then the answer is strategy or selecting scripts at runtime, according to the strategy design pattern.  Who creates the scripts in the first place? How do you select your main predicate unless you have a predicate set in mind?  To select an appropriate predicate from your database of scripts, you need predicate inheritance to appropriate display the taxonomy of appropriate predicates for your domain (T-box I think).  Scripts can be generated by the computer if given a starting point, a score for current point and goal (the highest score possible).  How do you score a recipe?   By just giving a score at the end?  Can DeepMind do that?  You can score a program by number of unit tests passed, but who writes the unit test?   That is the strategy behind writing X3D-big-bang…writing unit tests automatically for 3D by selecting extensions to unit tests from an inverted schema.   Can we write unit tests from an (inverted) ontology, object model, or schema?  How do we know we have coverage?

So we now have an IS_A model of predicates.   Can we have has_a and other predicate sets for predicates?  What do you think?

Can we really reduce the number of predicates to 10-12?   That would lay waste to the numerous jargons popping up all over.   See:  https://en.wikipedia.org/wiki/Predicate_(grammar)#Carlson_classes (unrelated) for a start.  How many actual predicate sets might we get from the work and volunteer professions (various X3D groups).

Do behavior or grammar predicates come with a set of rules or facts (thinking Prolog here)?  I know they come with a set of parameters.   Those are not facts or rules. Do the rules and facts help determine the label we assign to the behavior?   Should we have sets of behaviors where the analogical predicates help choose the set?   Is there such a thing as a predicate set?

 [ Node:  The 3rd between Produce and Consume is Mediate  (as I think you mentioned) /Copy/Transform/Translate/Paraphrase/Process/Apply/Evaluate/Exchange/… ]

I wrote the above last, so it makes the most sense. If you’re interested in background, read on. I view the following most as trash, or an old copy.
======================================================================================================
So I was trying to express things related to semantics and meaning today with my wife.  I finally decided to settle on *behavior* or *transformation* related to creating and deriving meaning, leaving the structure to syntax (graphs). This sending/receiving is a sign process, or, "semiotic.”  In particular, it is communication.  There are other semiotics.  One is metaphor.    This document is an attempt to bring the subject of discussion on virtual, systems, military and real world semantics to semiotics and sign systems, and finally, to sign worlds.  I think once we have a sign world, it will be easier to bring that back into X3D, the real world, structural, and systems thinking to achieve analogical predication (or just structural analogy) of worlds.

A geographic map (2D) is a metaphor for a projection (2D) of the earth (4D).  A globe is a 3D metaphor for the earth.  A time-series of metaphors of the earth is an animation of maps or globes.  An atlas is a collection of maps.  I would like to create a time-series atlas online, or find one.  First however, I need to find the metaphors I want to relate about the earth.   In other words, what is the meaning of earth that I want to communicate.  I would like to communicate the places on the earth where problems are being solved and places on the earth where problems are being created.

I would also like to present maps (in the more general sense) and  functions (as a generalization of maps) for relating meaning for VR/AR.   Maps are the primary compound data structure in computer science, and functions are the primary behavior in computer science.  The visual metaphor for functions in math are points, curves, surfaces and volumes in a coordinate system (4D graphics).   The symbolism (another semiotic) for functions in math are equations.   Sometimes functions fail to solve problems in the real world, and we need heuristics.   Heuristics are functions for dealing with a problem imperfectly. Much of communication is sending and receiving imperfect compound meanings. Thus we must derive our Plato’s form of the compound meaning from imperfection (or chaos).  Our grammars, languages and documents are imperfect ways to describe the world, I believe, and need validation and verification of meaning from other people.   In the semantic world, validation and verification is done with description logics.   There are alternative views of the world, including statistic (values from [0,1] which predict events), chaotic (animated values which follow a chaotic attractor), spatio-temporal (vector values), fuzzy (3 2-3 point values which form an angle), quantum (0 or 1 or undecided).  The primary values behind description logics are subject, predicate, object triples. But you can have fuzzy description logics for example. In general, there are quantitative values and qualitative values (combined to form symbolic values).   Currently, I believe, that much of semantics is qualitative, leaving the quantitative behind (to much dismay).  Much of X3D is quantitative, leaving the qualitative behind (to much dismay). Finding a mapping between the two may be a good idea for both (It’s a part of cartography).

Thus the primary task of understanding meaning in X3D is the mapping from entities and relationships in spatio-temporal space to triples in description logics and vica versa.  Or the mapping between the external world and the internal world and vica versa.  Or the mapping between objective and subjective worlds, which is the predicate, verb, behavior or function world.

The primary entity in the systems world is location.   Processors are collections of locations and processors.  Networks connect processors and networks. Objects (a combination of data structures and algorithms or a combination of physical and mental work) consume space and time and are made up of other objects are located at processors. Information flows between objects on networks. Location ties everything into spatio-temporal space.  Processors and networks produce space and time.   This system can be drawn on a geographic map using LOD. Objects are used to solve problems on a processor.  This is the systems world. [ SINAPS and DOSCAPS,NIF]

The primary entity in real world is human.  Corporations are collections of  humans and corporations.  Networks connect corporations and networks.  Products (a combination of matter and energy or a combination of physical and mental work) consume space and time and are made up of other products and are located at corporations.   Products move between corporations on networks.  Humans tie everything into spatio-temporal space.  Corporations and Networks produce space and time.  Products are used to solve problems in a corporation.  This is the physical world. [Work, becoming a people person]

The primary entity in virtual worlds is the shape.   Groups are collections of shapes and groups. Routes connect groups and shapes.  Scenes (a combination of visual and non-visual objects or a combination of physical and mental work) consume space and time and are made up of other scenes).  Transforms guide groups to their location.   Events move between nodes and fields on routes. Shapes are in spatio-temporal space.   The system can be rendered on a browser or canvas.  Shapes are used to make up animations in a scene.   This is the virtual world [X3D-Big-Bang]

So some problems are created and solved in the virtual world, some on systems, and some in the real world.  The atlas should reflect that.   Can someone describe the analogical (structural) predicate behind the 3 worlds.   How does that analogical predicate relate to the predicate behind document, syntax, semantics and semiotics?   How can the complete sign world (beyond semiotics or sign system) be best described?   I can see communicating entities, signs, speak and listen, networks, -- thinking…

The primary entity in the sign world is the sign.   There are collections of signs called semiotics and higher level sign systems.   Social networks connect humans (sorry, leaving out animals, plants and aliens for now) and networks.   Utterances (a combination of visual and non-visual signs, a combination of physical and mental work) consume network space and time, and move between humans on networks. Utterances tie signs to spatio-temporal space.   Utterances can travel through spacetime. [Software Visualization, Data Visualization, Equation Visualization, Translator’s Workbench]

Hmm. Imperfect for now. Anyone have a better heuristic?   This could be better refined I think, and made into a table.  Would it be better to stick to programming languages instead of trying to reach out to other sign systems?

[ scratch area, not really processed yet… How does X3D,systems relate to the sign world?
Shapes:  Humans
Groups:  Corporations
ROUTEs: networks  (visual, auditory, tactile).
Events: Concepts, Light, Sound, Motion (hands on keyboard)
Event Types: Signs, Speech, Phonemes, Morphemes, Words, Sentences, Functions, Maps, …
Lights: Mouth, Hands, Body
Camera: Ear, Eye
Field: Mouth, Ear, Eye

Books in the sign world are related to disks or drives in the systems world.   HTML is in the sign world.   HTTP/REST is a behavior in the sign world.
]

How are X3D events different from signs?   Should we think of a different structure for X3D events?  What is a network event?   Can we think of event types as including those from the semantics area to enhance X3D?  I’d love to be able to send an SFFunction, MFFunction, SFMap, or MFMap (JSON) event. MFNode?

What is this “sign world” similar to sign system or a semiotic and how do you describe it?  How would you map the sign world to a virtual world analogically or meta-analogically?   meta: instructions/words combined in instruction/word collections on networks going between actions, threads and processes in a system?

What is the abstraction of virtual world, system world, augmented world, real world and sign world?   What other worlds should I include?   Isn’t that abstraction, including medical, CAD,… the conceptual semantics we are searching for?   What is the pattern that I’ve created in my mind that I would like to communicate?  Oh, duh, world.   What are the semantics for the world?

Action, Logistic, Tactic, Strategy, Theatre.  How do these fit into X3D? Action is an animation? Logistics is scripting animation? Tactics is planning animation?  Strategy is a Scene idea?  Theater is an interactive set of Scenes?  How do these 5 things fit into the sign world?  Action is sign? Logistics is as line or sentence?  Tactics is structure or syntax?   Strategy is semantics or meaning?  Theater is conversation?  

Thanks,

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20190323/5a874788/attachment-0001.html>