[x3d-public] Prompt Engineering and X3D

[John Carlson]

Vince,

I’m not sure where to start this conversation.  Perhaps it’s worth a
discussion how Large Language Models (LLM) fit into X3D semantics, and how
we might do embedding of X3D (like Word2Vec, GloVe, Object2Vec, ChatGPT,
Bing, etc).

Essentially, how can “search engines” be queried with prompts in many
languages to produce X3D scenes, possibly with Blender code or better.

There’s are already great strides in building images, animations and
webpages with LLM.   NeRF is already building meshes with point clouds.
Recent research includes Text-to-4D (MAV3D), which includes not only a
moving mesh, but also a moving camera, and soon, multiple meshes might be
feasible.

Beyond simple scenes, how might entire worlds be constructed (No Man’s Sky)?

Indeed, one might ask if our SAI code examples and encodings might be fed
into something like GitHub Copilot or OpenAI Codex/InstructGPT.

Perhaps we could plan a discussion in the Scanning and Printing working
group?  Something like NeRF (Neural Radiance Fields) might be a good
starting point?

John

On Thu, Feb 16, 2023 at 7:31 PM Brutzman, Donald (Don) (CIV) <
brutzman at nps.edu> wrote:

> Nicholas will host our regular weekly meeting this week.
>
>
>
> The X3D Working Group meets weekly on Fridays 09-1000 Pacific on Web3D
> Consortium zoom channel.  Telcon information:
>
>    -
>    https://us02web.zoom.us/j/81634670698?pwd=a1VPeU5tN01rc21Oa3hScUlHK0Rxdz09
>    - https://zoom.us/j/148206572  Password 483805
>    - https://www.web3d.org/member/teleconference-information
>
>
>
>    1. *X3D 2023 goals.  *Our primary activities for X3D Working Group in
>    2023 are focused on broad and correct deployment.
>
>
>
>    1. Encourage consistent rendering, interaction and usage for the many
>    tremendous capabilities in X3D4.
>    2. Update ISO specifications and implementations for multiple
>    programming languages and file encodings to match X3D4.
>
>
>
>    1. *Focus: X3D 4.1 feature discussion.*
>
>
>
> Many possibilities!  The “X” in X3D is Extensible, so pursuit of further
> evolution is an essential part of our design.  We benefit from (but are not
> limited by) ISO stability, since implementers can experiment with
> innovative practices and also match Web3D Consortium recommended practices.
>
>
>
>    1. Near term: resolving Sound component differences with Web Audio
>       API.  Some noted issues will likely get deferred as X3D 4.1 issues.
>       2. In progress: HAnim2/X3D4 additions for site names, poses, and
>       clothing to provide support for apparel.
>       3. Expected: glTF 2.0 is normatively matched in X3D 4.0
>       specification.  Emerging glTF extensions are candidates for group adoption
>       and X3D 4.1 standardization.
>       4. What else? What’s next?
>
>
>
> Wishing best progress.  Have fun with X3D!  8)
>
>
>
> all the best, Don
>
> --
>
> Don Brutzman  Naval Postgraduate School, Code USW/Br
> brutzman at nps.edu
>
> Watkins 270,  MOVES Institute, Monterey CA 93943-5000 USA
> +1.831.656.2149
>
> X3D graphics, virtual worlds, Navy robotics https://
> faculty.nps.edu/brutzman
>
>
> _______________________________________________
> x3d-public mailing list
> x3d-public at web3d.org
> http://web3d.org/mailman/listinfo/x3d-public_web3d.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://web3d.org/pipermail/x3d-public_web3d.org/attachments/20230216/3e043475/attachment.html>