Nmd Logo Ill

AI's Impact on 3d Games and Virtual Environments

Version 1.0

Excerpt playing: (choose from the outline below)

The vocabulary of immersive media

00:00 John Bell introduced by Jon Ippolito

02:04 The evolution of AI buzzwords

03:42 The definition of AR, VR, MR, and XR

XR in teaching and research

04:32 Dartmouth's XR for classrooms

Simulating the ecology of lakes, visualizing periodic table, and reconstructing a virtual basilica in Rome.

05:41 Dartmouth's XR for research

Analyzing body language in film and TV, and using VR to simulate how cave paintings were perceived by their creators.

08:24 The changing costs of immersive media

Labor and expense have dropped dramatically. Creating VR in 1962 (Sensorama) required an ungainly rig to record; now the technology is in every smartphone.

Generative AI and related terms

10:50 Algorithmic art, analytic AI, and derivative AI

Algorithmic art was transparent, whereas today's large language models are inscrutable.

Democratizing character design

13:11 Pre-AI approach

The CIVIT AR volumetric capture studio costs $500,000 with another $10-20k per minute of analyzed video.

16:20 AI approach

Soon players will generate role-playing game avatars from their own bodies with just a smartphone.

Democratizing character animation

17:33 Pre-AI approach

Traditional rigging, keyframing, and tweening (eg, with Maya) is laborious.

18:37 AI approach

Generative AI is faster and can accommodate transition states/edge cases.

Democratizing environment creation

23:44 Pre-AI approach

Laser scanning to create a pointcloud is accurate but labor-intensive.

26:29 AI approach

Neural radiance fields (NeRFs) are "fuzzier" but easier to make.

Unprecedented 3d applications possible with AI

28:51 Nuanced conversations with character bots (NPCs)

30:32 Textured models generated directly in an environment with text prompts (Gepetto, BIBER)

33:13 Democratizing creativity for smaller teams

Just as YouTubers can attract millions of followers without a Hollywood budget, animators and game developers can create works without the hundreds of collaborators seen in end credits for a Pixar movie.

Live demo of AI-generated 3d models

34:48 Instructions for using Luma labs model generator

38:09 Prompt engineering for 3d

39:24 Examples of student work


41:49 What is the difference between XR, AR, VR, Mixed Reality?

43:42 How could these tools be used unethically?

44:47 What's stopping AI from replacing us with digital simulations?

Meta and TikTok haven't figured out how to make money from repurposing 3d user data to model virtual worlds--yet. Re-creating interactive avatars of deceased relatives is a likely near-term application.

47:05 Will smartphones integrate more 3d capture technology?

48:42 Will companies train models on 3d capture of your surroundings?

Yes, they have an incentive to develop content to sell headsets, even if that means a 3d recording of your 3-year-old walking across the kitchen. Meta VR headsets have "inside-out tracking" that captures images, and they are allowed legally to use them internally. This could let Meta make a map of your living room.

50:05 Will AI take over the human race?

In some ways, it already has encroached on the idea space by crafting the messages you get in social media and elsewhere.

This presentation was recorded by the University of Maine's New Media program. For more information, contact ude.eniam@otiloppij.

Timecodes are in minutes seconds

Slide from John Bell presentation

Artificial intelligence has already revolutionized how we generate text and images, but 3d applications from animation to games to augmented and virtual reality are major industries ripe for disruption by new generative AI tools. This presentation includes a talk, demo, and question-and-answer session on the technical possibilities and ethical risks of this emerging field.

Bell John Dartmouth Dev Lab Adjust Vga This presentation features software developer-artist John Bell, whose research focuses on collaborative creativity with a special focus on virtual environments. At Dartmouth Bell is Associate Director of the Data Experiences and Visualizations Studio, Associate Director of the Media Ecology Project, Manager of Dartmouth Research Computingā€˜s Digital Humanities Program, and Lecturer in Film and Media Studies. Bell is also a New Media alumnus who teaches in UMaine's Digital Curation graduate program. The conversation is moderated by Jon Ippolito, Professor of New Media, director of the Digital Curation program, and co-founder of UMaine's larger Learning With AI initiative.

This conversation took place on 15 November 2023 and is sponsored by the University of Maine's New Media program, which teaches animation, digital storytelling, gaming, music, physical computing, video, and web and app development. Learn more about New Media's hands-on webinars.

Watch the entire video or choose an excerpt from the menu on this page.