Materializing Imagination
through Prompting the Physical World
Materializing Imagination through Prompting the Physical World
November 2, 2026 · Detroit Marriott at the Renaissance Center
This workshop explores how generative AI can reshape the way people create in XR. While it is rapidly lowering barriers to creation, XR introduces fundamentally different challenges as creation moves beyond flat screens into the physical world. XR experiences must account for real spaces, embodied and multimodal interaction, collaboration across heterogeneous devices, and content that unfolds through space and time. These characteristics raise new questions about how creators express intent, preview and refine imagined outcomes, evaluate generated content, and share or transform what they create. The World as a Canvas brings together researchers and practitioners to examine these emerging challenges and develop a shared vision for more accessible, expressive, and collaborative XR creation.
Each challenge framed open questions we refined together and carried into the afternoon prototyping session.
How can generative AI create content and behaviors that fit the user’s physical surroundings, from lighting and scale to material and weight?
How can tools help creators preview imagined outcomes without constraining their own mental imagery?
How should generative AI interpret creative intent expressed through gesture, gaze, movement, and other signals beyond words?
How can people co-create seamlessly across headsets, tablets, and phones with different input, display, and compute capabilities?
How can creators detect and correct generative errors that emerge through interaction, across viewpoints, or over time?
As creations move across images, videos, XR, and physical artifacts, what should be preserved, and how can their provenance and ownership remain traceable?
Keynote I
KPKeynote II
AWMore photos! If you have photos of your own, please add them.
Anything you would like to know about the workshop, just ask.
UIST '26 · November 2, 2026 · Detroit, MI