A talk on world models
AI World Models
Can a machine imagine the future?
Opening question
Are you reacting to the present — or acting on a predicted future?
Catching a ball, crossing a street, pouring water, playing chess — intelligent action happens before the outcome arrives.
Roadmap
Where we're going
Part 1 of 6
What is a world model?
From hand-written physics to a model that learns how the world responds to actions.
A working definition
A world model is an internal predictive model of how an environment changes — especially in response to an agent's actions.
The minimum idea
Current state + Action → Possible next state
Click an action and watch how the same state + a different action produces a different result.
Prediction
What happens next?
Forecasts one timeline, projected from the past.
World-model reasoning
What happens if I do this?
The litmus test: if the future doesn't change when the action changes, it isn't a world model.
Part 2 of 6
What's inside:
memory & representation
What the agent sees isn't the state. It must compress, remember, and infer.
Partial observability
Out of sight isn't gone.
What the camera sees isn't the whole state of the world. A world model must keep an internal picture through occlusion and missing information.
Learning from experience
Observe → Act → See result → Learn
One piece of experience = what I saw · what I did · what happened next.
Baby version: try an action, get surprised, update the model.
Part 3 of 6
From prediction
to planning
Imagine many futures, score them, act — then observe and replan.
Uncertainty
The future is usually not one answer, but a set of possibilities.
In plain words: how likely each possible next moment is — given where things are and what you do.
A pedestrian at a corner might turn left, turn right, or stop. A good model predicts — and owns its uncertainty.
Rolling the model forward
Imagine further by running the model on its own predictions.
Like replaying a chess game in your head: a few moves ahead is sharp — twenty moves ahead, you're mostly guessing. Each step leans on the last, so small errors compound.
Live interaction
Same state.
Which action wins?
Goal: land the blue ball in the green zone without it falling off. Predict first, then reveal.
Receding horizon
Plan several steps.
Execute one.
Observe again.
Replan.
The agent need not commit to the whole distant future at once. It keeps updating its own plan.
End-to-end example
Let's watch one robot think it through.
Part 4 of 6
Where it fits —
and what counts
World model, policy, simulator, agent — and is an LLM one?
Environment / Simulator
decides what really happens
Agent
Concept boundaries
Generating a plausible future is not the same as having a reliable world model.
State of the art · 2025–26
This isn't sci-fi any more.
Google DeepMind
Genie 3
Type a sentence → a 3D world you can walk around in, in real time.
OpenAI
Sora 2
Generates video that mostly obeys real physics — "video as a world simulator."
Meta · Yann LeCun
V-JEPA 2
Learns the world from raw video, then drives a real robot arm to pick unseen objects.
NVIDIA
Cosmos
"World foundation models" that simulate the physical world to train robots & cars.
Wayve
GAIA
Imagines rare, dangerous driving scenes so self-driving cars can practice safely.
Scan any code, or click a card on the presenter laptop to open the demo.
See it move
Watch a world model dream up a place.
Genie 3 builds a world from a prompt and keeps it consistent as you move through it — exactly the "remember what you can't currently see" idea, at scale.
deepmind.google/models/genie
Drop a clip at assets/demo.mp4 to play it right here —
or scan the code to watch the real Genie 3 demo.
Your turn
Now you play a world model.
Decart · free
Oasis
Play AI-Minecraft — every frame is dreamed up live, with no game engine.
Microsoft Muse · free
AI Quake II
Walk through a shooter that no engine ever rendered.
Odyssey
Explorer
Step into a streamed, interactive 3D world you can move around in.
World Labs
Marble
Turn a sentence or a photo into a 3D world you can keep and explore.
Best on a laptop (Chrome) · click any card to open one live on stage.
Part 5 of 6
Why it's
still hard
Plausible isn't correct. Errors compound. How do we even evaluate it?
Model drift
Each step is only a little wrong — far ahead, it can leave reality entirely.
Short-term prediction can be good while a long rollout drifts off — which is why your GPS keeps recalculating. Better to glance at reality often than trust one long guess.
Evaluation
What makes a world model actually good?
One-step accuracy
Is the next-step prediction correct?
Multi-step consistency
Is a long rollout stable?
Counterfactual accuracy
When the action changes, does the prediction change sensibly?
Planning improvement
With the model, do real actions get better?
Part 6 of 6
Testing it:
Voyage
A social sandbox to probe whether an agent's world model is actually useful.
Our project
Voyage isn't a world model — but it can test whether an agent has a useful one.
decides real outcomes
chooses actions
predicts consequences
Counterfactual test
NPC trust: 72
A world model does not give AI a crystal ball.
It gives AI a sandbox.
Thank you
Questions?
AI World Models — does AI get a crystal ball, or a sandbox?