Guess what? AI’s godmother says today’s AI is a “wordsmith in the dark.”
Dr. Fei-Fei Li, one of the key figures behind the modern AI revolution (she created ImageNet), just dropped a must-read essay that basically calls out the biggest blind spot in today's AI.
She argues that while models like ChatGPT are amazing with words, they’re still "eloquent but inexperienced, knowledgeable but ungrounded." They can describe a room, but they can't navigate it. In the new essay and accompanying X thread, she lays out the roadmap for AI’s next great frontier: Spatial Intelligence.
Here's the core idea in brief: For AI to become truly useful in the physical world—powering robots, accelerating scientific discovery, or creating truly immersive experiences—it needs to understand space, physics, and interaction. Not just be able to describe them, but UNDERSTAND them. This requires a new type of AI called a "world model."
According to Dr. Li, a true world model must have three key capabilities:
- Generative: It can create endless, diverse 3D worlds from scratch that remain physically and geometrically consistent. Think of an AI that doesn't just make a 5-second video clip, but a persistent, explorable level of a video game.
- Multimodal: It can understand and process all kinds of inputs at once—text, images, videos, gestures, and even direct actions—to build a complete picture of the world.
- Interactive: It can predict what happens next when you take an action. If you push a block, it knows the block should fall over and make a sound, not turn into a butterfly.
This isn't just theory, it's already happening. As we covered in our "World Builders" report, the race to build these models is on. Dr. Li's vision is the "why," and these projects are the "how":
- Google's Genie 3 lets you generate and play inside an interactive world from a single prompt.
- Tencent's Hunyuan Gamecraft is trained on AAA games to create high-quality, controllable game environments.
- Open-source models like Matrix-Game 2.0 let anyone generate interactive game worlds trained on footage from titles like Grand Theft Auto V.
- And of course, there's World Labs' own Marble, which lets creators generate explorable 3D worlds from text prompts.
What to do about this: Dr. Li's essay is a signal that the AI industry is shifting its focus from passive content generation (text, images, short videos) to active, persistent, and interactive simulations (that would be one reason why we need $1.4 trillion in new AI compute, for example). For developers, this means the next wave of killer apps might not be another text-based wrapper, but tools that leverage these emerging world models for gaming, design, or robotics training. For creators, it signals the dawn of a new medium where storytelling is about building explorable worlds, not just linear narratives.
The era of AI that only talks is ending; the era of AI that does is just about to be born...
Below, we break down the essay and connect it back to the work being done to build the early spatial intelligent prototypes we've seen to date...
From Words to Worlds...
In 1950, Alan Turing famously asked, "can machines think?" (paper) For over 70 years, that question has driven the field of artificial intelligence (and even inspired the Turing test) Today, with large language models (LLMs) writing poetry, generating code, and answering complex questions, it feels like we’re closer than ever to an affirmative answer. But according to Dr. Fei-Fei Li, a pivotal figure whose work on the ImageNet dataset helped ignite the deep learning revolution, today's AI is still missing a fundamental piece of the puzzle.
In a new, widely-circulated essay titled "From Words to Worlds," Dr. Li argues that while current AI models have mastered language, they remain "wordsmiths in the dark; eloquent but inexperienced, knowledgeable but ungrounded." They can generate a description of a sunset, but they don't understand the physics of light scattering through the atmosphere. They can write code for a robot, but they can't intuitively grasp how that robot should navigate a cluttered room. This gap between abstract knowledge and grounded understanding is what holds AI back from its full potential.
The solution, she posits, is the pursuit of AI's next great frontier: Spatial Intelligence. This is the capability that connects perception to action, imagination to creation, and allows us to reason about the physical world. It’s the silent, intuitive intelligence we use every day to park a car, catch a set of keys, or pour coffee without looking. It's the same intelligence that enabled historical breakthroughs, from Eratosthenes calculating the Earth's circumference using shadows to Watson and Crick physically assembling models to uncover the structure of DNA. Without it, AI remains disconnected from reality, unable to effectively drive our cars, guide robots in our homes, or accelerate discovery in the physical sciences.
The Rise of World Models
To build machines with spatial intelligence, Dr. Li asserts that we need something far more ambitious than LLMs. We need world models—a new class of generative AI designed not just to process sequences of words, but to understand, simulate, and interact with complex, dynamic environments.
This isn't a new concept, but Dr. Li provides a clear and powerful framework for what a true world model must achieve. She defines it through three essential capabilities:
- Generative: A world model must be able to generate endlessly varied and diverse simulated worlds that remain consistent with the laws of physics, geometry, and motion. Unlike current AI-generated videos that often lose coherence after a few seconds, these worlds must be persistent. Whether you’re exploring a fantasy realm or a digital twin of a factory floor, the environment must behave predictably and logically.
- Multimodal: Intelligence is built on processing diverse sensory inputs. A world model must do the same, taking in information from images, videos, text instructions, depth maps, gestures, and user actions. It should be able to construct a complete and holistic understanding of a world from partial information, just as humans do.
- Interactive: This is the critical leap. A world model must be able to predict the future state of the world based on a given action. If a user in a simulation pushes a domino, the model must output a new state where that domino falls and triggers a chain reaction. This perception-action loop is the bedrock of embodied intelligence and the key to training agents that can operate in the real world.
From Theory to Reality
Dr. Li's essay provides the philosophical and scientific "why" for this shift, but the technological "how" is already taking shape at a breathtaking pace. As we recently explored in our report, "The World Builders," an entire category of AI dedicated to creating interactive realities has exploded onto the scene, validating her thesis in real-time.
These emerging models are the first true examples of the world models Dr. Li describes:
- Google's Genie 3 is a prime example of the generative and interactive principles. It can take a single text prompt and generate a fully playable, interactive environment in real-time. It moves beyond passive video generation, allowing users to navigate and act within an AI-created world, even introducing "promptable world events" like a sudden hurricane to alter the simulation on the fly.
- Open-source projects like SkyworkAI's Matrix-Game 2.0 demonstrate the power of specialized training data. By training on 1,200 hours of gameplay from titles like Grand Theft Auto V, it learns the specific physics and interaction dynamics of a game world, allowing users to generate and control their own interactive scenarios at a smooth 25 frames per second.
- Tencent's Hunyuan Gamecraft, trained on over 100 AAA games, focuses on replicating the high-fidelity look and feel of commercial video games, unifying complex keyboard and mouse inputs into a seamless camera control system.
These systems are made possible by foundational breakthroughs like Self-Forcing, a training technique that teaches the AI to build new frames based on its own, slightly imperfect previous frames. This forces the model to become self-correcting, solving the problem of long-term consistency and enabling the real-time performance necessary for interaction.
What This Means for the Future
The convergence of Dr. Li's vision and the emergence of these powerful world models signals a fundamental shift in the trajectory of AI development. The first wave of generative AI gave us tools that could assist with abstract, knowledge-based tasks. This next wave will give us tools that can engage with the physical and simulated world.
The applications, as Dr. Li outlines, span near-term, mid-term, and long-term horizons:
- Creativity (Now): The most immediate impact will be on storytelling and design. Tools like World Labs' Marble, which Dr. Li co-founded, are already putting these capabilities into the hands of filmmakers, game designers, and architects. They can now rapidly prototype and explore fully realized 3D worlds, transforming the creative process from a slow, technical endeavor to a fluid act of imagination.
- Robotics (Mid-Term): World models are the key to unlocking embodied AI. The ability to generate infinite, varied, and physically accurate simulations provides the perfect training ground for robots. An agent can learn to navigate millions of different cluttered rooms or drive in countless hazardous weather scenarios in simulation before ever being deployed in the real world, dramatically accelerating progress while ensuring safety.
- Science and Healthcare (Long-Term): The most profound impact may be in scientific discovery. Spatially intelligent systems can simulate molecular interactions to accelerate drug discovery, model complex climate systems, or help surgeons practice complex procedures in realistic virtual environments. This is AI not as a replacement for human expertise, but as a powerful amplifier for human intellect and ingenuity.
Dr. Li concludes her essay by returning to her North Star: that AI must be developed to augment human capability. Spatial intelligence is the ultimate expression of this vision. It is the technology that will allow AI to step out of the abstract world of text and into our physical reality—not as an alien intelligence, but as a true partner in building a better world. The quest has just begun.
