Google’s Gemini co-leads explain what comes next

Google’s most interesting Gemini interview from I/O was framed as a look back at the project’s origins.

The stronger read is that it was a roadmap.

In a Google for Developers conversation, Logan Kilpatrick sat down with Jeff Dean, Koray Kavukcuoglu, Noam Shazeer, and Oriol Vinyals to talk about where Gemini came from, why Google forced separate AI efforts into one model program, and what the team expects to matter next.

The big idea: Gemini is Google’s bet that the future of AI looks less like many separate models bolted onto products and more like one increasingly general intelligence layer powering everything from Search to coding agents to video generation to scientific work.

That connects directly to two recent Neuron pieces. In our verification ladder piece, we argued that AI progress moves fastest where work can be framed, tested, and verified. In our Transformer vs. Post-Transformer debate, we argued that the next architecture fight comes down to scaling, memory, reasoning, hardware, and continual learning.

Google’s Gemini leaders basically walked into the middle of both arguments. Watch the video below.

First up, The TL;DR

Here’s what Google’s Gemini co-leads seemed most focused on next:

• Self-learning and self-improvement: Koray predicted that by next year Google will probably be able to point to a significant part of Gemini that was generated by models and agents working under human guidance. (32:11)
• Continual learning: Oriol wants models to improve through experience and interaction without always needing weight updates, using something like a highly reliable knowledge-base update mechanism. (33:27)
• Long-running agents: Logan’s wish-list item was an agent that can run for something like 30 days of work, though the group immediately noted that better hardware and tool latency would need to make that work finish faster. (34:55)
• Faster tools around agents: Noam predicted that agents will reveal how slow today’s software tools are because those tools were designed for human-paced work, not model-speed work. (35:56)
• World models: Koray framed Gemini Omni as a shift from mostly understanding and text output toward a model that can simulate dynamics, physics, visuals, sounds, and future states. (10:48)
• Richer modalities: Jeff pushed “multimodal” beyond text, image, audio, and video into genomic sequences, chemical structures, robotic grasping data, and LiDAR data. (12:32)
• Better evaluation: Oriol called evaluation surprisingly hard because the field moved from tidy tables in papers to users, feedback, contamination problems, and real-world capability gaps. (26:27)
• More data-efficient learning: Jeff said human learning is roughly a thousand times more data-efficient than LLM learning, so one major research target is extracting far more information from every example or token. (25:21)
• More organic model architectures: Noam said he expected more progress on continual learning and less structured, more fluid architectures. He still finds that direction interesting, even though the current approach keeps working. (23:16)

That is the IO 2027 preview hiding inside the conversation: more self-improving research loops, more memory, longer-running agents, faster hardware, better world models, and maybe a product world where the model becomes the main product layer.

What Google wants Gemini to become

The immediate release context was Gemini 3.5 Flash, which Oriol described as a powerful new series focused especially on coding while preserving and improving the rest of Gemini’s capabilities. (1:15)

That coding emphasis matters because the Gemini leaders kept returning to the same point: coding and agents now define how many people experience AI.

At one level, that means Gemini 3.5 Flash has to be fast, cheap, useful, and strong enough for everyday developer work. At another level, coding is becoming the testbed for AI systems that can plan, act, check, revise, and improve.

That is why Flash matters beyond the product name. Oriol said one positive surprise was that Google keeps packing the intelligence of the previous Pro generation back into the next Flash generation. (19:28)

Noam’s explanation was wonderfully simple: distillation is still basically a teacher-student process. You have a really good teacher model, you train a smaller student model from it, and the “basic spirit” remains close to the original distillation paper. (20:29)

Then Koray gave the version any normal person can remember: squeezing a lemon, putting the good juice into a smaller glass. Finally, model compression explained by someone who has seen a kitchen. (21:04)

This is where the verification ladder starts to matter. Coding improves quickly because the feedback loop is clean. Code runs, tests pass, errors surface, and the model gets signal. Google’s own research workflows are now close enough to that loop that Koray predicted models will increasingly help improve Gemini itself.

The model becomes a worker inside the model factory.

The problems Google still wants to solve

The most revealing part of the interview came when Logan asked what progress had been slower than expected.

Noam pointed to continual learning and more organic model architectures. The current systems use many experts, but their structures remain highly organized and similar. He expected a more flexible, plastic, fluid architecture to make more progress by now. (23:16)

Jeff gave the broader version: AI still cannot solve problems like “invent me a cure for cancer” on demand. That sounds obvious, but it clarifies the gap between impressive model behavior and true scientific agency. (23:43)

Koray focused on capacity. He said current models are not dramatically bigger than those from three or four years ago, yet Google keeps packing more capability into them. His read is that there is still a lot of room in the models, and algorithmic innovation can extract much more from them. (24:29)

Jeff then put a sharper number on the research gap: human learning may be around a thousand times more efficient than LLM learning. A human hears roughly a billion words in a lifetime, while models train on trillions and trillions of tokens. (25:21)

Oriol named evaluation as another underappreciated bottleneck. The field has to evaluate capabilities in isolation, identify which future capabilities matter, prevent test leakage, and make sure users agree with the score. (26:27)

This maps almost perfectly to our earlier verification ladder. The easy benchmarks are the clean ones. The next frontier is turning messier work into reliable feedback without tricking ourselves into optimizing a fake scoreboard.

Or put more simply: Google can keep making the models smarter, but the harder question is whether the company can keep creating better answer keys.

What could show up by IO 2027

When Logan asked for predictions, Koray went straight to self-learning.

His forecast: as models become more agentic and better at writing code, Google will use them more in its own research. By next year, he expects Google will be on the path of relying on models to improve parts of Gemini, and probably talking about it publicly. (32:11)

Jeff sharpened that into a concrete prediction: Google will probably be able to point to “some very significant thing” in its models that was generated by models and agents working under human guidance. (32:55)

That is a huge sentence.

It does not mean Gemini will suddenly wake up and recursively rewrite itself into a sci-fi singularity. It means frontier AI research is beginning to look like a closed loop:

• Humans define the research direction.
• Models propose experiments or code changes.
• Agents run some of the work.
• Tests and evaluations grade the result.
• Successful changes feed the next model cycle.

That is also exactly why coding matters so much. Coding is one of the highest-leverage places where this loop can close because feedback is fast, automated, and objective enough to compound.

Oriol’s prediction was more about memory. He wants models that improve through experience and interactions without needing full weight updates every time. He described something like a knowledge-base update that works so well it becomes an obvious capability everyone uses. (33:27)

That points to a softer version of continual learning: the model may not rewrite its brain every second, but the system around it remembers more, retrieves better, updates more safely, and uses personal or task-specific context without producing bizarre irrelevant callbacks.

Anyone who has watched a chatbot drag a friend’s birthday party into a totally unrelated work question knows exactly why this is still unsolved. Logan mentioned that failure mode directly. (34:06)

Then came the long-running agent dream. Logan said a model that could run for 30 days before I/O would surprise a lot of people. Jeff immediately noted that this would require memory systems, continual learning, and better hardware because it would cost “a zillion tokens.” (34:55)

Noam added another constraint: if the agent ran for 30 days, you would rather better hardware make it finish in one day. (35:43)

Then he offered one of the best practical predictions in the whole interview: agents will expose how slow all our tools are. A model can become infinitely fast and still waste most of its time waiting on software designed for human latency. (35:56)

This is the boring bottleneck that becomes exciting once you notice it. The agent future depends on models, memory, hardware, and evaluation, but it also depends on every work tool learning to operate at machine tempo.

Why Gemini became one model in the first place

The historical origin of Gemini makes the future roadmap easier to understand.

Jeff said that before Gemini, people across Google and DeepMind were already thinking about very general models. Oriol was leading efforts at DeepMind. Jeff was helping steer efforts around Pathways, PaLM, PaLM 2, and related projects. (5:43)

Then Jeff wrote what he described as a half-page memo: fragmenting research teams and compute was silly. The company needed to come together and build a single model. (6:52)

That was where the name Gemini came from: the twins. (6:00)

The decision was organizationally hard. Google had teams in London and California, eight hours apart, with separate research cultures and separate efforts. Jeff’s argument was that the best ideas and the best compute needed to stop living in parallel silos. (6:59)

Koray gave the systems version of the same story. A decade earlier, AI research was more academic and exploratory. Once frontier models became major operations, the winning structure became a focused organization with researchers, infrastructure teams, data teams, and product teams pushing one system forward. (7:40)

This is the part of the story that sounds least technical and explains the most.

Gemini was not simply a model family. It was a decision about how a frontier lab should be organized. One model meant one place to concentrate compute, data, talent, infrastructure, product feedback, and research bets.

That explains why Google cares so much about Flash, Omni, agents, Search, coding, glasses, and future hardware in one conversation. They are outlets for the same underlying engine.

The model and the product now shape each other

Logan asked whether the product story was obviously important from the beginning.

Jeff said yes. Search had already taught Google that usage creates lessons. If lots of people use a system, you learn what works, what fails, what people need, and where to improve. AI models should be no different. (3:23)

Koray pushed the point harder: you do not want to build intelligence in a black box. You want people using it. The frontier is both technical capability and the next thing that capability enables for users. (4:20)

That sentence explains a major difference between lab benchmarks and deployed AI.

A benchmark tells you whether the model can climb a known frame. Product usage tells you which frames matter, where the model breaks, what users actually try, and which failures are expensive enough to fix first.

Noam later connected this to Google’s old “one box” philosophy. Search looked like one box, even though many separate systems powered it underneath: sports scores, stock quotes, spelling correction, and other specialized backends. (21:39)

Now, he said, Google has finally built the general-purpose backend for that one-box front end. (22:21)

That is the strategic dream: one model, many product surfaces.

Koray was careful about the user side, though. People still consume information and accomplish tasks in different ways. Search may become a more magical box, but people will still want to reach information, learn from it, and choose different product contexts. (37:22)

Oriol added the human-factor version: he likes separation of concerns. Checking a calendar, buying something, or managing email may remain distinct actions because people want focus, even if one model could technically power all of them. (38:30)

That creates a useful counterweight to the “one model, one product” dream. The intelligence layer may unify. The user experience may stay plural.

World models are the biggest technical clue

The most future-facing technical section of the interview was the discussion of Gemini Omni and world models.

Jeff traced part of Gemini’s ancestry back to Pathways, which explored a single model that could do many things, handle many modalities, and activate different pieces for different tasks. He said those three ideas are represented in Gemini today. (9:17)

Then he pointed to Omni as a new level of multimodality because the model can generate video and use reasoning capabilities across inputs and outputs. (9:42)

Koray explained the “world model” framing: a world model understands dynamics, physics, visuals, and simulation. It can roll a situation forward, then base decisions on those future simulations. (10:48)

In simple terms, a chatbot predicts text. A world model starts to predict how things change.

That matters for video generation, robotics, embodied agents, and science. If a model can simulate a scene forward, keep objects consistent, understand physical constraints, and revise based on future states, it becomes more useful than a model that only describes what it sees.

Oriol said Google is seeing capabilities emerge from scale and mixed data that previously seemed almost impossible. In older video systems, researchers had to manually think through consistency, object disappearance, camera changes, and other visual problems. Now those capabilities are starting to emerge from training recipes at scale. (11:42)

Jeff then widened multimodality beyond the usual human senses. The model should understand scientific and physical data: genomic sequences, chemical structures, robotic grasping data, and LiDAR data. Exposing the model to some of that data can make it better when it later encounters more. (12:32)

This is where the Gemini conversation meets the architecture debate. The Transformer side wins today because the stack scales and absorbs capabilities. The Post-Transformer side keeps asking whether memory, state, continual learning, and physical dynamics need deeper architectural changes.

Google’s answer, at least from this conversation, seems pragmatic: keep scaling the unified model, keep adding modalities, keep improving the recipes, and let the system absorb what works.

The quiet human story behind the machine

The middle of the interview turned into a mini oral history of Google AI.

Jeff called Noam in 2000 to convince him to accept a Google offer. Noam later became Jeff’s office mate and joked that his assigned mentor seemed to know everything because the mentor was Jeff, who had written much of the codebase. (13:27)

Jeff also helped recruit Oriol around 2012, when the Google Brain team was crammed into a small office near the Googleplex patio. (14:59)

Oriol and Jeff worked on distillation at scale, inspired by Geoffrey Hinton’s earlier work. Jeff described training a 50-model ensemble on 300M images, grouping categories by model specialty, then transferring that knowledge into a single model that was more accurate than one trained on raw data. (16:20)

Koray’s first deep interaction with Jeff came during Google’s DeepMind acquisition discussions. Jeff asked to see the code, Koray walked him through directories and files, and the moment became their first code review together. (17:16)

These stories could sound like nostalgia. They explain why Gemini was possible.

The “one model” decision required people who already trusted each other enough to smash teams together, share compute, merge cultures, and keep arguing through results. Koray later described Gemini as a whole technology transformation involving hardware, model design, product, infrastructure, agents, and research. (29:59)

That is the unglamorous part of frontier AI. Model progress depends on architecture and compute, but also on whether the organization can make thousands of decisions without fragmenting itself.

What people may get wrong about this interview

The easy read is “Google is hyping Gemini.”

The better read is that Google’s leaders described several bottlenecks they have not solved yet.

They want models that learn from experience more safely. They want architectures that get more out of each token. They want evaluations that survive contamination and reflect real user value. They want long-running agents that can work without burning oceans of tokens. They want hardware that makes agentic work lower latency. They want product surfaces that preserve human focus while one model powers more of the stack.

They also described a credible counter-narrative to the one-model dream: products may remain separate because humans use tools through intention, context, and habit. A single model can power many surfaces, but users may still prefer different rooms for different tasks.

The second counter-narrative comes from the architecture debate. Google’s current path assumes the unified model keeps absorbing capabilities. The challenger view says some future capability, like continual learning, compact latent reasoning, or dynamic memory, may require a more fundamental redesign.

The honest answer is that both paths can coexist for a while. The frontier stack may keep the Transformer lineage while quietly absorbing state, memory, retrieval, tool use, world modeling, sparsity, recurrence-like behavior, and agent loops.

At some point, the label matters less than the curve.

What to watch next

By the next I/O cycle, watch for five signals:

• Self-improvement receipts: Can Google point to a concrete Gemini improvement generated by model-driven research or agentic experimentation?
• Memory that behaves: Does Gemini get better at using personal, project, and organizational context without pulling in irrelevant details?
• Longer autonomous work: Can agents move from minutes and hours toward multi-day projects with fewer resets?
• World-model evidence: Do Omni-style systems show stronger simulation, physical consistency, robotics transfer, or scientific usefulness?
• Evaluation upgrades: Does Google create better tests for agentic coding, continual learning, long-context learning, and real user value?

The biggest unresolved issue is verification. Self-improving models sound impressive, but the important question is how Google will know which self-generated changes actually make Gemini better, safer, cheaper, and more useful in the real world.

That is where this interview connects back to the larger AI story. Frontier progress is moving from raw model capability into closed-loop systems: models that act, observe, test, remember, revise, and eventually help build their own successors.

Google’s Gemini origin story was about bringing scattered teams together to build one model.

The next chapter is about whether that one model can help build the next version of itself.

Timecoded insight appendix

The roadmap hiding in the interview

• (1:01) Logan frames the moment around the launch of the Gemini 3.5 era, starting with Flash.
• (1:15) Oriol says Gemini began in 2023 and has kept building on multimodal, tool-use, and agentic foundations.
• (1:35) Oriol says Gemini 3.5 Flash is especially focused on coding while preserving and improving the rest of the model’s capabilities.
• (1:50) Oriol says coding capabilities and agentic experiences are now defining how people experience AI.
• (2:14) Noam says the most exciting model release pressure is now internal daily use: what engineers and researchers will use tomorrow.
• (19:28) Oriol says he did not expect Google to keep packing previous Pro-level intelligence into the next Flash generation.
• (20:29) Noam says modern distillation still follows the teacher-student spirit of the original technique, with modest tweaks.
• (21:04) Koray gives the lemon metaphor: squeeze the intelligence from a large model into a smaller glass.
• (32:11) Koray predicts Google will increasingly use models in Gemini research and may talk about that path next year.
• (32:55) Jeff predicts Google may point to a significant model improvement generated by models and agents under human guidance.
• (33:27) Oriol wants continual learning through experience and interactions, potentially via knowledge-base updates rather than constant weight updates.
• (34:55) Logan says a model running autonomously for 30 days before I/O would surprise people.
• (35:27) Jeff says long-running agents require memory systems, continual learning, and better hardware because the token cost would be enormous.
• (35:56) Noam predicts agents will stress the fact that many existing tools are too slow.

What Google still thinks is hard

• (23:16) Noam expected more progress on continual learning and more organic, less structured model architectures.
• (23:43) Jeff says AI still cannot simply invent a cure for cancer on request.
• (24:04) Logan says merging capabilities into one model takes much more energy and effort than people may assume.
• (24:29) Koray says models still have enormous unused capacity, and algorithmic innovations can get more out of them.
• (25:21) Jeff says the field needs algorithms that get far more information out of each data example or token.
• (26:27) Oriol says evaluation has been surprisingly hard and underappreciated.
• (27:26) Jeff says the dream is systems that generalize to things they have never confronted before.

Gemini’s origin story

• (0:00) Jeff says Gemini began from the realization that Google and DeepMind were fragmenting people, ideas, and compute.
• (0:23) Jeff says the name Gemini came from bringing the “twins” together to work on one model.
• (3:23) Jeff says broad product usage gives the team lessons about what works and what needs improvement.
• (4:20) Koray says Google did not want to build intelligence in a black box; products and model capability define the frontier together.
• (4:54) Koray says one more powerful model could leapfrog many separate ML systems already used across Google.
• (6:52) Jeff says he wrote a half-page memo arguing that fragmented teams and fragmented compute were silly.
• (6:59) Jeff says the main problems were fragmented ideas and fragmented compute across teams.
• (7:40) Koray says AI research shifted from academic exploration toward focused operations requiring many researchers to solve many problems together.
• (8:52) Noam says one big language model can do many things, requiring many people, compute, infrastructure teams, and data teams.
• (9:17) Jeff says Pathways already aimed at a single large sparse multimodal model that could do many things.

World models, products, and the one-box idea

• (9:42) Jeff says Omni adds multimodal capabilities including video generation and editing with reasoning across modalities.
• (10:48) Koray says world modeling means understanding dynamics, physics, visuals, simulation, and future states.
• (11:20) Koray says Gemini Omni is a different category because it turns understanding and video modeling into a jointly trained world model.
• (11:42) Oriol says consistency in complex video scenes is emerging through scale and better data mixing rather than manual fixes.
• (12:32) Jeff says multimodal systems should understand scientific data like genomics, chemical structures, robotic grasping, and LiDAR.
• (21:39) Noam compares Gemini to Google’s original “one box” search philosophy.
• (22:21) Noam says Google finally built the general-purpose AI backend for the one-box interface.
• (36:42) Koray answers that Google may have “one” product: the model.
• (37:02) Jeff says Search can generate custom little apps, visualizations, and code inside the experience.
• (37:22) Koray says Search will remain fundamental because people want to reach and consume information for themselves.
• (38:16) Jeff says distinct products like AI glasses and Search can share the same model improvements.
• (39:33) Jeff points toward future physical products that move atoms, not only bits.

The people and research culture behind Gemini

• (13:27) Jeff says he reviewed early Google engineering resumes and recruited Noam in 2000.
• (14:24) Noam says Jeff became his office mate and mentor, and seemed to know everything because he had written much of the codebase.
• (14:59) Jeff says he recruited Oriol around 2012 to join the early Google Brain team.
• (15:45) Oriol recalls working with Jeff on distillation and KL divergence before coding agents existed.
• (16:20) Jeff describes scaling distillation from Hinton’s small MNIST work to a 50-model ensemble over 300M images.
• (17:16) Koray recalls Jeff asking to see DeepMind’s code during acquisition discussions.
• (18:37) Jeff recalls seeing 13 consecutive 30-minute DeepMind talks before asking to inspect the code.
• (28:41) Koray says Jeff has always liked the idea of something more flexible, plastic, and fluid, even though current systems won empirically.
• (29:39) Jeff says Gemini is data-driven: many people run small-scale experiments, examine results, and combine promising ideas.
• (29:59) Koray says Gemini pulls together hardware, model design, product, and research, with leaders focusing on different parts of the transformation.