DeepMind Founder Interview: AGI Architecture, Agent Status, and Scientific Breakthroughs in the Next Decade

Original Video Title: Demis Hassabis: Agents, AGI & The Next Big Scientific Breakthrough

Original Source: Y Combinator
Original Compilation: Deep Tide TechFlow

Editorial Introduction

Google DeepMind CEO, Nobel Prize in Chemistry winner Demis Hassabis visits Y Combinator, discussing key advancements toward AGI, offering advice to entrepreneurs on how to stay ahead, and where the next major scientific breakthrough might occur.

A very practical judgment for deep tech entrepreneurs is that if you start a ten-year deep tech project today, you must include the emergence of AGI in your planning. Additionally, he revealed that Isomorphic Labs (a biotech AI company spun off from DeepMind) will have major news soon.

Highlights Quotes

AGI Roadmap and Timeline

· “Almost certainly, these existing technological components will become part of the final AGI architecture.”

· “Problems like continual learning, long-term reasoning, and certain aspects of memory haven’t been solved yet; AGI needs to get all of these right.”

· “If your AGI timeline is around 2030, like mine, and you start a deep tech project today, you must consider that AGI might appear halfway through.”

Memory and Context Windows

· “The context window roughly corresponds to working memory. Humans have an average of about seven items in working memory, while we have hundreds of thousands or even millions of tokens in our context window. But the problem is, we stuff everything in—including unimportant or incorrect information, which is quite crude.”

· “Processing real-time video streams and storing all tokens would mean a million tokens are only enough for about 20 minutes.”

Limitations of Reasoning

· “I like to test Gemini by playing chess. Sometimes it realizes it’s making a bad move but can’t find a better one, so it circles around and ends up making that bad move. A precise reasoning system shouldn’t have this kind of issue.”

· “On one hand, it can solve IMO gold medal-level problems; on the other, ask it differently and it makes elementary math mistakes. It seems to lack something in self-reflection on its own thinking process.”

Agent and Creativity

· “To achieve AGI, you need a system that can proactively solve problems for you. Agents are the way forward, and I think we’re just getting started.”

· “I haven’t seen anyone use vibe coding to create a top-ranked AAA game. With the current effort, it should be possible, but it hasn’t happened yet. It indicates something is missing in tools or processes.”

Distillation and Small Models

· “Our hypothesis is that a cutting-edge Pro model released in half a year to a year can have its capabilities compressed into a very small model that runs on edge devices. We haven’t yet hit the theoretical information density limit.”

Scientific Discovery and the “Einstein Test”

· “Sometimes I call it the ‘Einstein Test’—can you train a system with knowledge from 1901 and have it independently derive Einstein’s 1905 results, including special relativity? If it can, these systems are not far from inventing truly new things.”

· “Solving a Millennium Prize problem is already impressive. But even more challenging is whether it can propose a new set of Millennium Prize problems, considered equally profound and worth a lifetime of research by top mathematicians.”

Deep Tech Entrepreneurship Advice

· “Chasing hard problems and simple problems are quite similar, just approached differently. Life is short; better to focus your energy on things that no one else will do if you don’t.”

Pathways to AGI

Gary Tan: You’ve been thinking about AGI longer than almost anyone. Based on current paradigms, how much of the final AGI architecture do you think we already have? What is fundamentally missing right now?

Demis Hassabis: Large-scale pretraining, RLHF, thought chains—I’m quite sure these will be part of the final AGI architecture. These techniques have proven a lot of things so far. I can’t imagine in two years we’ll find they’re dead ends—that doesn’t make sense to me. But on top of what we have, maybe one or two more things are needed. Continual learning, long-term reasoning, certain aspects of memory—some problems remain unsolved.

AGI needs to get everything right. Maybe existing tech plus some incremental innovations can extend to that level, but there might still be one or two critical breakthroughs needed. I’d say the probability of unresolved key issues is about fifty-fifty. So at DeepMind, we’re pushing both lines.

Gary Tan: I deal with many agent systems, and what shocks me most is that the underlying weights are often the same across different runs. So the concept of continual learning is very interesting because right now, we’re basically patching things together with tape, like those “dream cycle” ideas.

Demis Hassabis: Exactly, those dream cycles are pretty cool. We’ve thought about this in the context of integrating episodic memory. My PhD research was on how the hippocampus elegantly integrates new knowledge into existing schemas. The brain does this extremely well.

It does this during sleep, especially during REM sleep, replaying important experiences to learn from them. Our earliest Atari program, DQN (DeepMind’s 2013 deep Q-network that first used deep reinforcement learning to reach human-level performance on Atari games), mastered Atari by using experience replay.

This concept, learned from neuroscience—replaying successful paths repeatedly. That was 2013, quite ancient in AI terms, but it was crucial back then.

I agree, we’re basically patching things together now—stuffing everything into context windows. It feels a bit crude. Even if we’re talking about machines rather than biological brains, theoretically, we could have millions or tens of millions of tokens in context, and perfect memory, but retrieval costs still exist. Finding truly relevant information at the moment of specific decision-making isn’t easy, even if you can store everything. So I believe there’s huge room for innovation in memory systems.

Gary Tan: Honestly, a million-token context window is already bigger than I expected and can do a lot.

Demis Hassabis: Yes, for most use cases, it’s enough. But think about it—context roughly equals working memory. Humans have an average of about seven items in working memory, yet we have hundreds of thousands or even millions of tokens in our context window. The problem is, we stuff everything in—including unimportant or incorrect info, which is quite crude. And if you process real-time video streams, naively recording all tokens, a million tokens only last about 20 minutes. But if you want the system to understand your life over a month or two, that’s still far from enough.

Gary Tan: DeepMind has always invested heavily in reinforcement learning and search. How deeply is this philosophy embedded in your current development of Gemini? Is RL still underestimated?

Demis Hassabis: It might indeed be underestimated. Attention to RL has fluctuated over time. From day one at DeepMind, we’ve been working on agent systems. All the work on Atari and AlphaGo essentially belongs to reinforcement learning agents—systems capable of autonomous goal achievement, decision-making, and planning. Of course, we started with games because their complexity is controllable, then gradually moved to more complex games, like AlphaStar after AlphaGo, covering most of the games we could.

The next question is whether these models can generalize into world models or language models, not just game models. We’ve been working on this for years. Today’s leading models’ reasoning patterns and thought chains are essentially a return to what AlphaGo pioneered.

I think much of what we did back then is highly relevant today. We’re re-examining those old ideas, scaling them up, making them more general—Monte Carlo tree search and various reinforcement learning methods. The ideas from AlphaGo and AlphaZero are highly related to foundational models today, and I believe much of the progress in the next few years will come from here.

Distillation and Small Models

Gary Tan: Now, to be smarter, you need bigger models, but at the same time, distillation techniques are improving, making small models quite fast. Your Flash models are very strong, reaching about 95% of the capabilities of the state-of-the-art models, but at only a tenth of the cost. Is that right?

Demis Hassabis: I think that’s one of our core advantages. You need to build the largest models first to gain cutting-edge capabilities. One of our biggest strengths is quickly distilling and compressing those capabilities into smaller models. We invented the distillation method ourselves, and we’re still among the world’s top. Plus, we have strong business motivation to do this. We’re probably the largest AI application platform globally.

With AI Overviews, AI Mode, and Gemini, every Google product—Maps, YouTube, etc.—is integrating Gemini or related tech. These serve billions of users, including products with hundreds of millions or billions of users. They must be extremely fast, efficient, low-cost, and low-latency. This drives us to optimize Flash and smaller Flash-Lite models to be highly efficient, aiming to serve various user needs.

Gary Tan: I’m curious—how smart can these small models get? Is there an upper limit to distillation? Can 50B or 400B models be as smart as today’s largest frontier models?

Demis Hassabis: I don’t think we’ve hit the information-theoretic limit yet, at least no one knows if we have. Maybe someday we’ll reach a density ceiling, but our current assumption is that a cutting-edge Pro model released in half a year to a year can have its capabilities compressed into a very small model that runs on edge devices.

You can see this in Gemma models—our Gemma 4 performs very strongly at the same scale. This involves extensive distillation and efficiency optimization techniques. So I really see no fundamental theoretical limit; we’re still far from it.

Gary Tan: Currently, there’s an astonishing phenomenon—engineers are doing about 500 to 1000 times the work they did six months ago. Some people here are doing work equivalent to what a Google engineer in the 2000s did a thousand times over. Steve Yegge mentioned this.

Demis Hassabis: I find it exciting. Small models have many uses. One is cost and speed—faster iteration, better collaboration. Even if the system isn’t cutting-edge, say at 90-95% of the frontier, that’s enough, and the speed gains outweigh the small performance gap.

Another big trend is running these models on edge devices—not just for efficiency, but for privacy and security. Think of devices handling highly personal data, or robots. For your home robot, you’d want a local, efficient, powerful model, only outsourcing specific tasks to cloud-based large models when necessary. Processing audio and video locally, keeping data on device—this could be the ultimate scenario.

Memory and Reasoning

Gary Tan: Back to context and memory. Currently, models are stateless. If they gain continual learning, what would the developer experience be like? How would you guide such models?

Demis Hassabis: That’s a very interesting question. The lack of continual learning is a key bottleneck for current agents to complete full tasks. Today’s agents are useful for local sub-tasks—you can piece them together for cool things—but they can’t adapt well to your specific environment. That’s why they can’t truly “launch and forget”; they need to learn your particular context. Solving this is essential for achieving full general intelligence.

Gary Tan: How far along are we in reasoning? The current thought chain is strong, but it still trips over mistakes that smart undergraduates wouldn’t make. What needs to change? What progress do you expect in reasoning?

Demis Hassabis: There’s still a lot of room for innovation in thinking paradigms. What we’re doing is still quite rough and brute-force. There are many directions for improvement, like monitoring the thought chain process and intervening mid-thought. I often feel that both our systems and competitors’ systems tend to overthink, falling into loops.

I like to observe Gemini playing chess. All leading foundational models are quite poor at chess, which is interesting.

Watching their thought trajectories is valuable because chess is a well-understood domain. I can quickly tell if they’re going off track or if their reasoning is effective. Sometimes, they consider a move, realize it’s bad, but can’t find a better one, so they circle back and make that bad move. A precise reasoning system shouldn’t do that.

This huge gap still exists, but fixing it might only require one or two adjustments. That’s why you see the so-called “jagged intelligence”—it can solve IMO gold medal problems but, when asked differently, makes elementary math mistakes. It seems to lack something in self-reflection.

The True Capabilities of Agents

Gary Tan: Agents are a big topic. Some say it’s hype. I personally think we’re just at the beginning. What’s DeepMind’s real assessment of agent capabilities, and how does it compare to the public hype?

Demis Hassabis: I agree, we’re just starting. To reach AGI, you need a system that can proactively solve problems for you. That’s always been clear to us. Agents are the way forward, and I think we’re just getting started.

Everyone is exploring how to better integrate agents into workflows. We’ve done a lot of experiments personally, and many here probably have too. How to make agents part of the workflow, not just a nice addition but fundamentally doing real work. We’re still in the experimental phase. Only recently, in the past two or three months, have we started to find particularly valuable scenarios. The technology has just reached a point where it’s no longer toy demos but genuinely adds value to your time and efficiency.

I often see people launching dozens of agents running for dozens of hours, but I’m not sure if the output justifies the effort.

We haven’t yet seen anyone use vibe coding to create a AAA game that tops app store charts. I’ve made prototypes myself, and many here have done some nice demos. I can now make a “Theme Park” prototype in half an hour, whereas I spent six months on it at 17.

I have a feeling that if you spend an entire summer, you could create something truly incredible. But it still requires craftsmanship, human soul, and taste—you must bring these into whatever product you build. In fact, no kid has yet made a blockbuster game selling over ten million copies, but with current tools and effort, it should be possible. Something is missing—maybe in the process, maybe in the tools. I expect in the next 6 to 12 months, we’ll see such results.

Gary Tan: To what extent will that be fully automated? I don’t think it will be fully automatic from the start. The more likely path is that people here first achieve 1000x efficiency, then someone uses these tools to make a hit app or game, and only afterward will more steps be automated.

Demis Hassabis: Exactly, that’s what you should expect first.

Gary Tan: Part of it is also that some people are already doing this, but they’re reluctant to say how much agent assistance helped.

Demis Hassabis: Maybe. But I want to talk about creativity. I often cite the example of AlphaGo—everyone remembers move 37 in game two. For me, I’ve been waiting for that moment to happen, and once it did, I started projects like AlphaFold. We began AlphaFold the day after returning from Seoul—ten years ago. I went to Korea to celebrate the tenth anniversary of AlphaGo.

But just making move 37 isn’t enough. It’s cool, very useful. But can this system invent the game of Go itself? If you give it a high-level description—“a game learned in five minutes, but mastering it takes a lifetime, elegant in aesthetics, finished in an afternoon”—and the system returns Go, that’s different. Today’s systems can’t do that. Why?

Gary Tan: Maybe someone in this room can.

Demis Hassabis: If someone can do it, then the answer isn’t that the system is missing something, but that we’re using it wrong. Maybe that’s the right answer. Perhaps today’s systems already have that capability, just needing a sufficiently talented creator to drive it, to imbue the project with soul, and to be highly integrated with the tools. If you immerse yourself in these tools day and night, with deep creativity, maybe you can create something beyond imagination.

Open Source and Multimodal Models

Gary Tan: Switching topics to open source. Recently, Gemma’s release allows very powerful models to run locally. What’s your view? Will AI become something users control themselves, rather than mainly staying in the cloud? Will this change who can build products with these models?

Demis Hassabis: We are strong supporters of open source and open science. We fully open-sourced AlphaFold. Our scientific work is still published in top journals. For Gemma, we aim to create world-leading models at similar scales. So far, Gemma has been downloaded about 40 million times in just two and a half weeks.

I also think it’s important to have Western tech stacks in open source. Chinese open source models are excellent and currently leading in open source, but we believe Gemma is very competitive at similar scales.

A resource issue remains: no one has excess compute to develop two full-scale frontier models simultaneously. Our current decision is to use edge models for Android, glasses, robots, etc., and keep them open. Once deployed on devices, they are exposed, so it’s better to open them fully. We’ve unified an open strategy at the nano level, which makes strategic sense.

Gary Tan: Before the talk, I demonstrated my AI operating system, where I can interact with Gemini via voice. I was nervous showing it, but it worked. Gemini has been multimodal from the start. I’ve used many models, but the deep integration of voice interaction, tool invocation, and contextual understanding in Gemini is unmatched.

Demis Hassabis: Exactly. One underappreciated advantage of Gemini is that we built it from the start as a multimodal system. This makes initial development more challenging than just text, but we believe the long-term benefits are worth it, and we’re already seeing results.

For example, in world models, we built Genie (DeepMind’s generative interactive environment model) on top of Gemini. In robotics, Gemini Robotics will be based on multimodal foundation models, creating a competitive moat. We’re also increasingly using Gemini in Waymo (Alphabet’s autonomous driving company).

Imagine a digital assistant that follows you into the real world, perhaps on your phone or glasses, understanding your physical environment. Our system excels here. We’ll continue investing in this direction, and I believe our lead in these kinds of problems is significant.

Gary Tan: As reasoning costs rapidly decline, what becomes possible? Will your team’s focus shift because of this?

Demis Hassabis: I’m not sure reasoning will truly become free—Jevons’ Paradox (efficiency improvements leading to increased total consumption) is still there. I think everyone will eventually use all the compute they can get.

Imagine millions of agents collaborating, or a small group of agents thinking along multiple paths and integrating results. We’re experimenting with these directions, and all will consume reasoning resources.

In terms of energy, if we solve issues like controlled nuclear fusion, room-temperature superconductors, and optimal batteries, I believe we can get close to zero energy costs through materials science. But physical manufacturing of chips still has bottlenecks, at least for decades. So reasoning quotas will remain, and efficiency will be crucial.

The Next Scientific Breakthrough

Gary Tan: Fortunately, small models are getting smarter. Many founders here are in biotech and related fields. AlphaFold 3 has surpassed proteins, extending to broader biomolecules. How far are we from modeling complete cellular systems? Is that a completely different level of difficulty?

Demis Hassabis: Isomorphic Labs is making great progress. AlphaFold is just one step in drug discovery. We’re working on adjacent biochemical research—designing molecules with the right properties—and will have major releases soon.

Our ultimate goal is to create a full virtual cell—a comprehensive, perturbable cell simulator that produces results close to experiments and has practical use. It can skip many search steps, generate large amounts of synthetic data to train other models, and predict real cell behavior.

I estimate about ten years to a fully virtual cell. We’re starting from the nucleus, which is relatively self-contained. The key is to carve out a complexity-appropriate slice that’s self-contained enough to approximate its inputs and outputs, focusing on this subsystem. The nucleus is a good candidate.

Another challenge is data scarcity. I’ve spoken with top scientists in electron microscopy and imaging. If we could image live cells without killing them, it would be revolutionary—turning it into a visual problem we know how to solve.

But, as far as I know, no current technology can image live cells at nanometer resolution without damaging them. Static images at that resolution are very detailed, exciting even, but not enough to directly turn into a visual reasoning problem.

So, there are two paths: hardware and data-driven solutions, or building better learnable simulators to model these dynamic systems.

Gary Tan: You’re not only looking at biology. Materials science, drug discovery, climate modeling, mathematics—if you had to rank, which scientific field will be most thoroughly transformed in the next five years?

Demis Hassabis: Every field is exciting, which is why AI has been my greatest passion for over 30 years. I’ve always believed AI will be the ultimate scientific tool—advancing understanding, discovery, medicine, and our comprehension of the universe.

Our initial mission statement was two steps: first, solve intelligence—build AGI; second, use it to solve everything else. We later refined it because some asked, “Do you really mean to solve all problems?”

And yes, that’s exactly what we mean. Now people are starting to understand what that entails. Specifically, I mean tackling what I call “root node problems” in science—those breakthroughs that unlock entirely new branches of discovery. AlphaFold is a prototype of what we want to do.

Over three million researchers worldwide, nearly every biologist, now use AlphaFold. I’ve heard from pharma executives that almost every new drug discovery will involve AlphaFold at some stage. We’re proud of that impact, and it’s just the beginning.

I can’t think of any scientific or engineering field where AI can’t help. The fields you mentioned are still in the “AlphaFold moment”—promising, but not yet solving the big challenges. In the next two years, we’ll see progress across all these areas, from materials science to mathematics.

Gary Tan: It feels Promethean—giving humanity a whole new capability.

Demis Hassabis: Exactly. But, as the myth of Prometheus warns, we must be cautious about how this power is used, where it’s applied, and the risks of misuse of the same tools.

Success Stories

Gary Tan: Many here are trying to start companies applying AI to science. In your view, what’s the difference between truly cutting-edge startups and those just layering APIs on foundational models claiming “AI for Science”?

Demis Hassabis: I imagine if I were in your shoes, looking at Y Combinator projects, what would I do? One thing is to predict the trajectory of AI—very hard in itself. But I believe combining AI with other deep tech fields offers huge opportunities. This intersection—whether in materials, medicine, or other tough sciences, especially involving atomic-level work—won’t have shortcuts in the foreseeable future. These fields won’t be crushed by the next model update. If you want a resilient direction, that’s what I’d recommend.

I’ve always favored deep tech. Truly durable and valuable things aren’t easy. I’ve been attracted to deep tech since 2010—investors told me “we already know this won’t work,” and academia thought it was a niche that failed in the ’90s.

But if you believe in your ideas—why this time is different, your unique background—ideally, you’re an expert in machine learning and applications, or can assemble such a founding team—then there’s enormous impact and value to create.

Gary Tan: That’s valuable insight. Once something is done, it seems obvious, but before that, everyone opposes you.

Demis Hassabis: Exactly. So you must do what you’re truly passionate about. For me, I’ll keep working on AI no matter what. I decided early on that it’s the most impactful thing I can think of. Turns out, that’s true—though maybe we’re 50 years early.

And it’s also the most interesting thing I can think of. Even if today we’re still in a garage, and AI isn’t fully realized, I’d find ways to keep going. Maybe I’d return to academia, but I’d keep pushing forward.

Gary Tan: AlphaFold is an example of chasing a direction and betting right. What makes a scientific field suitable for breakthroughs like AlphaFold? Are there patterns, like certain objective functions?

Demis Hassabis: I should probably write this down someday. From all Alpha projects like AlphaGo and AlphaFold, I’ve learned that our current techniques work best when:

First, the problem has a huge combinatorial search space—bigger is better, so large that brute-force or special algorithms can’t solve it. The move space in Go and the conformational space of proteins far exceed the number of atoms in the universe. Second, the goal function is clearly defined—like minimizing free energy in proteins or winning in Go—so the system can perform gradient ascent. Third, there’s enough data, or a simulator that can generate large amounts of synthetic data within the distribution.

If these three conditions are met, current methods can go far—finding that “needle in the haystack.” Drug discovery follows the same logic: if a compound can treat a disease without side effects, and physics allows it, the only challenge is how to find it efficiently. AlphaFold proved that such systems can navigate vast search spaces to find these needles.

Gary Tan: I want to elevate this—humans used these methods to create AlphaFold, but on a meta-level, humans are using AI to explore the hypothesis space. How far are we from AI systems doing real scientific reasoning (not just pattern matching in data)?

Demis Hassabis: I think it’s very close. We’re building such general systems. We have an AI co-scientist, and algorithms like AlphaEvolve that go beyond basic Gemini. All top labs are exploring this direction.

But so far, I haven’t seen a truly major scientific discovery made solely by these systems. I believe it’s coming soon. It might relate to the creativity we discussed—breaking through known boundaries. At that level, it’s no longer pattern matching, because there are no patterns to match. It’s more like analogy reasoning—I think these systems currently lack that, or we’re not using them correctly.

A standard I often mention in science is: can it propose a genuinely interesting hypothesis, not just verify one? Verifying a hypothesis can be a huge breakthrough—like proving the Riemann Hypothesis or solving a Millennium Prize problem—but maybe we’re only a few years away from that.

Even harder is whether it can propose a new set of Millennium Prize problems, considered equally profound and worth a lifetime of study by top mathematicians. That’s an order of magnitude more difficult, and we don’t yet know how to do it. But I believe it’s not magic; I trust these systems will eventually do it, maybe missing just one or two pieces.

A way to test this is what I call the “Einstein Test”—can you train a system with knowledge from 1901 and have it independently derive Einstein’s 1905 results, including special relativity? I think we should seriously run this test, repeatedly, to see when it’s achievable. Once it is, these systems are not far from inventing truly new things.

Entrepreneurship Advice

Gary Tan: One last question. Many here have deep technical backgrounds and want to build something at your scale. You’re part of one of the world’s largest AI research organizations. Having been at the forefront of AGI research, what’s something you now know that you wish you knew at 25?

Demis Hassabis: We’ve actually touched on part of this. You’ll find that chasing hard problems and simple problems are quite similar—just approached differently. Different challenges have different difficulties. But life is short, and energy is limited—better to focus on things that no one else will do if you don’t.

Also, I think cross-disciplinary combinations will become more common in the coming years. AI will make crossing fields easier.

Finally, it depends on your AGI timeline. Mine is around 2030. If you start a deep tech project today, it’s usually a ten-year journey. You must include the possibility of AGI appearing midway. What does that mean? Not necessarily bad, but you must consider it. Can your project leverage AGI? How will AGI systems interact with your project?

Referring back to AlphaFold and general AI systems, I foresee a scenario where general systems like Gemini or Claude call specialized systems like AlphaFold as tools. I don’t think we’ll put everything into one giant system.

Original Video Link

Click to learn about Rhythm BlockBeats job openings

Join the Rhythm BlockBeats Official Community:

Telegram Subscription Group: https://t.me/theblockbeats

Telegram Group Chat: https://t.me/BlockBeats_App

Twitter Official Account: https://twitter.com/BlockBeatsAsia