Derby Mill co-host Richard Sutton and his former student, David Silver, recently published a paper about the future of artificial intelligence, called Welcome to the Era of Experience. So in this episode the show’s other hosts—Ajay Agrawal, Sendhil Mullainathan and Niamh Gavin—take their chance to interview Rich about the essay, and provide their take on its implications.
Today’s large language models (LLMs) are trained on human-generated data. So far, this has led to the development of incredible capabilities, such as mastering complex games like backgammon or chess, or absorbing content created by humans and creating fascinating new iterations of art.
While the evolution of LLMs—from AlphaZero (2017) to ChatGPT (2022) to DeepSeek (2025) and beyond—can make it seem as though their possibilities are endless, the agents remain constrained by the scope of the data they are given. In the paper, Silver and Sutton write that “in key domains such as mathematics, coding, and science, the knowledge extracted from human data is rapidly approaching a limit.” Consequently, AI agents will have to be trained on other data, such as their own experiences, which could lead to rapid innovation and superhuman capabilities—a time period which Silver and Sutton refer to as the “age of experience.”
This episode, a roundtable discussion, focuses on the following quotes pulled from the paper:
Why now? "This will become possible, as outlined above, when agents are able to autonomously act and observe in streams of real-world experience, and where the rewards may be flexibly connected to any of an abundance of grounded, real-world signals."
Why science? "Perhaps most transformative will be the acceleration of scientific discovery."
Human-like vs superhuman AIs. "This era of experience will likely be characterised by agents and environments that, in addition to learning from vast quantities of experiential data, will break through the limitations of human-centric AI systems... Furthermore, the pursuit of this agenda by the AI community will spur new innovations in these directions that rapidly progress AI towards truly superhuman agents.”
GUESTS AND HOSTS
Ajay Agrawal, co-founder and partner, Intrepid Growth Partners
Richard Sutton, senior advisor, Intrepid Growth Partners, 2024 Turing Award recipient, pioneer of reinforcement learning and professor, University of Alberta
Sendhil Mullainathan, senior advisor, Intrepid Growth Partners, MacArthur Genius grant recipient and professor, MIT
Niamh Gavin, senior advisor, Intrepid Growth Partners, Applied AI scientist, CEO, Emergent Platforms
LINKS
Derby Mill show website
Read Rich Sutton’s latest paper Welcome to the Era of Experience
Rich Sutton’s 2019 paper The Bitter Lesson
Co-founder of OpenAI, Ilya Sutskever, says AI reasoning power will become less predictable
Listen to our previous episode about DeepSeek
Check out co-author David Silver’s website
Rich Sutton’s home page. Follow Rich on X
Sendhil Mullainathan’s website. Follow Sendhil on X
Be sure to catch every episode of The Derby Mill Series by subscribing on the following platforms: YouTube // Spotify // Apple Podcasts
DISCUSSION POINTS
00:00 Introduction
01:34 Context about the paper
02:56 Chronology of AI paradigms
03:10 Why now?
06:09 Niamh’s chronology of AI development
14:10 Why science?
20:36 Sendhil on scientific research and AI
27:07 Grounded vs. ungrounded rewards
29:21 Rich on RL temporal difference errors
31:10 Human-like vs. superhuman AIs
36:40 Final comments
Nugget 01 - AI for Scientific Discovery
Nugget 02 - Is Science like RL?
Nugget 03 - The Value of Experience
DISCLAIMER
The content of this podcast is for informational and educational purposes only and should not be construed as marketing, solicitation, or an offer to buy or sell any securities or investments. The opinions expressed in this video are those of the participants and do not necessarily reflect the views of Intrepid Growth Partners or its affiliates. Any discussion of specific companies, technologies, or industries is for illustrative purposes and does not constitute investment advice. Viewers are encouraged to consult with their own financial, legal, and tax advisors before making any investment decisions.
Share this post