BLUF: The template the AI-for-science field has chased since AlphaFold won a 2024 Nobel Prize – dump a massive curated dataset into a model and let it pattern-match – is running out of road, according to the people who helped scale it. At the same time, the transformer architecture underneath every major LLM is showing its age on cost grounds, and academic researchers say they are being priced and locked out of studying the very models reshaping their fields. If your team is betting on AI-for-science tooling or planning multi-year model architecture bets, the ground is shifting under the assumption that “more data plus a bigger transformer” is the path forward.
The AlphaFold template is hitting its limit
In a MIT Technology Review opinion piece, Eric Schmidt and Suhas Mahesh lay out why the AI-for-science field’s dominant success story may not generalize. Demis Hassabis and John Jumper of Google DeepMind won part of the 2024 Nobel Prize in chemistry for AlphaFold, which predicts three-dimensional protein structures by learning from tens of thousands of experimentally measured shapes. DeepMind trained it on the Protein Data Bank – roughly 170,000 experimentally validated protein structures that took 53 years of international scientific cooperation and, by Schmidt and Mahesh’s estimate, about $21 billion of experimental work to assemble.
That is the part often left out of AlphaFold’s success story: the breakthrough required a dataset that took over half a century and tens of billions of dollars to build. Schmidt and Mahesh argue that replicating a Protein Data Bank-scale resource in most other scientific fields would take decades, not years, which means the data-first template can’t simply be copy-pasted onto chemistry, materials science, or any other domain lacking an equivalent. Their proposed alternative is AI agents that model the human process of research itself – hypothesis generation, experiment design, iteration – rather than agents that only pattern-match against an existing mountain of labeled data.
The distinction matters because it changes where the industry should be investing. A data-first strategy says: find or build the next Protein Data Bank for your field, then train a large model on it. Schmidt and Mahesh’s argument is that this strategy only works in the narrow set of fields where a dataset of that scale and quality already exists or can plausibly be built in a reasonable timeframe – and most fields, from materials science to synthetic biology to climate modeling, simply do not have a 53-year, $21 billion curated dataset waiting to be exploited. An agent-first strategy instead bets that the bottleneck in scientific progress was never purely a data availability problem; it was also a research-process problem, and an AI system that can generate hypotheses, design experiments, and iterate on results can make progress even in domains where the labeled-data mountain does not exist yet.
Why transformers themselves are being rethought
The data problem is only half the story. In a separate MIT Technology Review piece, Will Douglas Heaven reports on a wave of startups trying to move past the transformer architecture that has powered every major LLM since Google researchers published “Attention Is All You Need” in 2017. Justin Dangel, cofounder and CEO of the startup Subquadratic, put the stakes plainly: “The entire AI industry is built on transformers. They are one of the most important innovations in the history of computer science, and they’ve changed the world.”
The problem is the mechanism that makes transformers work – dense attention, which compares every token in a passage against every other token – scales badly. Heaven notes a document of 10,000 words can require a transformer to run roughly 50 million multiplications, which is a direct driver of the compute and power costs now dominating AI infrastructure conversations. OpenAI is on track to spend $50 billion on computing this year, according to company president Greg Brockman, and the International Energy Agency projects data-center electricity consumption will double by 2030. MIT Technology Review’s “What’s Next” franchise has taken to calling the coming post-transformer generation “LLMs+” – not a clean extension of the current architecture, but a new one built to route around its cost structure.
What makes this notable is the timing relative to the AI-for-science debate above: the two arguments compound rather than run in parallel. If dense attention’s multiplication-heavy design is a meaningful share of why frontier compute spending is climbing toward $50 billion a year at a single lab, then any AI-for-science strategy that assumes ever-larger transformer models trained on ever-larger datasets is affordable at scale is fighting two cost curves at once – the cost of building a Protein Data Bank-equivalent dataset, and the cost of running a transformer large enough to exploit it. Startups like Subquadratic are betting that whichever architecture solves the second problem will also reshape what is economically viable for the first.
Academia’s access problem
The third piece of this picture, also from MIT Technology Review, comes from Grace Huckins’ reporting on the Schmidt Sciences AI2050 program, a fellowship for academics working on AI. At a recent convening, UC Berkeley computer science professor Nika Haghtalab offered a comparison that sticks: being an AI academic today is like being a biologist in a world where private companies hold exclusive control over CRISPR. Researchers outside the frontier labs can observe how ChatGPT and Claude behave from the outside, but they cannot examine the design or training internals of either, and they cannot steer that design themselves (MIT Technology Review).
The economics compound the access problem. Universities generally cannot afford the GPU clusters needed to train or even run frontier-scale models. AI2050 fellows get some funding earmarked for GPU access, which multiple researchers cited as a major benefit of the program – but even for researchers who never train a model themselves, the cost of repeatedly querying OpenAI’s, Anthropic’s, and Google’s APIs to study those systems rigorously can be prohibitive, a problem sharpened by reductions in US federal science funding. The practical result: many academic fellows now steer away from questions the frontier labs are likely to answer on their own timeline, and toward gaps those labs have no commercial reason to fill.
Haghtalab’s CRISPR comparison is worth sitting with. In biology, the fact that a small number of companies held exclusive commercial control over a foundational gene-editing tool did not stop academic biology – but it did shape which questions academic labs pursued, steering them away from anything requiring proprietary access and toward areas where independent replication was still possible. The same dynamic appears to be forming around frontier AI models: independent verification of how Claude, ChatGPT, or Gemini actually work internally is becoming structurally harder, not because anyone is hiding wrongdoing, but because the economics of training and running frontier models increasingly exclude everyone who isn’t already inside one of three or four companies.
What this means for you
- Don’t assume the current architecture is the long-term bet. If you’re evaluating vendors or building internal tooling around today’s transformer-based models, treat the underlying architecture as a variable, not a constant. A wave of funded startups is explicitly targeting the compute-cost weakness in dense attention, and MIT Technology Review’s own framing – “LLMs+” – signals this is expected to be a real architectural transition, not a fringe research direction.
- Budget for compute costs to keep climbing before they plateau. OpenAI’s own $50 billion compute spend this year and the IEA’s data-center doubling projection both point the same direction as the ongoing AI infrastructure buildout this site has tracked, including AMD’s and Google’s recent infrastructure spending (see our infrastructure arms race coverage). If your product roadmap assumes inference costs will fall in a straight line, this data set argues for more caution than that.
- If you’re building AI-for-science tooling, plan for agents that reason, not just retrieve. Schmidt and Mahesh’s thesis – that the next wave of scientific AI progress comes from agents modeling the research process, not from bigger labeled datasets – lines up with other frontier-lab moves toward research-automating agents, including Jeff Dean’s departure from Google specifically to build AI that runs its own research loop (our coverage of that move). Teams building in this space should weight agent-driven experiment design over pure data-scaling bets.
- If your team relies on academic AI research, expect a widening gap. Access to frontier model internals is concentrating inside Anthropic, OpenAI, and Google, which means independent verification and academic scrutiny of these systems will get harder, not easier, over the next few years. Compute-constrained academic partners may need direct API budget lines from you if the collaboration depends on rigorous testing at scale.
- Separate the marketing claim “AI accelerates science” from the specific mechanism being claimed. AlphaFold’s success gets cited constantly as proof that AI can compress decades of scientific work, but Schmidt and Mahesh’s own framing shows the mechanism was a rare, expensively curated dataset paired with pattern-matching, not a repeatable recipe. When you evaluate a vendor’s AI-for-science pitch, ask which mechanism they are actually relying on: do they have (or claim to have) a Protein Data Bank-equivalent for their domain, or are they betting on the reasoning-agent approach instead? Those are different technical bets with different risk profiles, and a pitch that conflates the two is worth pushing back on.
Next step
If your AI-for-science or model-architecture roadmap for 2027 assumes the current transformer-plus-scale approach holds steady, pressure-test that assumption against this week’s reporting: track at least one post-transformer startup and one reasoning-agent-for-research effort over the next two quarters before locking in a multi-year architecture bet.
