Top-down design of protein architectures with reinforcement learning

Today we report in Science [PDF] the successful application of reinforcement learning to a challenge in protein design. This research is a milestone in the use of artificial intelligence for science, and the potential applications are vast, from developing more effective cancer treatments to new biodegradable textiles. 

A team led by scientists in the Baker Lab developed powerful new protein design software based on a strategy that has proven adept at board games like chess and Go. In one experiment, proteins made with the new approach were found to be more effective at generating useful antibodies in mice, suggesting that this breakthrough may soon lead to more potent vaccines.

This research was led by Isaac Lutz, Shunzhi Wang, PhD, and Christoffer Norn, PhD of the Baker Lab. The manuscript is titled Top-down design of protein architectures with reinforcement learning.

“Our results show that reinforcement learning can do more than master board games. When trained to solve long-standing puzzles in protein science, the software excelled at creating useful molecules,” said senior author David Baker, PhD, director of the Institute for Protein Design. “If this method is applied to the right research problems, it could accelerate progress in a variety of scientific fields.”

A game-inspired approach 

Reinforcement learning is a type of machine learning in which a computer program learns to make decisions by trying different actions and receiving feedback. Such an algorithm can learn to play chess, for example, by trying millions of different moves that lead to victory or loss on the board. The program is designed to learn from these experiences and become better at making decisions over time.

To make a reinforcement learning program for protein design, the scientists gave the computer millions of simple starting molecules. The software then made ten thousand attempts at randomly improving each toward a predefined goal. The computer made the proteins longer or bent them in specific ways until it learned how to contort them into desired shapes.

“Our approach is unique because we use reinforcement learning to solve the problem of creating protein shapes that fit together like pieces of a puzzle. This simply was not possible using prior approaches and has the potential to transform the types of molecules we can build.”

— Isaac Lutz, co-lead author

Atomically-accurate design

As part of this study, the scientists manufactured hundreds of AI-designed proteins in the lab. Using powerful electron microscopes and other instruments, they were able to confirm that many of the protein shapes created by the computer were indeed realized in the lab.

“We asked the software to make spherical structures with no holes, small holes, or large holes, and it worked most of the time. Its potential to make all kinds of architectures has yet to be fully explored.”

— Shunzhi Wang, PhD, co-lead author

The team focused on designing new nano-scale structures composed of many protein molecules. This required designing both the protein components themselves and the chemical interfaces that allow the nano-structures to self-assemble. Electron microscopy confirmed that numerous AI-designed nano-structures were able to form in the lab. As a measure of how accurate the design software had become, the scientists observed many unique nano-structures in which every atom was found to be in the intended place. In other words, the deviation between the intended and realized nano-structure was on average less than the width of a single atom. This is called atomically-accurate design.

The authors foresee a future in which this approach enables them and others to create therapeutic proteins, vaccines, and other molecules that could not have been made using prior methods. 

Controlling cell signaling

Researchers from the UW Medicine Institute for Stem Cell and Regenerative Medicine used primary cell models of blood vessel cells to show that the designed protein scaffolds outperformed previous versions of the technology. For example, because the receptors that help cells receive and interpret signals were clustered more densely on the more compact scaffolds, they were more effective at promoting blood vessel stability. 

Hannele Ruohola-Baker, a professor of biochemistry and one of the study’s authors, speaks to the implications of the investigation for regenerative medicine. “The more accurate the technology becomes, the more it opens up potential applications, including vascular treatments for diabetes, brain injuries, strokes, and other cases where blood vessels are at risk. We can also imagine more precise delivery of factors that we use to differentiate stem cells into various cell types, giving us new ways to regulate the processes of cell development and aging.”


This work was funded by the National Institutes of Health (P30 GM124169, S10OD018483, 1U19AG065156-01, T90 DE021984, 1P01AI167966); Open Philanthropy Project and The Audacious Project at the Institute for Protein Design; Novo Nordisk Foundation (NNF170C0030446); Microsoft; and Amgen. Research was in part conducted at the Advanced Light Source, a national user facility operated by Lawrence Berkeley National Laboratory on behalf of the Department of Energy.