MediaFrontier AIs handle conventional chemistry problems very well. They can analyse an experimental plan, correctly characterise a reaction class and find the relevant literature. They read and write SMILES without trouble. Ask the AI whether a reaction will work and you get a fluent, confident answer and plausible explanation.
Where the frontier AIs start to struggle, however, is when we ask them to work on what we call “novel” chemistry, challenges that test the limits of known chemistry, what we know from published literature. When the data is sparse or non-existent, frontier AI performance decays.
In our work at Molecule.one, we've known for some time that those challenges are where specialty AIs, like Molecule.one's Maria, not only excel, they far surpass the frontier AIs in predicting outcomes.
To demonstrate this point, we devised a competition between Maria, GPT-6 Astra and human chemists.
Maria vs GPT-6 Astra vs. Human Chemists
As the foundation for the competition, we used data from 1,500 reactions that we know from our work. They span six reaction classes ranging from the amide coupling, one of the most common reactions in drug discovery and manufacturing, to Chan–Lam and one-pot Suzuki couplings, which are used far less frequently.
Importantly, our Maria AI did not have any data about the substrates, or building blocks, used in these reactions. Maria had never seen any of these molecules and had no knowledge of these reactions. We set this control because it mirrors the real world, where chemists constantly work with molecules no one has used before.
We could not apply the same control to Astra. OpenAI does not disclose its training data, and some of these building blocks, possibly even some of the reactions, appear in patents and public databases. Astra will have seen part of the test before, which, if anything, works in its favor. We estimate how much in its favor below.
We also enlisted PhD chemists, three per reaction class, and they had only their knowledge and chemical “intuition” to draw upon.
We asked Maria, GPT-6 Astra and PhD chemists the same question for each reaction: will it produce the intended product with better than 10% yield? In medicinal chemistry the goal is to isolate enough of the product for biological testing, which usually does not require achieving a high yield – 10% is a minimal benchmark.
Ranking The Results
When a medicinal chemist is designing a compound library, they start with the reactions that are most likely to succeed. We asked Maria, Astra, and the human chemists to do the same: rank the reactions according to the likelihood of success.
Astra and Maria gave each reaction a probability of clearing 10% yield. The chemists just answered yes or no to the 10% yield, and we averaged their answers. We also added a randomly chosen set of outcomes.
Within each reaction class, we sorted the reactions from most to least likely to achieve the specified yield. This gives each predictor (Maria, Astra, human chemists) a ranked list, like a chemist's shortlist of what to run first. Then we worked down each list until we had found a quarter of the reactions that actually worked, and checked how many of the picks along the way were hits. We report this relative to random picks: a score of 2× means a predictor's picks worked twice as often as random ones.
The Winner
Maria's picks worked 3.8× more often than random ones. Astra's score was 2.7× better than the random selection and the human chemists attained just 1.2×.
The chart below shows the average over all six classes (left) and each class separately (right), where the dashed line marks the best possible score.
Why did our PhD chemists' picks work only 1.2× more often than random ones? Chemists are good at judging whether a reaction makes sense based on compatibility and conditions. With our 1,500 reactions, almost all made sense on paper. The challenge went beyond what's on paper: Will this particular combination of molecules clear a 10% yield, at microliter scale, under fixed conditions? The answer often depends on small interactions at the molecular level that humans have not observed often enough to incorporate in their “chemical intuition.” (See our post on the limits of chemical intuition). AIs, on the other hand, are far more likely to pick up on patterns associated with these minute or unobserved effects.
At this point, we know that Maria outperforms Astra on average across the six reaction classes, but we have not really explained why. Further, we have not addressed why there is so much variability in the outcome by reaction classes. Why are Maria and Astra roughly equal in predictive ability on amide couplings, for example, but Maria is much superior on Chan–Lam couplings?
The Less Known, the Better Maria Performs
In our competition, one key difference between Astra and Maria was the data.
We noted earlier that Astra, unlike Maria, would have seen data on some of the molecules used in our test reactions, if not complete reactions, in the public data it was trained on. We can estimate how much data based on patents filed and the Open Reaction Database, which we present below. The data available to Maria, on the other hand, was only Maria's proprietary data set, generated by thousands of weekly experiments over the past 4 years.
What we see is that Maria's advantage accelerates as the available public data diminishes.
For example, when it comes to the widely used amide coupling, there are about 2 million reactions described in patent filings. In other words, there is a lot of public information about amide couplings. And when it comes to predicting outcomes with the amide coupling, Maria and Astra are more or less equal – 1.9× better than random.
If we look at the data available for Chan–Lam and one-pot Suzuki reactions, on the other hand, available public data drops a hundredfold to about 20,000 reactions each. For these couplings, Astra's performance versus random dips to 1.6× and Maria's soars to 5.1× – essentially 3× better than Astra.
The same pattern holds when we look not at how often a reaction type is published, but at whether there are published reactions of the same type involving the starting molecules in our test reactions. For each reaction class, we checked what share of these molecules appear in such reactions in patents or the Open Reaction Database. It ranges from under 10% for Chan–Lam to around 70% for amide coupling. The fewer published results there are for these molecules, the bigger Maria's lead (right chart).
How does Maria Do Better With Less?
To understand why Maria does better with less, it's important to understand the key differences between the available public data and Maria's data.
As already noted, public data can be vast, as in the case of amide couplings, but it's also not systematically presented, nor is it complete. Failed reactions, for example, are rarely documented, and not all reaction conditions or substrates are necessarily captured. In other words, lots of data never leaves the lab bench. If results for a particular family of reactions have been written up countless times, however, sheer volume compensates for the messiness.
Maria's data is fundamentally different in at least two important ways. First, Maria's HTE lab has systematically captured all the data from more than 500,000 reactions to date, including dozens of data points – what formed, what didn't, under what conditions, and at what yield. LC-UV-MS documents each outcome. That's terabytes of data. Nothing is left out. Second, Maria is increasingly covering chemistry that has never been explored before, at least in the public data. In Maria's Chan–Lam work, for example, many of the substrates tested had, as far as we know, never been tried.
Maria captures all that data in a way that can be readily used for fresh training runs. When it comes to less well documented reaction classes, like Chan–Lam, the detail, nuance and completeness of reaction data is the key to Maria's superior predictions. The data is not messy, which makes it more useful, and it covers many reaction classes. That systematic breadth of data helps Maria to perform on par with or far better than frontier AIs, and Maria's advantage is advancing each month with tens of thousands of new reactions conducted in Maria's lab. The further we push the boundaries of known chemistry, the more interesting the discoveries.
For clarity's sake, we should add that outside of this competition, we also train Maria on publicly available data.
More on Maria's Acceleration
In real life, Maria does not advance by looking backwards. When working on a new challenge, Maria develops hypotheses, designs experimental pathways, executes them in Maria lab, analyzes results, and then repeats, as necessary. Chemistry is still poorly mapped, which is why Maria is built on the foundation of constant experimentation and data generation. Today our lab runs thousands of reactions every week, and they go exactly where the map of chemistry is still sparse: reaction types that are very useful in medicinal chemistry but still poorly understood. Each week we aim them further from charted territory, and Maria learns from every result.
This loop is only accelerating. In our Chan–Lam project with OpenAI, going from an idea to a confirmed result (OAI-M1-03: adding TEMPO improved yields for more than 80% of the substrates tested) took seven weeks and 10,080 reactions, a project we estimate would have taken at least a year with a more traditional approach.
This month, we set Maria to search the LC-UV-MS data from thousands of our Chan–Lam reactions for unexplained signals. Just over an hour later, it had found an unexpected product. Within 11 days of starting the search, we had reproduced its signal at bench scale, an early step towards a possible new reaction class. (Read more in my colleague Stan's latest post).
In less than six months, the time to major discoveries has dropped from weeks to days. Soon it will be hours. Chemistry will never be the same, which is to say, better than anyone would have believed possible a year ago. The incredible implications for industry, pharmaceuticals and drug discovery are hard to comprehend, but they will be arriving soon.
Appendix: The benchmark
The test set was held out from our high-throughput experimentation (HTE) campaign. It covers six types of reaction that medicinal chemists use to join two building blocks:
- Amide coupling: joins an acid and an amine; the most common reaction in drug discovery.
- N-alkylation: attaches a carbon chain to a nitrogen atom.
- Reductive amination: joins an amine and an aldehyde or ketone.
- Suzuki coupling: joins two aromatic rings with a palladium catalyst.
- One-pot Suzuki: a Suzuki coupling between two aryl halides. One of them is first converted into a boron compound, then coupled with the other in the same vessel, with no isolation in between. It saves a step, but it is run far less often than the standard version.
- Chan–Lam coupling: joins an amine and an aromatic ring with a copper reagent.
For each type we tested 125 pairs of building blocks, each under two sets of conditions: 1,500 reactions in total. A reaction counts as a hit if it gives more than 10% yield.
The task is the same for every predictor. Given two substrates and two candidate conditions, predict whether each reaction gives more than 10% yield.
- GPT-6 Astra (medium reasoning) gets the substrate SMILES and a full description of both conditions. It returns a hit probability for each condition.
- Chemists (three per class) fill in the same answer sheets, yes or no per condition. We average their answers.
- Maria is our AI agent. It has access to Molecule.one's internal models trained on our lab data. For this test we use its reaction-outcome prediction: an ensemble of models trained on Molecule.one HTE data.
What each predictor may have seen
Maria's training is substrate-excluded, so neither substrate of a test reaction appears in the training or validation data. Every test reaction involves molecules the model has never seen. Astra's training data is not public, and some of the building blocks, possibly even some of the reactions themselves, appear in published sources it may have learned from. We estimated this from patents and the Open Reaction Database: the share of test substrates that appear in published reactions of the same class ranges from under 10% for Chan–Lam to around 70% for amide coupling. Where that share is high, Astra's results may partly reflect what it has already seen.
The full precision-recall curves for each class are below; the dotted line marks 25% recall, where we read off the scores above.
Back to Media