MIT's CheMeleon Predicts Molecular Behavior from Scant Data

Earlier this year, OpenADMET organized a contest to predict the properties of molecules that had never been synthesized. "It was a big contest, a lot of people participated," Green said. When the winners were announced, nine of the top 20 entries had used CheMeleon. "So then we started getting all kinds of inquiries from the pharma companies about this, but our paper had just been submitted; it wasn't even peer-reviewed or anything yet."

During the review process, adoption accelerated. At least one pharmaceutical company, Green said, "sent a message to all their people, and said 'Anytime you're modeling a property, start with CheMeleon, that's our new company policy'" when making initial decisions about which molecules to pursue.

How does the model work?

CheMeleon is a pre-trained foundation model (a model first trained on a broad dataset, then adapted to specific tasks). Its design echoes language models such as ChatGPT, which learn from massive amounts of internet text. Instead of text, CheMeleon learned from a million molecules drawn from PubChem, a large, freely available database of chemical information maintained by the U.S. National Institutes of Health.

The key was pre-training the model to learn correlations between molecular structure and roughly 1,600 descriptors that chemists have developed over decades — quantities like molecular weight or the number of oxygens in a molecule. After that, CheMeleon can be fine-tuned to predict well-known properties such as solubility and viscosity, or more esoteric ones such as bioactivity or reactivity with a particular protein or enzyme.

A million molecules is a minuscule fraction of the ways atoms can bond together, but it provides enough examples for the system to work out underlying patterns on its own and apply them to new, unknown molecules.

"Most of today's deep-learning models require more data than traditional methods," Zalte said. Some successful models draw on extensive published chemical literature. "We don't look at the literature at all. We just calculate a bunch of molecular descriptors like molecular weight or number of oxygens, among others," and predicting those descriptors forms the pre-training task.

By contrast, classical expert-driven models rely on specialists manually encoding knowledge — for example, that solubility may relate to molecular weight, oxygen count, or how branchy or bulky a molecule is. The MIT approach instead trains an existing neural network architecture as a general model that "understands some level of chemistry," Zalte said, so that fine-tuning with just a few data points still yields good performance, because the model is "not having to learn everything from scratch."

How well does it perform?

The team compared the model both against other foundation models and against classical models leveraging expert-designed principles. Across 58 test cases, "we see that our model comes out on top," Zalte said.

Green added: "The CheMeleon foundation model drastically reduces the number of molecules that need to be tested experimentally to accurately predict the properties of new molecules. This is a huge advantage, since synthesizing and testing a large number of molecules is slow and expensive."

The work was supported by the Machine Learning for Pharmaceutical Discovery and Synthesis consortium, along with fellowship support from the U.S. Department of Energy and MathWorks.

A model that learns chemistry from a million examples, and then needs only a few data points to answer new questions.