Review article on transformers for molecular property prediction

We are happy to announce the publication of our review article “Transformers for Molecular Property Prediction: Lessons Learned from the Past Five Years” in the Journal of Chemical Information and Modeling: https://doi.org/10.1021/acs.jcim.4c00747.

In this review, we take a closer look at how transformer models have been used for molecular property prediction over the past five years. We examine the different models and try to answer some of the key questions that come up when developing these models: How much pretraining data is needed? Which model architectures and pretraining objectives are promising? And how can we meaningfully compare the performance of different models? We also highlight gaps in the current research and the need for more standardized data splitting and robust statistical analysis when benchmarking molecular transformer models.

The review was written together with Jochen Sieg (BASF SE), with whom I share first authorship, Miriam Mathea (BASF SE), and Andrea Volkamer (Saarland University).