Computational & AI Journal of Chemical Information and Modeling (ACS)

Curated, bias-aware benchmark shows protein–ligand scoring performance drops versus CASF-2016

  • A new benchmark built from PDB and BindingDB entries uses tiered filtering, manual electron-density checks, affinity verification and property-matched decoys to improve structural reliability and reduce bias.
  • Nine scoring functions were evaluated retrospectively on scoring, ranking, docking and a screening-like decoy task.
  • Most tasks showed weaker results than on CASF-2016, while the decoy task shifted model rankings rather than lowering all scores uniformly.
  • Overlap with training data varied widely between methods, so performance depends on curation, protocol and data exposure.
Points Based on the abstract

A decoy task that reshuffles rankings rather than just lowering scores is the useful signal: it suggests some functions were leaning on property or pose artifacts that CASF-2016 never penalized. Before trusting any leaderboard, check training overlap per method. Then rerun your shortlisted scorers on our own series with matched decoys and known inactives, and see if the order holds.