Leave-one-paper-out splits erase machine learning's apparent skill at predicting chiral separations, pointing to the data rather than the model
- The authors curated the largest public chiral chromatography dataset into over ten thousand enantiomer pairs across hundreds of source papers and many columns.
- Under a random split, elution-order prediction looked clearly better than chance; under leave-one-paper-out it fell to the level of a within-paper permutation null.
- Much of the variance is between papers and is not predictable from the recorded conditions, and most of the remainder is an unidentifiable molecule-by-condition interaction plus noise.
- They argue the bottleneck is data structure and call for condition metadata, systematic grids, replicates and recorded failures.
💡 Points Based on the abstract
This is the same lesson as the ADMET and peptide leakage audits earlier this week, now in analytical chemistry: group-aware splits by source, not molecule, are what show whether a model transfers to the next lab. If your team ranks chiral methods or predicts retention from literature data, expect nothing beyond a prior. The data-collection checklist at the end is the actionable part.