Quantizing local LLMs speeds molecule generation, but reference-guided design varies by model
- Four open-source LLMs were tested at full precision and three quantization levels for local molecule generation.
- Tasks were free generation and generation guided by a set of kinase inhibitors given in the prompt.
- Quantization raised throughput without hurting validity, diversity or other quality metrics in free generation.
- Following the reference chemical space differed sharply by model, and one model lost it at the lowest precision.
💡 Points Based on the abstract
Validity and diversity are weak yardsticks here; both can stay flat while the output drifts away from the chemistry you asked for. For reference-guided design the number I'd want is similarity to the supplied kinase set, plus novelty against it, and a hinge-binder or scaffold check. A model that quietly stops following the prompt at low precision would still look fine on free-generation metrics. On my own project I'd run that reference-adherence test at the exact quantization level I plan to deploy, per model.