Evaluation of a machine learning system for genomic antimicrobial susceptibility determination on a clinically representative test set featuring Day Zero Diagnostics
ABSTRACT
Next-generation sequencing is anticipated to transform infectious disease treatment by enabling faster diagnosis, yet a clinical-grade method for genomic AST determination is still lacking. We assessed the accuracy of Keynome gAST, a k-mer based machine learning system for genomic AST, on 956 clinical bacterial isolates. We compared 7,801 predictions of susceptible (S), intermediate (I), and resistant (R) across 97 species-drug combinations against phenotypic AST results. Across the full data set, the Keynome gAST Qualified panel achieved a categorical agreement of 96.9% with very major and major error rates of 1.4% and 0.8%, respectively. At the level of individual species-drug combinations with sufficient data to assess, performance was comparable to the aggregate, with median categorical agreement of 97.4%. We also compared the performance of Keynome gAST to ResFinder, a simple resistance marker approach, in distinguishing susceptible vs non-susceptible samples. On the full data set, Keynome gAST performance was significantly better than that of ResFinder (binary accuracy of 96.0% vs 83.5%). When analyzed at the species-drug level, ResFinder’s poor performance was seen to be driven by either low sensitivity due to phenotypically resistant samples lacking resistance markers (18.0% of such samples) or low specificity due to the presence of resistance markers in phenotypically susceptible samples (15.1% of such samples). In total, these results demonstrate the dual advantages of machine learning and whole-genome profiling that Keynome gAST has over simple resistance marker approaches, enabling high accuracy genomic AST across a range of clinically relevant species-drug combinations.
