Pan-cancer image-based detection of clinically actionable genetic alterations

Molecular alterations in malignant tumors can cause phenotypic changes in tumor cells and their microenvironment. Routine histopathology tissue slides – which are ubiquitously available for patients with solid tumors – can reflect such morphological changes. Here, we show that deep learning can consistently infer a wide range of genetic mutations, molecular tumor subtypes, gene expression signatures and standard pathology biomarkers directly from routine histology images of cancer. We developed, systematically optimized, validated and publicly released a one stop-shop workflow and applied it to routine tissue slides of more than 5000 patients across a broad spectrum of common solid tumors including lung, colorectal, breast and gastric cancer. Our findings show that a single deep learning algorithm can be trained to predict a wide range of molecular alterations from routine, paraffin-embedded histology slides stained with hematoxylin and eosin. These predictions generalize to other populations and yield spatially resolved predictions. Our method can be implemented on mobile hardware, potentially enabling point-of-care diagnostics for personalized cancer treatment. More generally, this approach can be used to elucidate and quantify genotype-phenotype links in cancer.