Table 2 Summarizes the representative multimodal fusion works that combine pathomics and genomics for better clinical prediction tasks
From: Pathogenomics for accurate diagnosis, treatment, prognosis of oncology: a cutting edge overview
Tumor type | Models | References | Data modality | Performance |
|---|---|---|---|---|
Pan-cancer | Resnet-50; direct fusion on patch-level | Schmauch et al. [39] | RNA-seq and WSI | MSI status prediction, AUC: 0.81 (RNA + WSI) |
PRAD | Resnet-50; autoencoder fusion in combined late space | Tan et al. [15] | RNA-seq and WSI | Malignant/benign tissue recognition, AUC: 0.74 (RNA + WSI) |
BRCA | DenseNet-121; direct fusion on patch-level | He et al. [7] | RNA-seq and WSI | Subtype prediction, AUC: 0.83 ± 0.05 (RNA + WSI) |
KIRC | Customized architecture; multivariate feature selection | Ning et al. [82] | RNA-seq and WSI | Survival prediction, C-index:0.79 (0.73–0.86) (RNA + WSI) |
KIRC, GBM | VGG19; multimodal tensor fusion | Chen et al. [5] | RNA-seq, CNV, gene mutation and WSI | Survival prediction, KIRC C-index: 0.72 ± 0.031 (RNA + WSI); GBM C-index: 0.82 ± 0.010 (RNA + WSI) |
Pan-cancer | SqueezeNet; unsupervised compression in a single feature vector | Cheerla et al. [63] | Clinical data, WSI, miRNA and mRNA | Survival prediction, C-index: 0.78 (miRNA + mRNA + WSI) |
ER-BC | ResNet-101; canonical correction analysis | Xu et al. [83] | RNA-seq and WSI | Cancer-specific prediction, P-value 7.23e−06 (gene + WSI) |
GBM | Customized architecture; two-stage feature aggregation | Hao et al. [84] | RNA-seq and WSI | Survival prediction, C-index: 0.70 ± 0.029 (RNA + WSI) |
LIHC | Two-stage feature aggregation | Zhan et al. [85] | WSI and RNA-seq | Survival prediction, C-index: 0.75 (RNA + WSI) |
KIRC, LIHC, LUAD | PCA; shared representation learning | Ning et al. [86] | WSI and RNA-seq | Survival prediction, C-index: 0.658–0.685 (WSI + RNA) |