Skip to main content

Table 1 Definition of terms used in this analysis; it should be noted that the goal of unimodal self-supervised methods (e.g., Uni-Self) is to predict survival, regularized using a self-supervised learning objective

From: Assessment of emerging pretraining strategies in interpretable multimodal deep learning for cancer prognostication

Concept

Term

Description

Data Integration Strategy

Unimodal

Methods that operate on data from a single modality (ie; gene expression, DNAm, or WSI)

 

Multimodal

Methods that operate on data from multiple modalities, at the same time (ie; combining gene expression, DNAm, and WSI)

Pretraining Strategies

Self-Supervised (Self)

Methods which use properties of an individual data type to learn meaningful representations

Crossmodal (Cross)

Methods predict a complementary modality from an input one

Transfer Learning (Transfer)

Methods which learn information from particular subtype(s), and aim to apply them to other subtypes

Model Names

(Data-Pretraining-Modality)

Uni-Self-Omics

Gene expression and DNAm networks pretrained using VAEs

 

Uni-Self-WSI

WSG GCNs pretrained using surival prediction

 

Uni-Cross-Omics

Gene expression and DNAm networks pretrained using crossmodal prediction

 

Uni-Cross-WSI

WSG GCNs pretrained using crossmodal pretraining and GCL

 

Uni-Transfer

Unimodal models which were pretrained using transfer learning from other subtypes

 

Multi-Self

Multimodal models which leveraged embeddings from self-supervised pretraining on the individual modalities

 

Multi-Cross

Multimodal models which leveraged cross-modal pretraining

 

Multi-Transfer

Multimodal models which were pretrained using transfer learning