Assessment of brain cancer atlas maps with multimodal imaging features

Abstract

Background

Glioblastoma Multiforme (GBM) is a fast-growing and highly aggressive brain tumor that invades the nearby brain tissue and presents secondary nodular lesions across the whole brain but generally does not spread to distant organs. Without treatment, GBM can result in death in about 6 months. The challenges are known to depend on multiple factors: brain localization, resistance to conventional therapy, disrupted tumor blood supply inhibiting effective drug delivery, complications from peritumoral edema, intracranial hypertension, seizures, and neurotoxicity.

Main text

Imaging techniques are routinely used to obtain accurate detections of lesions that localize brain tumors. Especially magnetic resonance imaging (MRI) delivers multimodal images both before and after the administration of contrast, which results in displaying enhancement and describing physiological features as hemodynamic processes. This review considers one possible extension of the use of radiomics in GBM studies, one that recalibrates the analysis of targeted segmentations to the whole organ scale. After identifying critical areas of research, the focus is on illustrating the potential utility of an integrated approach with multimodal imaging, radiomic data processing and brain atlases as the main components. The templates associated with the outcome of straightforward analyses represent promising inference tools able to spatio-temporally inform on the GBM evolution while being generalizable also to other cancers.

Conclusions

The focus on novel inference strategies applicable to complex cancer systems and based on building radiomic models from multimodal imaging data can be well supported by machine learning and other computational tools potentially able to translate suitably processed information into more accurate patient stratifications and evaluations of treatment efficacy.

Graphical Abstract

Introduction

Glioblastoma multiforme (GBM)

GBM is the most common type of malignant primary brain tumor but is still not well understood. It presents a poor prognosis with a median survival of about 15 months for patients who receive standard therapy (Stupp protocol, based on surgical resection and radiotherapy combined with concomitant chemotherapy). Population-RWD (real-world data) based meta-analyses suggest that 2-, 3- and 5-year survival in GBM patients only partially improved since 2005 [1]. Recent advances in understanding the molecular biology of GBM leveraged studies on genetic alterations and genomic profiles [2,3,4], including subclone diversity, via single cell DNA sequencing, and transcriptome profile diversity, via single cell RNA-sequencing [5, 6].

In common with other cancers, the tumor microenvironment (TME) plays a complex role in GBM too. Generally responsible for growth and invasion, TME is highly variable at the intra-tumor spatial level, and both cellular and molecular interactions have not yet led to causal explanations [7, 8]. However, the increased imaging centrality driving radiomic studies, especially magnetic resonance imaging (MRI) in GBM, has drawn novel interest in computational modeling to improve diagnosis, prognostication and clinical decision support [9,10,11].

Radiomics

Radiomics is a multidisciplinary field that engages scientists in processing various interconnected tasks such as tumor segmentation, image preprocessing, feature extraction, model development, testing and validation. Usually, segmentation is performed on annotated images by targeting whole tumors, subregions or peritumoral areas (all defined through regions of interest or ROI) to extract features describing distributions of signal intensities and spatial relationships. Selected features are those bringing differentially informative content regarding texture, shape, statistical descriptions, intensity-based measures etc. The major interest is in texture, as it reveals patterns characterized by brightness, color, slope, size. The analysis of features aims to quantify variations in intensity values and gray levels. Other relevant tumor information comes from shape characteristics whose analysis elucidates geometric aspects through shape descriptors.

There are a few factors in the radiomics workflow that influence these features and their significance. First, features may undergo manual, semi-automatic or automatic treatment. For instance, automatic learning typically involves Machine Learning (ML) and Deep Learning (DL) algorithms that tend to outperform more classical statistical techniques with complex and big datasets [12,13,14]. The search for feature saliency may require a reduction of dimensionality to embed only the most informative characteristics then used to train models and determine the best possible accuracy. Alternatively, analyses supported by hand-crafted features may also be highly valuable due to superior interpretability [15, 16], even when combined with ML-derived ones [17,18,19]. This choice might be instrumental to clinical translational, given multiple criteria to be applied [20, 21].

Due to the substantial image data space that has become available through data repositories for research, examples being The Cancer Imaging Archive, TCIA (https://www.cancerimagingarchive.net/) [22] or the newborn EuCanImage (https://molgenis.eibir-edc.org/), there is a massive call for analysis based on computational and predictive modeling and developments of novel integrative multi-omics inference tools. The applications range from tumor segmentation and anatomical lesion detection [23] to computer-aided diagnosis and prognosis [24, 25], just to mention a few. Some criticism persists relatively to a certain lack of interpretability [26], nonetheless inspiring the emerging explainable artificial intelligence (XAI) [27,28,29,30] research. In general, despite the necessity to measure any radiomic tool in terms of clinical value, our understanding of intra-tumor heterogeneity in GBM has received a strong impulse from radiomics with reference to a few specific domains. No doubt that a better characterization of intra-tumor heterogeneity will determine any future progress in terms of outcome prediction and personalized treatment.

Focus areas

TME, habitats and saliency

As tumor regions are biologically different, the most specific use of radiomic features is one that leverages these quantitative measures to support genetic or epigenetic evidence as well as phenotypes (aggressiveness degree, resistance mechanisms, etc.) elucidating differentiated patterns of metabolism, hypoxia, proliferation, neovascularization. Radiomics that informs on tumor surrounding tissues, including the TME, provides additional prognostication power (for instance, in high-grade glioma [31, 32] this affects the possibility to predict tumor’s aggressiveness) and ability to monitor patient’s response to therapy via follow-up imaging and detect new or modified areas of enhancement, e.g., assessment of risk of tumor progression (TP) versus treatment-related changes or pseudo-progression (PsP) [33, 34].

One aspect of heterogeneity in tumors consists of differentiated TMEs, something which has motivated the recent investigation of the so-called habitats [35,36,37,38,39,40]. Imaging habitats can quantify the grey-level heterogeneity appearing from scans, which helps detect variations in tumor blood supply. Generally obtained by supervised segmentation of the tumor into sub-regions that map its complex organizational structure, the complexity of habitats naturally translates into a very rich data milieu needing ad hoc computational treatment, even beyond the association of habitats with somehow defined clusters [41]. Clustering methods can identify tumor sub-regions contributing with varying prognostic performances. However, the identified clusters or sub-regions are usually the result of the application of globally defined thresholds that often have minimal biological value. Also, biology may not be able to guide selectively the cluster being formed or a reasonable weighting based on the retrieved radiomics features. Finally, clustering often employs simple structures determined with some degree of arbitrariness (say, the choice of the number of clusters) or may assume data hierarchies that are algorithmically valid but have limited contextual relevance (say, random forest, trees etc.).

All the above aspects influence the definition of saliency, for instance relatively to the hypotheses about the importance to assign to tumor regions based on their phenotypic contributions. In principle, a clinically useful saliency map would be predictive for the risk of relapse given the improved model performance expected when selected (salient) features are chosen. However, each specific context must be carefully analyzed, and for instance GBM has a highly complex background with lots of redundant information due to its the multiscale nature. This somehow contrasts with the idea of targeting specific ROI embedding the saliency of contextual characteristics, and is something not linearly solvable (i.e., by fixing thresholds to establish saliency that identify significant foci or connecting to them surrounding regions through some functions or distances).

Multimodal data integration

To address saliency, it is important to consider the fact that imaging informs at the spatial level by identifying sub-regions that may vary across multi-modal sequences (e.g., T1, T1-post contrast, T2 and FLAIR sequences etc.). Interestingly, multi-modality extends beyond the imaging combinations to include the associations with phenotypes, i.e., gene expressions and correlation with molecular subtypes [42], or outcome via survival. These developments have inspired the field of radiogenomics revealing associations between imaging phenotypes (tumor location, neo-angiogenesis, tumor enhancement etc.) and molecular marks, ultimately leading to refined patient stratifications [43,44,45]. Also, the differentiation of tumor molecular profiles based on imaging traits implies that identified MRI phenotypes may be used to probe the underlying genotypes [46,47,48]. MRI-driven radiomics has probed substantial relationships also between genomic GBM profiles and imaging traits [49, 50].

Leveraging imaging multi-modality implies taking decisions on what modes to combine and why, based on the general expectation of building more effective radiomic models compared to those dependent on any single modality only. Decisions face a spectrum of possible combinations of different MRI sequences [51], with relevance assigned to their clinical impact [52]. At data and computational levels, the fusion of multiple modalities may also vary, for instance depending on the stage. At early stages, i.e., before feature classification, when combined with clinical information (risk factors etc.). Inherent technological aspects may also prevail, for instance with CT presenting insufficient soft-tissue contrast. More in general, one problem is the presence of imbalanced classes in the data sets, something affecting the classifier learning ability, likely inducing bias towards the majority class, incorrect predictions and less robust models.

Potential gains from modality fusion in terms of model performance can be tested both during the modeling stage, when fusions may depend on active or incremental learning, and during later stages (decision level) too. [53,54,55,56]. Data heterogeneity is another aspect, as the fusion of mixed data types should achieve superior predictive accuracy. In both such regards, model performance improvements and data diversity, the role of ML toward handling large amounts of radiomic features characterizing tumor phenotypes integrated with other data types has primarily focused on building accurate and replicable models for tumor classification and outcome prediction. As an example, multimodal MRI radiomics can effectively differentiate GBM from lower grade gliomas and characterize the IDH and 1p/19q status using a ML approach useful to clinical practice [57]. In general, treating multimodality presents several complexities. One is inherent to the definition of multimodality (restricted to imaging or inclusive of other data types) [58]. Another is more contextual and may refer for instance to type of treatment and disease evolution [59]. Then, while more robust estimations can be reasonably expected with multimodal methods, scalability can be an issue, although deep neural networks show high accuracy with hyper-scale data sets (about 5 ml images) [60].

Finally, another interesting domain linked to both habitats and data integration/multimodality is imaging synthesis [61,62,63,64,65], where an increased number of image features retrieved from multiple regions and different sources may require relatively large cohort sizes. This may offset the limited biological meaning found in habitats by leveraging large-scale validations involving tissue phenotypes and histology. Note that spatial co-registration of images and histology can be instrumental to the use of MRI habitats for delineation of hypoxia, necrosis etc. [66]. Especially with reference to necrosis, significant contributions have come from multiple studies [67,68,69,70,71,72], indicating the informativeness and classification impact of both handcrafted and deep features extracted from multimodal MRI images. Of note that radiomic texture features informative about both the lesion and the peri-lesion environment (defined as “lesion habitat”) can distinguish radiation necrosis and tumor recurrence [73].

Pillars

Brain atlases as integrative tools

Brain function is related to brain organization, which in turn refers to spatial heterogeneity. Brain Atlases usually operationalize these concepts through the parcellation or partitioning of the brain in multiple closely interacting regions [74]. Many atlases exist that collect brain data (tumor and non) and describe high-resolution location-specific maps centered on imaging data, becoming tools that support quantitative analyses. For instance, fMRI or functional MRI-driven brain studies help define probabilistic maps based on functional and structural data [75]. In general, non-random localizations of lesions are identified through specific patterns and marks in capture points or ROI [76]. Then, brain’s information is processed from characteristics at both structural and functional levels [77]. A common goal is to infer causal influences from experimental measures and exploit the interconnectivity between multiple data to provide mechanistic insights on the nature of their relationships. While fMRI estimates functional connectivity (brain activity), the discovery of brain disease signatures emerging from different brain network patterns can also result from integrating other MRI types, e.g., diffusion (dMRI) and structural (sMRI) to study anatomical and pathological connectivity from shape, size, and integrity of brain structures.

Atlases can be useful reference tools and represent templates for brain segmentation tasks. Among the many available resources and tools, The neuromaps software toolbox [78] (https://github.com/netneurolab/neuromaps) offers structural and functional annotations of the human brain through a variety of reference maps and biological ontologies. Then Neuroparc [79] (https://github.com/neurodata/neuroparc), conceived to standardize existing atlas repositories (46 different adult human brain parcellations of various type, surface-based, volume-based etc.). Global integrative analyses may require other types of tools such as BCGene [80] (http://soft.bioinfo-minzhao.org/bcgene/), which explores genetic mechanisms from about 1400 literature-curated human genes in 40 brain cancer subtypes and about 3000 patients. Specific to GBM, a well-known resource is the Ivy Glioblastoma Atlas Project (Ivy GAP) [81, 82] (http://glioblastoma.alleninstitute.org/), a comprehensive resource on GBM anatomy and genetics characterizing the cellular and molecular structures.

Connectome for functional studies

Connectome research [83, 84] is an area potentially receiving a strong impulse from brain atlases. A reference study [85] in brain tumor connectome analyzed neuroimaging data from 335 adult patients with high- and low-grade glioma and combined them into a replicable tumor frequency map correlated with multiple graph-theoretical metrics establishing high functional connectedness. The application of a regression model with connectome, cellular, and genetic variables has explained 58% of the variance in glioma frequency, showing the independently exerted influences over the anatomic localization of oncogenesis. Another study [86] has leveraged the mapping of independent sources of glioma localization determining their relationships with neurogenic niches, genetic markers, and large-scale connectivity networks. Then, by applying independent component analysis (ICA) to lesion data from 242 adult patients with high-/low-grade glioma, three lesion covariance networks were identified to represent clusters of frequent glioma localization. These networks were associated with clinical variables and genomic information, and structural/functional connectivity was derived from neuroimaging data to uncover brain networks prone to tumor development.

Neuroplasticity

Research on GBM radiomics requires precise definition of structural and functional features to infer factors explaining pathogenetic mechanisms together with disease progression or evolution [87]. By combining such features, MRI technologies can explore the neuroplasticity of structural, topological, biochemical metabolism, and related mechanisms [88]. MRI is typically used to provide detailed anatomical and pathological information in addition to physiological detail. Neuroplasticity caused by highly heterogeneous brain tumors could benefit from multimodal MRI offering individualized prediction of functional prognosis of patients based on ML algorithms [89]. Through the (semi-) automatic identifications of image features, ML brings accuracy in the classification and facilitates the integration with molecular profiles, histological tumor grade, and prognostic factors by using images acquired both at diagnosis and treatment, including follow up to enable differentiation between response and post-treatment-related effects.

Brain atlas and radiomics integration

To demonstrate the potential use of brain atlases in combination with radiomics, we chose the atlases and databases publicly available from UNC-Chapel Hill (as part of NITRC, ‘NeuroImaging Tools & Resources Collaboratory’, https://www.nitrc.org/projects/unc_brain_atlas), and chose ‘UNC_Adult_Brain_Atlas_1’, i.e., the atlas of normal adult human brain anatomy generated from 50 + healthy adult cases (20–59 years old). The atlas comes with T1-weighted images (with and without skull), tissue segmentation probability maps (white matter, gray matter, CSF, rest) and a 27-lobe parcellation map.

The radiomic data were obtained from the multimodal MRI of GBM samples and TCIA radiological data (clinical images) matched to TCGA subjects, considering only a couple of patients (for demonstrative scopes). We subjected the samples to standard open source tools, i.e., 3D Slicer (https://www.slicer.org/) and pyradiomics [90], https://www.radiomics.io/pyradiomics.html. The details of the data sets from The Cancer Genome Atlas Glioblastoma Multiforme Collection (TCGA-GBM) are available at https://wiki.cancerimagingarchive.net/display/Public/TCGA-GBM, and they aim to connect radiological phenotypes to tissue genotypes and patient outcomes. Note that the collected data focus on routine care rather than controlled studies or clinical trials, tissues from multiple sites and heterogeneous images due to different scanner modalities and acquisition protocols.

The visualizations of Fig. 1 display examples of parcellation of the normal brain regions. By co-registering the frame that the atlas parcellation provides and the GBM images across the MRI modalities, we obtained spatially matched and integrated brain maps covering both normal and tumor brain regions. Single and combined MRI characteristics were then mapped (Figs. 2 and 3, respectively) also considering the two brain hemispheres (with and without GBM) for measuring differential effects.

The visual associations between brain regions with or without GBM signatures retrieved from MRI radiomic features is well represented by biclusters. We looked for differential effects captured by the different MRI modalities (modality-1 = ‘FLAIR’; modality-2 = ‘T1w’; modality-3 = ‘T1w postCA’; modality-4 = ‘T2w’). In principle biclustering can identify associations between atlas-defined brain regions and radiomic features, offering a few advantages: (a) Facilitating the investigation of complex disease aspects (pseudo-progression and other treatment-related effects); (b) Bringing translational power for patient stratification and evaluation of treatment efficacy when combined with clinical information; (c) Help select predictive MRI readouts-based models from ML approaches and libraries and use the fusion of information from multimodal features to improve the model performance in terms of prediction accuracy.

Fig. 1

Parcellation map obtained from the NITRC Brain Atlas (https://www.nitrc.org/)

Some limitations may apply. Radiomics performs well by training over large data sets with accurate labels (from expert annotations) to facilitate the process of reproducing results of predictive models, generalizing across multiple training data sets, validating over independent patient cohorts and ultimately translating into the clinics. Nevertheless, radiomic methods that integrate multimodal images can establish robust signatures of functional regions (such as tumor habitats) and associate them to outcomes.

Fig. 2

MRI modalities (top) and Biclustering maps. For one GBM patient, four MRI modalities support biclustering maps displayed with all features (centre) or selected ones (bottom). The applied spectral biclustering algorithm https://scikit-learn.org/stable/modules/biclustering.html [91] assumes a hidden checkerboard structure for the input matrix and thus partitions its rows and columns according to a blockwise-constant checkerboard matrix. The most significant Gauss filtered biclusters appear at the bottom-right corner

By looking at Table 1 and considering that these annotations refer to the top-scored biclusters in Fig. 2 (bottom-right corner), the imaging results in patient A show three recurrent brain regions (l = left, r = right), i.e., l_parietal, r_frontal and r_parietal_cingulate across three modalities, i.e., FLAIR, T1w, T2w. Then 2 regions, l_CSF and r_prefrontal informed by T1w and T2w. In patient B, r_parietal_cingulate recurs across FLAIR and T1w, and r_corpus_callosum, l_cerebellum, r_subcortical, l_temporal recur across T1w postCA and T2w. At the opposite, r_cerebellum and l_subcortical appear only in patient B, while r_frontal only in patient A. Interestingly, very different radiomic feature combinations appear in correspondence with the modalities for the two patients (see the Additional file 2 ‘SM_Allregions’ with all the atlas brain regions).

The FLAIR modality indicates the relevance of texture aspects more in Patient A than B, although with a limited overlap of brain regions involved. T1w informs on new brain regions from both patients and includes similar features with respect to local homogeneity properties of the images. T1w postCA involves a mix of brain regions in part overlapping between the two patients but associated to different features referred to shape descriptors (from ROI in both 2D and 3D) and texture heterogeneity (via glcm or gray level co-occurrence matrix, i.e., second-order joint probability function of image regions). Finally, T2w informs in Patient A on three brain regions seen with T1w but associated to totally different features, relatively to texture. In Patient B some similarities emerge from brain region overlaps with Patient A and previously informed by T1w postCA modality through texture-related features, but here additionally related to shape features (see Additional file: 1 ‘SM_ALLfeatures’ with all the listed features, and for the source details https://pyradiomics.readthedocs.io/en/latest/features.html).

Table 1 Biclusters from 2 patients across all MRI modalities combine brain regions and radiomic features. Note that for reasons of space only the data of patient A was previously visualized in Fig. 2

Discussion and concluding remarks

We have reviewed GBM and some emerging research topics that hopefully can advance the field in the immediate future. Among the identified needs that require pre-clinical investigation, there are integrative and multimodal imaging data approaches. These are especially associated in brain to imaging tools such as MRI that offer multiparametric solutions and interpretations.

As examples of emerging concepts falling into these new developments there are imaging habitats that capture tumor heterogeneity through differentiated TME features whose characterization translates into complex and highly contextualized data fusions. New targeted computational inferences are here needed.

Saliency maps were also addressed to assign phenotypic relevance to the brain regions and delineate predictively the features value for an assessment of the risk of relapse, again requiring consideration of contextual characteristics to be suitably represented in computational models.

We then addressed an interesting space, atlas-driven radiomics, which complements the traditional targeted radiomics. The latter is based on consolidated main steps which include building tumor segmentations, extracting features from ROI and classifying patients based on them. As an alternative, we showed that the use of brain atlases allows the design of interpretable templates that can become implementable tools depending on their definition and how their structures adapt to the various contexts, brain regions in our case.

Fig. 3

Top: Combined MRI modalities in Gauss filtered bicluster maps. Left: single modality. Centre: two combined modalities. Right: effects of modality subtraction (from two to one). Bottom: Visualization of differential effects induced by GBM in left and right hemispheres (N = 13 regions used to map) and by subtracting unsorted biclustering maps (rightmost plot). The MRI modality used is FLAIR

We illustrated the potential use of such templates with GBM patients, and we easily foresee their generalizability to other brain cancers, always with the possibility of visualizing parcellation maps through applied computational tools, i.e., biclustering or similar. Concerning the value of this type of analysis for clinical scopes, the associations between brain regions and radiomic features can become more significant with the scale of the study, in which case features distributions classified by brain regions and patients may become informative for stratification purposes. Finally, the characterization that the template maps offer is useful for monitoring the disease evolution, and in such regard, concepts like imaging habitats and saliency are extremely informative and deserve further investigation.