Spatial-temporal radiogenomics in predicting neoadjuvant chemotherapy efficacy for breast cancer: a comprehensive review

Abstract

Radiomics is undergoing a paradigm shift from single-omics to multi-omics, from single-temporal to multi-temporal analysis, and from global to subregional analysis. These transformations have shown great potential in addressing key challenges related to imaging changes before and after neoadjuvant chemotherapy (NAC) in breast cancer. Furthermore, radiomics has achieved remarkable progress in tasks such as exploring tumor heterogeneity and uncovering underlying biological mechanisms. Integrating imaging data with gene data offers novel perspectives for understanding imaging changes driven by specific genetic alterations. However, current radiomics studies on neoadjuvant chemotherapy for breast cancer have not yet achieved a close integration of imaging changes with underlying biological mechanisms. They are largely limited to simple associations between models and genomic data, without in-depth interpretation of the biological significance inherent in imaging features, which is essential to directly link these features with the dynamic progression of the disease. This review seeks to explore the spatial-temporal heterogeneity of imaging alterations observed during NAC for breast cancer, while assessing their biological implications using established analytical approaches. It highlights the distinct advantages of spatial-temporal radiomics in predictive model development and examines potential correlations between imaging dynamics and gene expression profiles before and after NAC. Additionally, we critically examines previous radiogenomics studies, providing theoretical insights into their limitations. Finally, the review proposes future directions and innovative approaches for applying spatial-temporal radiogenomics in NAC for breast cancer, serving as a valuable reference and roadmap for researchers and clinical practitioners in this field.

Introduction

The incidence of breast cancer has significantly increased over the past few decades, accounting for 11.6% of all newly diagnosed malignant tumors, which is the most common cancer among women and one of the leading causes of cancer-related mortality in this population about 6.9% [1]. Advances in diagnostic imaging, such as mammography, breast ultrasonography, and magnetic resonance imaging, have significantly improved early detection rates. Historically, imaging-based diagnosis has primarily relied on the clinical expertise of radiologists. However, this dependence on human judgment introduces variability in diagnostic accuracy. Factors such as economic disparities, geographical limitations, and differences in professional experience further exacerbate this variability [2, 3].

Despite advancements in imaging that have significantly improved early screening, a considerable proportion of patients are already at an advanced stage of disease at the time of diagnosis. Statistics show that approximately 5–10% of breast cancer patients present with metastatic disease at the time of their initial diagnosis [4]. For patients presenting with locally advanced disease at diagnosis, immediate surgery may not be feasible due to tumor size, nodal involvement, or other factors. Traditionally, these patients had limited treatment options and poorer prognoses. As clinical research evolved, it became increasingly clear that administering chemotherapy before surgery—known as neoadjuvant chemotherapy (NAC)—could downstage tumors, thereby enhancing the likelihood of successful surgical resection and improving long-term outcomes. Since the efficacy of NAC varies greatly among individuals, accurate early prediction of a patient’s response to NAC is therefore of critical clinical importance. It enables oncologists to tailor treatment strategies, optimize chemotherapy regimens, avoid unnecessary toxicity in non-responders, and ultimately improve survival outcomes. In this context, radiomics and artificial intelligence–driven models have emerged as promising tools to assist in such individualized decision-making. However, the inherent complexity and heterogeneity of breast cancer significantly challenge the accuracy of imaging-based assessments of NAC efficacy [5, 6]. Additionally, NAC induces dynamic changes in tumor heterogeneity, further altering the tumor’s imaging characteristics and complicating the evaluation process [7]. Given these challenges, artificial intelligence (AI) has gained growing attention as a promising tool for analyzing imaging data of malignant tumors, offering the potential to improve diagnostic accuracy and optimize therapeutic evaluations.

The integration of AI in breast cancer basic research and clinical practice encompasses various domains, including diagnosis, therapeutic response prediction, survival analysis, and image reconstruction [8,9,10]. To address this predictive challenge, the integration of advanced imaging biomarkers with machine learning has opened new frontiers. Magnetic Resonance Imaging (MRI), as a non-invasive modality providing radiation-free, high-resolution multiparametric visualization, including three-dimensional dynamic contrast enhancement and diffusion tensor imaging, has become indispensable for longitudinal monitoring of NAC-induced pathophysiological changes [11]. When coupled with radiomics– the automated extraction of high-dimensional quantitative features capturing subvisual tumor characteristics such as morphological shape irregularity, intratumoral texture heterogeneity, and peritumoral spiculation patterns [12, 13]. By applying AI technology, enable AI-driven predictive modeling beyond conventional RECIST criteria.

AI-based diagnostic models have demonstrated performance levels comparable to, and occasionally surpassing, those of experienced radiologists in assisting with diagnosis [14]. Breast cancer detection currently relies primarily on two main approaches. The first is traditional radiological assessment, which is based on visually identifiable features such as lesion shape, margin characteristics, aspect ratio, and the presence or absence of calcifications. These methods are straightforward, cost-effective, and applicable to a wide range of breast lesions. However, their accuracy can vary significantly depending on the radiologist’s expertise and experience, leading to potential false positives and false negatives [15]. With the advancement of technology, AI-assisted image analysis tools have increasingly been integrated into clinical practice. These tools, while promising, often rely on predefined features or direct pattern recognition, without fully capturing the underlying pathophysiological mechanisms. As a result, while AI has shown potential to improve consistency and sensitivity, many current models still lack interpretability and generalizability, especially across diverse patient populations and imaging modalities. To bridge this translational chasm, the convergence of radiophenotypes with histopathologic and genomics ground truth is gaining momentum. Pathological histology captures early cellular changes that radiomics cannot detect, revealing precancerous lesions and alterations in the tissue microenvironment. Genomics provides critical information at a higher level of detail, including nuclear morphology, stromal tissue distribution, angiogenesis, and molecular expression, effectively compensating for radiomics’ limitations in disease identification and quantification. By integrating radiomics-predicted tumor invasion patterns with genetic mutation analysis, the accuracy of tumor classification and the personalization of treatment strategies can be enhanced [16]. Genomics reveals genetic variations through whole-genome sequencing, deciphering the genetic drivers of diseases and laying the foundation for precision diagnosis and personalized treatment. By detecting gene mutations and epigenetic changes, genomics stratifies diseases into molecular subtypes, guiding therapeutic strategies. Furthermore, genomics delves into disease progression and treatment responses, such as tumor mutational burden and microsatellite instability, providing critical insights for immunotherapy and drug sensitivity [17] (Fig. 1).

Fig. 1

Workflow of mutiomics model. (A) Radiomics Workflow Diagram: A diagram illustrating the steps involved in radiomics analysis. (B) Pathomics Workflow Diagram: A diagram showing the process of pathomics analysis. (C) Genomics Workflow Diagram: A diagram outlining the steps in genomics data analysis. (D) Metadata Diagram: A diagram displaying the structure of associated metadata. (E) Multi-Omics Data Diagram: A diagram representing the integration of multi-omics data

This multidimensional analytical framework addresses a critical limitation of conventional single-modality approaches: the inability to resolve clonal evolutionary trajectories masked by bulk tumor sampling. The incorporation of multimodal data strategy enables a more comprehensive characterization of tumor heterogeneity, thereby enhancing the biological significance and interpretative power of radiomics models [18,19,20]. And this incorporation of multimodal data strategy enables a four-dimensional deconstruction of tumor heterogeneity. Intertumoral heterogeneity can manifest as spatial heterogeneity and temporal heterogeneity [21]. Temporal heterogeneity reflects the differences in the tumor before and after treatment, while spatial heterogeneity represents the heterogeneity of the tumor across different regions at a single point in time [22, 23]. These advancements not only refine the scope of radiomics but also position AI as a transformative tool in achieving biologically informed, clinically impactful outcomes in breast cancer management. Spatial-Temporal Radiogenomics integrates spatial-temporal imaging heterogeneity with genomic data, researchers have begun to investigate changes in gene expression before and after NAC, as well as the relationships between these changes and alterations in distinct tumor subregions. This approach establishes a robust explanatory framework for predicting the efficacy of NAC. A key application of combining genomic and imaging data is the identification of biomarkers. Genomic data provides insights into the molecular characteristics of tumors, while radiomics captures their spatial heterogeneity and phenotypic features. Through joint analyses of these datasets, researchers can uncover gene-imaging biomarkers associated with therapeutic responses or clinical prognoses.

Moreover, tumor subregions often exhibit inconsistent regression before and after treatment, with significant heterogeneity within the tumor leading to distinct differences among subregions, which in turn result in varied responses to therapy. Furthermore, traditional methods that rely solely on imaging features to infer molecular subtypes fall short in explaining the intricate interplay between genetic and imaging characteristics. This innovative framework integrates dynamic spatial-temporal imaging features with genomic data, correlating changes in gene expression before and after neoadjuvant chemotherapy with alterations in tumor subregions. This approach to interpret imaging changes in tumors after drug treatment, laying the foundation for improved insights and predictive models in tumor biology. To elaborate, there is typically 4–6 months interval between pre- and post-NAC imaging. During this period, tumors undergo dynamic changes in response to systemic therapy—this reflects temporal heterogeneity, referring to alterations in imaging features over time under therapeutic pressure. Additionally, following NAC, tumors often regress in irregular patterns or asymmetrically, suggesting that different regions within the tumor exhibit varied sensitivity to treatment. This is known as spatial heterogeneity. When combined, these dimensions—spatial-temporal heterogeneity—may provide valuable insight into tumor biology, therapeutic resistance, and treatment outcomes.

By focusing on this concept, we aim to bridge the gap between image-based phenotypic changes and their potential biological underpinnings, while evaluating how advanced AI-driven radiomic approaches can aid in uncovering these associations to improve clinical decision-making. In this review we present an overview of radiomics over the years (Fig. 2.) for studying the Spatial and Temporal heterogeneity on cancer.

Fig. 2

Timeline of Spatial-Temporal heterogeneity on radiomics and radiogenomics

Temporal heterogeneity in neoadjuvant treatment of breast cancer

Temporal heterogeneity of neoadjuvant therapy in magnetic resonance imaging

The evaluation of the response to NAC is conventionally performed through post-treatment imaging assessments. However, the clinically driven demand for early efficacy prediction can be addressed by employing radiomics models that analyze imaging data acquired before and after NAT. Temporal heterogeneity enriches the dataset and adds extra dimensions, reflecting dynamic changes in pathological tissues and allowing for an evaluation of physiological alterations in tumors induced by NAC [24] (Fig. 3). We have compiled a comparative summary of representative post-NAC MRI radiomics studies (Table 1). This table details each study’s cohort size, Dataset Num, radiomics modeling strategy and key performance metrics, thereby enabling a direct comparison of different approaches and highlighting emerging trends in early treatment-response prediction.

Fig. 3

The structure of Spatial-Temporal heterogeneity in radiomics. A. Spatial Heterogeneity: Tumor Subregions; B. Temporal Heterogeneity: Tumor Shrinkage Before and After Neoadjuvant Chemotherapy; C. Spatial-Temporal Heterogeneity: Four Response States of Tumors After Neoadjuvant Chemotherapy and Subregional Changes; D. Spatial-Temporal Radiomics-Genomics Integrated Model. Diagram D uses partial responders after neoadjuvant therapy from Diagram C as an example, integrating tumor subregional changes before and after neoadjuvant chemotherapy with genetic information to provide a biological interpretation of these changes. (Complete Response, CR; Partial Response, PR; Stable Disease, SD; Progressive Disease, PD)

Table 1 Comparison of Spatial-Temporal genomics research based on magnetic resonance imaging

Image acquire and time-series analysis method

Imaging data from different time points is typically divided into two categories: sequential enhancement within a single MRI scan [25] and imaging acquired at different therapy time points [26]. Specifically, breast MRI is typically performed at three critical time points: baseline, midway through treatment and preoperative. Each of these scans serves a distinct clinical purpose: the baseline MRI is used for initial tumor staging and surgical planning; the interim MRI can help assess early response and guide treatment adjustments; and the post-NAC MRI is essential for evaluating residual disease and determining the appropriate surgical strategy.

MRI acquisition follows standardized protocols, including high-resolution dynamic contrast-enhanced MRI (DCE-MRI) and diffusion-weighted imaging (DWI). DCE-MRI captures enhancement kinetics after intravenous contrast administration, providing detailed insights into tumor vascularity and morphology, while DWI assesses changes in cellular density by measuring water diffusion. These techniques together enable comprehensive, multiparametric assessment of tumor response over time.

Temporal heterogeneity of continuous imaging

Sequential enhancement within a single MRI scan captures tumor signal intensity through tissue enhancement dynamics before, during, and after contrast agent administration, reflecting a composite of tumor perfusion, vessel permeability, and the volume of the extravascular-extracellular space. By analyzing the exchange interactions between different compartments, it is possible to evaluate the perfusion and vascular permeability in the lesion area, ultimately inferring differences in the distribution and metabolism of substances between pathological and normal tissues [27]. The study analyzes changes in imaging features related to pathophysiological characteristics such as tumor infiltration, extramural vascular invasion, and lymph node involvement before and after NAC. Among patients achieving pathological complete response, these prominent imaging characteristics show a significant reduction, with a decrease of up to 88.9% observed within the same patient cohort [28]. These approaches collectively offer valuable insights into the dynamic processes underlying tumor progression and therapeutic response.

For analyzing short-term imaging changes, time-series radiomics [29] and dynamic radiomics feature extraction [30] are frequently employed. Francesco Prinzi et al. [29] conducted a comparative analysis of time series analysis methods, including Rocket, MultiRocket, Time Series Forest, Supervised Time Series Forest, and K-Nearest Neighbors with several distance metrics (Table 2).

Table 2 Time-series analysis method

These methods use imaging data collected at multiple, closely spaced time points. Unlike longitudinal analysis, time-series radiomics requires data from several intervals with relatively short gaps to track dynamic trends in tumor characteristics. DCE-MRI is one of the most used imaging techniques in this method. It enables imaging and analysis of microcirculation and perfusion in blood vessels through short-term sequential imaging [31]. In addition, compared to delta features, the dynamic information in DCE-MRI can reflect crucial temporal data regarding the directional flow of contrast agents. Based on this, it enables exploration of the relationship between dynamic information, prognosis, and tumor heterogeneity [32]. It is particularly effective in capturing continuous, nonlinear changes. Dynamic radiomics feature extraction, on the other hand, focuses on even shorter intervals, such as the temporal patterns of contrast agent uptake and blood flow. This method analyzes subtle variations in imaging signals, making it especially suitable for detecting rapid physiological changes in the tumor microenvironment.

Temporal heterogeneity of imaging at different treatment time points

Methods for capturing and quantifying tumor imaging differences before and after NAC often rely on longitudinal image analysis, which leverages imaging data from pre- and post-NAC stages to build efficacy prediction models and describe specific tumor characteristic changes through delta features generated by imaging differences [33]. This method focuses on point-by-point comparisons and is well-suited for studies involving a limited number of time points (e.g., before treatment, during treatment, and after treatment). Its primary advantage lies in its ability to effectively interpret significant changes over extended periods.

Other approaches include temporal dimension modeling with deep learning and multi-time-point feature fusion. Li et [34]. analyzes multi-temporal data using a single-temporal feature extraction network, a Co-attention module, and a GAN-based image generation network. The single-temporal network extracts unique features from the first and third temporal images, while the Co-attention module fuses features from different temporal images, capturing remote pixel dependencies to create hybrid feature maps. Saba et [35]. uses a many-to-one Gated Recurrent Unit (GRU) model to capture temporal relationships between feature vectors from four prior examinations. GRU addresses the vanishing gradient problem in standard RNNs using update and reset gates. It processes the four feature vectors (each size 1 × 4096) as a time series input and outputs a 1 × 128 feature vector, effectively capturing dynamic temporal relationships for improved diagnostic prediction. These advanced techniques integrate complex temporal data, offering innovative tools to enhance the understanding of tumor response and refine predictions in diverse clinical settings.

Image registration and temporal heterogeneity analysis in medical imaging

From a temporal perspective, MRI can accurately monitor the dynamic changes of tumors during NAC, specifically manifested as the gradual reduction in tumor volume, which represents a morphological response, changes in vascular permeability, with a decrease in Ktrans values indicating the effect of anti-angiogenic therapy, and changes in cell density [36]. An increase in ADC values reflects tumor cell necrosis or apoptosis [37]. These changes provide important information for evaluating treatment efficacy, predicting outcomes, and adjusting therapeutic strategies. For example, a reduction in tumor volume typically indicates cell death or suppression of proliferation, changes in vascular permeability reveal the effects of anti-angiogenic treatments on tumor blood supply, and changes in cell density reflect the cytological alterations within the tumor microenvironment, further revealing tumor response during the treatment process.

From a spatial perspective, MRI can reveal the heterogeneity patterns within different tumor regions, reflecting the varying sensitivities of the tumor to chemotherapy. Different regions of the tumor may respond differently to chemotherapy; some areas may be highly sensitive, while others may show resistance or minimal response. For example, DCE-MRI can detect changes in the uptake and washout patterns of contrast agents within tumor regions [38], highlighting areas with different perfusion and vascular permeability characteristics.

The combination of these imaging techniques allows MRI to provide a comprehensive evaluation of tumors from multiple angles, not only capturing the dynamic changes of the tumor during treatment but also revealing its spatial heterogeneity, thus offering crucial insights for personalized treatment.

Temporal heterogeneity based on image registration

The analysis of temporal heterogeneity in medical imaging leverages multi-time-point imaging data to reveal dynamic changes in pathological lesions, such as tumors, and plays a critical role in treatment monitoring and evaluation. In recent years, advancements in imaging technologies, computational power, and data processing algorithms have significantly expanded the use of temporal heterogeneity analysis in tumor radiomics. This approach goes beyond examining spatial morphological and structural differences, focusing on the temporal evolution of tumors to better capture disease progression and therapeutic responses. Key technological advancements in this field include enhanced image registration techniques, the integration of multi-modal fusion analysis, and the use of deep learning algorithms for temporal feature extraction. These innovations are driving the development of more sophisticated and precise methods for analyzing tumor dynamics and evaluating treatment efficacy over time, paving the way for improved clinical decision-making and personalized care.

To ensure reliable temporal analysis, the first step is spatially aligning multi-time-point images with high accuracy. Recent deep learning–based registration frameworks such as VoxelMorph and MIRNet have outperformed traditional rigid or elastic methods by learning complex local and global nonlinear deformations directly from large-scale training data [39, 40], thereby improving alignment in non-rigid scenarios. However, the soft, highly deformable nature of breast tissue and the need to reconcile multimodal imaging modalities introduce additional complexity. In response, the Learn2Reg challenge benchmarks few-shot learning, large-deformation estimation, and cross-modal alignment specifically on breast MRI tasks—fostering robust, generalizable registration solutions across varied clinical conditions [41]. Beyond these deep learning approaches, classic feature-based [42] and region-based registration [39] techniques further enrich the toolbox for achieving precise alignment, forming the cornerstone of accurate temporal heterogeneity analysis (Table 3).

Table 3 Image registration methods

Region-based registration and feature-based registration

Region-based registration methods do not rely on specific image feature points but instead use image intensity information or statistical properties directly, aligning images by optimizing similarity metrics such as mutual information or mean squared error. These methods are particularly suitable for images with sparse textures and multi-modal imaging scenarios, such as PET-CT registration. PET/CT registration demonstrates high adaptability and robustness in complex tasks, primarily due to the complementary nature of multimodal imaging: PET provides metabolic functional information, while CT offers high-resolution anatomical structures, supplying stable feature matching references for registration.

In comparison, feature-based registration methods achieve geometric alignment by identifying and matching distinctive feature points, such as corners and edges, across images. Techniques like Scale-Invariant Feature Transform (SIFT) [43] and Speeded-Up Robust Features (SURF) [44] are particularly effective for managing geometric transformations and are widely used for registering texture-rich single-modal images. These algorithms are not only robust in handling scale and rotation variations but are also computationally efficient, making them well-suited for large-scale image processing tasks.

On the other hand, region-based methods excel in scenarios such as medical or remote sensing image registration, where images may lack clear texture details or display substantial appearance variations due to differences in imaging modalities or sensors. In these cases, feature-based approaches often struggle to identify reliable matching points. However, region-based methods, which leverage statistical properties or intensity values for alignment, provide a robust alternative. Despite their strengths, these methods can face challenges when dealing with significant geometric distortions or highly localized deformations, which may require supplementary techniques or advanced pre-processing for improved performance.

While feature-based registration methods are highly effective, their performance is heavily reliant on the quality of feature point extraction. In scenarios with sparse textures or significant noise, these methods may struggle to identify reliable feature points, leading to reduced accuracy or even registration failure. This dependency on well-defined features makes them less suitable for images with low contrast or those affected by substantial variability, such as multi-modal medical imaging or heavily degraded datasets [39].

Multimodal image matching: bridging imaging gaps for interpretation

After receiving NAC, breast cancer patients often experience four distinct outcomes, each associated with specific imaging changes. Relying solely on imaging data from a single time point inevitably leads to the omission of critical patient information. Unfortunately, this missing information often represents the most significant aspect of imaging changes induced by NAC. Despite growing awareness of the importance of imaging changes in medical image research, existing studies on temporal heterogeneity primarily focus on differences in whole-tumor imaging, with limited attention to subregional changes. Understanding the impact of subregional heterogeneity on treatment efficacy and clarifying the underlying biological mechanisms will enable us to target specific treatment strategies for tumors with distinct heterogeneity, thereby achieving precision medicine.

Current methods focus on unimodal feature detection and description becomes insufficient when confronting the modality gap paradox– where MRI T2-weighted signals and ultrasound elastography measurements exhibit fundamentally different physical representations of the same tumor biology. This limitation is being overcome through three generations of multimodal matching evolution [45], has showcased remarkable potential by integrating information from diverse imaging modalities, correcting errors, enhancing details, and broadening the scope of applications. By incorporating unified frameworks, feedback mechanisms, and advanced deep learning models, multi-modal image matching not only improves accuracy in matching and data fusion but also facilitates the efficient and comprehensive use of multi-modal data. These methods include patch-based learning, Long Short-Term Memory (LSTM) networks, and Graph Neural Networks (GNNs), significantly enhance diagnostic precision by aligning data from different imaging modalities. These approaches enable a more comprehensive understanding of pathological conditions, facilitating the effective integration of information across modalities.​.

Convolutional neural networks and patch-level image matching

Based on image learning methods, convolutional neural networks (CNNs) are widely employed for extracting hierarchical latent features, measuring similarity, and estimating geometric relationships from images [39]. Within this framework, patch-based learning has emerged as a popular extension for region-level image registration and stereo matching [46]. Unlike traditional sliding window approaches, patch-based learning leverages deep learning techniques to streamline the similarity measurement process, significantly reducing computational complexity.

Multi-time-point dynamic feature extraction and temporal modeling

As the understanding of temporal heterogeneity deepens, the extraction of imaging features has expanded beyond static attributes. The focus now lies on analyzing the temporal evolution patterns of these features. State-of-the-art approaches integrate temporal modeling with multi-modal imaging data and enabling the extraction of dynamic features across multiple time points [47]. For instance, in DCE-MRI, researchers quantify hemodynamic properties by examining the rates of contrast agent uptake and washout within tumors [48, 49]. Temporal data modeling techniques, including LSTMs [50,51,52], have been extensively employed to discern dynamic patterns in temporal features. These models effectively capture morphological and functional changes in tumors both pre- and post-treatment, facilitating predictions about future tumor behavior and therapeutic outcomes.

Application of graph neural networks in tumor evolution and therapeutic response prediction

Moreover, recent research has investigated the application of GNNs for predicting responses to NAC [53, 54]. Unlike conventional artificial intelligence methods, which predominantly process Euclidean-structured data, GNNs are uniquely designed to handle graph-structured data. These networks learn features from both the nodes and edges within a graph, enabling the capture of intricate relationships and structural information among entities. By representing tumor imaging data collected at various time points as graph structures, GNNs can effectively model the dynamic interactions and evolutionary patterns of tumor cells. This innovative approach leverages the relational properties of graph data to enhance analytical accuracy, offering a powerful framework for understanding tumor dynamics and optimizing therapeutic strategies.

Transformer models: overcoming limitations of LSTM, GNN, and Patch-based learning in complex tasks

The methods mentioned earlier all have certain drawbacks. Traditional patch-based learning methods typically neglect global context and spatial information [55], limiting their performance in complex tasks. Long Short-Term Memory (LSTM) networks face challenges such as high computational complexity and limited ability to capture long-range dependencies when processing long sequence data [56,57,58]. Graph Neural Networks (GNNs), when handling graph-structured data, suffer from theoretical limitations in expressive power, particularly in graph isomorphism discrimination [59]. The Transformer model, through its self-attention mechanism and global context modeling capability [60], effectively addresses the challenges faced by traditional patch-based learning methods, LSTM, and GNNs in complex tasks. Through the self-attention mechanism, the Transformer not only captures global dependencies when processing image data, thereby effectively integrating global context information, but also handles long sequence data efficiently, capturing long-range dependencies and enhancing model performance [61, 62]. Furthermore, the Transformer is capable of capturing complex global relationships between nodes in a graph, providing stronger expressive power and overcoming the limitations of traditional GNNs in graph isomorphism discrimination [63].​.

Medical image segmentation and Spatial heterogeneity analysis in medical imaging

Medical image segmentation

Segmentation of conventional medical images

Region of Interest (ROI) segmentation is a critical step in radiomics analysis, as its accuracy directly affects the reliability of subsequent feature extraction and model evaluation. Radiomics segmentation methods are typically categorized into three main types: manual delineation, semi-automatic segmentation, and automatic segmentation.

Manual delineation offers high accuracy and is particularly effective for complex or irregularly shaped lesions. However, it is time-consuming and heavily dependent on the operator’s subjective judgment, resulting in variability between users. To address these challenges, semi-automatic methods, such as 3D Slicer [64], and fully automatic segmentation approaches [65] have been developed. These methods allow for the efficient processing of large-scale datasets while ensuring better standardization and reproducibility. Despite these advantages, their performance relies heavily on the quality of training data and can be limited when dealing with complex or low-contrast images.

In breast cancer imaging, the evaluation of segmentation algorithms is paramount and is based on rigorous benchmarks. The CBIS-DDSM dataset [66], comprising 2,620 pathologically confirmed digital mammograms with ROI labels, serves as the gold standard for mammography segmentation. Additionally, the BC-MRI-SEG dataset [67], which includes multi-center, multi-sequence breast MRI, offers both supervised and zero-shot evaluation tracks. This diversity in data enables the testing of deep learning models under varied clinical conditions, ensuring a comprehensive assessment of segmentation performance.

Once the ROI is accurately segmented, various features are extracted for further analysis. These features are typically categorized into three main types: shape features, texture features, and intensity features. Shape features describe the geometric structure of the ROI, providing metrics such as volume, surface area, and boundary complexity to characterize the tumor’s morphological attributes. Texture features, derived from techniques like the gray-level co-occurrence matrix (GLCM) and gray-level run-length matrix (GLRLM), capture the heterogeneity of tissue structures and are widely applied in tumor classification and prognosis analysis. Intensity features represent the distribution of gray values within the ROI, including metrics like mean gray level and standard deviation, which reflect the enhancement patterns and density variations of the lesion.

Tumor subregion construction methods

Effective clustering analysis in radiomics relies on rigorous benchmarking to ensure method validity across diverse scenarios. Two notable platforms exemplify this effort. Simulation-based Benchmarking in R generates 7,000 heterogeneous datasets spanning seven controlled scenarios—varying sample size, cluster count, variable types, noise level, and correlation—and integrates real EPHESUS clinical data to evaluate algorithms such as Kamila, latent class analysis, latent class model, and MixMod using metrics like the adjusted Rand index, concordance index, and log hazard ratio [68]. The HAWKS framework employs evolutionary algorithms to flexibly produce synthetic benchmarks of tunable difficulty, enabling targeted testing and fair, comprehensive comparison of clustering methods [69].

For tumor subregion imaging, clustering is then applied to group similar features from shape, texture, and intensity categories for deeper analysis. This step enables a more nuanced evaluation of tumor heterogeneity by isolating subregions with distinct characteristics. Table 4 provides an overview of commonly used clustering methods employed in this process, illustrating their utility in radiomics-driven investigations.

Table 4 Tumor subregion construction methods

The relationship between Spatial heterogeneity and tumor subregions

Temporal heterogeneity provides valuable insights into differences in tumor characteristics before and after NAC in breast cancer patients, it does not clarify changes in the tumor’s specific location. Significant changes in the tumor microenvironment and internal structure can occur before and after NAC, leading to the development of spatial heterogeneity [70]. Peritumoral edema, which arises in the surrounding tissues during tumor progression, represents a distinct pathological feature from the tumor’s imaging characteristics. Patients with peritumoral edema exhibit significantly higher rates of lymph vascular invasion (44.1% vs. 17.6%, P < 0.001), marked vessel ectasia (64.7% vs. 20.6%, P < 0.001), and moderate to severe stromal fibrosis (32.4% vs. 10.8%, P = 0.003) compared to those without peritumoral edema [71]. These spatial changes reflect the complexity of the tumor microenvironment, the heterogeneity of tumor cells, and their adaptive capabilities to environmental shifts, all of which are closely associated with tumor biology, therapeutic response, and prognosis [72].

Spatial heterogeneity in radiomics is closely related to tumor subregions. By identifying imaging features within distinct tumor subregions, researchers can gain a deeper understanding of tumor biology and treatment responses. Spatial heterogeneity refers to differences in imaging characteristics across various tumor locations, such as changes in cell density, blood supply, and metabolic states [73]. These differences result in the formation of multiple subregions within a tumor, each potentially possessing unique biological properties and varying sensitivity to treatment.

The potential biological significance of spatial heterogeneity

Spatial evaluation approaches in imaging

Texture-based spatial feature has shown excellent performance in assessing intratumoral heterogeneity from imaging, which may correlate with tumor biology and behavior. Spatial evaluation approaches in imaging are generally divided into two categories: spatial filtering and neighborhood-based methods. Spatial filtering enhances regions of interest by detecting areas with rapid gray-level changes, emphasizing edges and textures in the image. Spatial filters are particularly effective in enhancing spatial features of interest, such as edges and textures, within the image [74]. High-pass filters enhance local heterogeneity by highlighting rapid gray-level transitions, while low-pass filters smooth broader patterns to reveal macro-scale heterogeneity [75]. By enabling directional and scale-specific analysis, spatial filters capture heterogeneity across multiple orientations and granularities. Metrics derived from filtered outputs, such as variance, entropy, and gradient magnitudes, provide quantitative measures of spatial heterogeneity. The most commonly used neighborhood-based methods, such as the GLCM [76], local binary patterns (LBP) [77], and gradient magnitude analysis [78], are intrinsically linked to spatial heterogeneity as they quantify local variations in gray-level intensities within a specified spatial range. GLCM captures the directional and distance-based dependency of gray levels, revealing the texture heterogeneity within a region. LBP encodes the micro-level gray-level differences into binary patterns, effectively characterizing local irregularities and fine-scale heterogeneity. Gradient magnitude methods measure the intensity of gray-level transitions, highlighting abrupt changes and identifying boundaries in highly heterogeneous regions. These methods provide crucial tools for characterizing tumor heterogeneity, offering valuable insights into its biological behavior and therapeutic implications.

However, traditional image feature extraction methods, such as Spatial Filtering, GLCM, LBP, and Gradient Magnitude Analysis, have several limitations when processing complex images. Spatial filtering methods tend to cause the loss of image details, are sensitive to parameter adjustments, and lack adaptability, while traditional assumptions fail to handle non-stationary noise effectively. To address these issues, non-local means filtering and deep learning-based denoising methods can be employed. GLCM suffers from high computational complexity, a single texture scale, and sensitivity to rotation and lighting changes. Improvements include the use of multi-scale GLCM and rotation-invariant texture descriptors. LBP is sensitive to noise, has poor rotational invariance, and lacks global context. Improvements can include multi-scale LBP, GCN, and Vision Transformers. Gradient Magnitude Analysis faces issues such as noise interference, loss of directional information, and poor background adaptability. Improved approaches include combining directional consistency constraints with deep learning methods. These improvements aim to enhance the robustness and accuracy of image feature extraction, particularly in applications involving complex backgrounds and multi-scale analysis.

Histological heterogeneity in tumor subregion

The spatial heterogeneity of tumors, grounded in their internal microenvironment and structural characteristics, is elucidated through various radiomics methods. These approaches reveal regional differences in cell density, blood supply, metabolic state, morphological features, and immune microenvironment. Fusco et al. [79] demonstrated that combining morphological characteristics of breast lesions with dynamic information from contrast-enhanced scans could achieve a diagnostic accuracy of 91.7% in binary classification tasks.

Structural heterogeneity at the tissue level is commonly analyzed through image segmentation techniques and voxel, providing insights into cytological changes within tumors. In tissue structure analysis, images are typically decomposed into fundamental units, followed by identifying the rules required to reconstruct the image. For instance, fractal dimension (FD) provides an objective metric for evaluating the self-similarity of shapes by quantifying the relationship between image details and scale variations, assessing image complexity, and measuring uniformity within the region of interest through self-repeating structural patterns [80, 81]. This method provides a powerful tool for capturing and quantifying spatial patterns, advancing the understanding of tumor heterogeneity and its implications for diagnosis and treatment.

In subregion analysis, structural heterogeneity within tumors can be evaluated by analyzing the texture and signal intensity of distinct subregions, such as the core, margin, and necrotic zones [82, 83]. Fliedner et al. identified a significant correlation between ADC values and apoptotic status in the CT26 model (P = 0.0031). A strong correlation was observed between two measurements of ADC values and apoptotic status in both models, and this correlation also extended to the comparison between ADC values and cell density [84]. Cellular proliferation and death are significantly influenced by blood flow and oxygen supply, which can be characterized through dynamic contrast-enhanced imaging and blood oxygen level-dependent functional analysis. These techniques provide detailed insights into the hemodynamic properties and oxygenation status of different tumor regions [85, 86], facilitating an evaluation of the tumor’s resistance to therapy.

Blood and oxygen heterogeneity in tumor subregion

Heterogeneity in blood flow and oxygen supply reflects the complexity of the tumor microenvironment and is a valuable predictor of therapeutic response [86]. Metabolic heterogeneity, shaped by tissue oxygenation, can be assessed through PET-CT and magnetic resonance spectroscopy to analyze intratumoral metabolic activity [87,88,89,90]. Closely linked to genetic mutations, especially those involving abnormal activation of metabolic regulatory pathways such as PI3K/AKT/mTOR, metabolic imaging offers critical insights into potential genetic abnormalities [87].

Structural heterogeneity, characterized by spatial variations in cellular proliferation and necrosis, is closely linked to differences in blood flow and oxygen supply. These variations profoundly affect metabolic activity in tumor cells, creating a dynamic interplay between structural, blood flow, oxygen supply, and metabolic heterogeneities. Together, they underscore the intricate complexity of the tumor microenvironment and its influence on therapeutic response.

Immuno-microenvironment heterogeneity in tumor subregion

Immuno-microenvironment heterogeneity, characterized by the infiltration and spatial distribution of immune cells within tumors, can be inferred using multimodal imaging technologies and deep learning models. These insights are essential for understanding tumor responses to immunotherapy, particularly in relation to PD-L1 expression and immune checkpoint inhibitor-related gene expression [91, 92].

Morphological and structural heterogeneity in tumor subregion

Morphological and structural heterogeneity is primarily analyzed through shape analysis and three-dimensional reconstruction techniques. He et. demonstrates the effectiveness of machine learning models based on morphological MRI radiomics for classifying parotid gland tumors. Using a dataset of 298 patients, radiomic features were extracted and selected with Select K Best and LASSO algorithms, followed by the development of three-step models using XGBoost, SVM, and DT. Performance evaluation showed that XGBoost outperformed other models, achieving the highest AUC in most steps (up to 0.908 in training and 0.826 in testing), and its overall accuracy (70.8%) surpassed that of radiologists (49.2%) [93]. Yang et al. developed an MRI morphological feature model (MRI-MF) based on edge markers and peritumoral edema, combined with radiomic features extracted from post-contrast DCE-MRI images and deep learning methods. This approach achieved the best diagnostic performance (AUC = 0.857) for assessing lymph vascular invasion (LVI) status in breast cancer patients [94]. Chen et al. utilized MRI morphological features, including tumor number, tumor margin, and tumor necrosis, combined with radiomic features extracted from T2-weighted fat-suppressed imaging and diffusion-weighted imaging. By applying the LASSO method to generate Rad_Scores, they further enhanced the model’s performance. The resulting nomogram demonstrated excellent predictive capabilities in both the training group (MVI AUC = 0.874; tumor grading AUC = 0.827) and the validation group (MVI AUC = 0.869; tumor grading AUC = 0.848) [95]. These methods uncover insights into tumor growth patterns and spatial morphological changes, offering valuable indicators of tumor progression and potential predictive value for therapeutic outcomes.

Challenges in tumor imaging and future research directions

During the NAC process for breast cancer, the internal properties of the tumor often vary across different time points. Spatial heterogeneity is typically composed of multiple factors, including but not limited to tissue composition, structure, shape, blood flow, oxygen supply, and immune heterogeneity. However, most studies focus on clustering and analyzing tumor subregions without delving into how these heterogeneities reflect potential biological mechanisms in tumor imaging. Moreover, temporal and spatial heterogeneity are inherently interconnected. In fact, spatial heterogeneity in blood flow and oxygen supply is closely linked to treatment response. In tumors with good treatment response, the reduction in tumor size is often accompanied by decreased blood flow and oxygen supply, indicating simultaneous changes in temporal and spatial heterogeneity. Additionally, tumor shape, tissue composition, and structure undergo significant alterations before and after treatment. Therefore, disregarding any dimension of spatiotemporal heterogeneity and analyzing the remaining dimensions in isolation will inevitably result in the loss of critical information. When developing imaging models, it is crucial to emphasize the spatiotemporal heterogeneity of tumors and refine the underlying logical framework linking imaging features to biological mechanisms, thereby enhancing the model’s scientific rigor and clinical applicability.

Although the combination of spatial-temporal heterogeneity has, to some extent, elucidated the potential relationship between imaging changes before and after neoadjuvant chemotherapy and therapeutic outcomes, this insight remains at a macroscopic level and cannot be conclusively validated through well-established biological mechanisms. Therefore, future studies should incorporate genomics for in‑depth analyses, enabling the elucidation of fundamental biological mechanisms and their intrinsic links to therapeutic efficacy.

Spatial-temporal radiogenomics

Genomics for explaining tumor biological mechanisms

The advancement of genomics has established a robust foundation for understanding the genetic basis of diseases, uncovering individual genomic characteristics, and driving personalized medicine [96]. In recent years, breakthroughs in high-throughput sequencing technologies and computational biology have propelled genomics to the forefront of research on complex diseases such as cancer [97]. Techniques such as whole-genome sequencing [98], whole-exome sequencing [99], and single-cell genomic sequencing have enabled researchers to dissect tumor genomic heterogeneity, identify driver mutations, analyze copy number variations, and uncover gene expression regulatory mechanisms at multiple levels.

Genomic heterogeneity analysis primarily investigates genetic mutation differences both among individuals and within distinct cellular subpopulations in tumors. Tumor genomic heterogeneity is recognized as a key contributor to recurrence, metastasis, and treatment resistance [21, 100]. Recent advancements in single-cell genomics [101] have introduced transformative tools for studying tumor heterogeneity and evolutionary dynamics. For example, whole-genome sequencing of individual glioblastoma cells has revealed significant variations in EGFR copy number at the single-cell level. Moreover, the single-cell sequencing approach allowed researchers to deduce that the coexistence of two oncogenic EGFR variants encoding truncated EGFR forms in the bulk tumor was driven by distinct subclonal tumor cell populations harboring non-overlapping variants [102]. By revealing genetic mutation differences among tumor subpopulations, single cell sequencing sheds light on tumor evolution and resistance mechanisms. Studies tracking tumor evolution through serial biopsy samples have demonstrated that chemotherapy can reshape the molecular composition of tumors by altering their mutational spectrum. Mutations in key genes involved in replication and cell cycle regulation, particularly those contributing to genomic instability, play a pivotal role in this process [21]. For example, treatment of glioblastomas with temozolomide may lead to the accumulation of transition mutations in mismatch repair genes, resulting in the development of a hypermutated phenotype [103]. When applied to tumor samples collected at multiple time points, this technology allows researchers to trace genomic changes before and after treatment, providing critical insights into the spatial-temporal aspects of tumor heterogeneity.

Genomics serves as a powerful tool to decode the spatial-temporal complexities of tumor biology, paving the way for precision medicine strategies and improved therapeutic interventions.

Temporal heterogeneity and radiogenomics

Temporal heterogeneity facilitates dynamic observation of changes in cellular density and gene expression over time, offering critical temporal data for understanding cellular processes. Variations in blood flow and oxygen supply, influenced by both regional vascular differences and temporal changes, emphasize the dynamic complexity of the tumor microenvironment. By analyzing blood flow and oxygen supply across tumor regions at various time points, researchers can link cellular behaviors to angiogenesis-related genes, such as VEGF, and identify poorly responsive tumor regions [104,105,106]. Matsubayashi et al. [104] evaluated the relationship between early and delayed edge enhancement, as well as delayed internal enhancement, and histological features in 35 patients using subtraction imaging. Their findings demonstrated a significant association between early edge enhancement and high peripheral VEGF expression, along with a high peripheral-to-central microvascular density ratio. Julia et [106]. employed [18 F]-FMISO-PET/MRI to assess hypoxia indices in estrogen receptor-positive (ER+) breast cancer, focusing on their association with histological markers, including CD31, HIF-1α, and CAIX. The findings revealed a significant negative correlation between [18 F]-FMISO Ki and CAIX expression, vessel diameter, and microvascular density (MVD) (p = 0.002, 0.03, and 0.02, respectively). These results suggest that reduced blood flow and oxygen supply, as indicated by [18 F]-FMISO Ki, play a key role in promoting increased CAIX expression, further emphasizing the relationship between hypoxia and tumor vascular characteristics. These findings are vital for tailoring treatment strategies and optimizing therapeutic outcomes.

Spatial heterogeneity and radiogenomics

Links between metabolic heterogeneity and biological mechanisms

Metabolic heterogeneity reflects spatial-temporal variations within tumors, with metabolic characteristics such as glucose uptake and lactate production dynamically regulated over time and influenced by genetic mutations, particularly via the PI3K/AKT/mTOR pathway [107,108,109]. MRI enables spatial detection of metabolites, reflecting metabolic changes during therapy [110]. Metabolic imaging is a critical tool for identifying potential genetic abnormalities and serves as a valuable basis for personalized treatment planning, while also highlighting metabolic heterogeneity as a key indicator of tumor responsiveness to therapy [111, 112]. Genetic mutations are linked to tumor metabolic profiles, enabling adaptation or resistance to therapies [113]. This metabolic reprogramming is intertwined with endovascularization, which facilitates the transition of cancer cells from a proliferative to an invasive and migratory phenotype, often associated with epithelial-mesenchymal transition and partially regulated by TGFβ-dependent transcriptional changes [114]. This dynamic interplay between metabolic changes and genetic alterations underscores the importance of integrating metabolic imaging into therapeutic assessments, offering insights into tumor adaptation and strategies to overcome treatment resistance.

Links between morphological and structural heterogeneity and biological mechanisms

Morphological and structural heterogeneity, assessed through three-dimensional reconstruction and shape analysis, reveals temporal changes in geometric features of tumors, such as volume growth, boundary irregularities, and rates of volumetric change. Although the relationship between such spatial-temporal heterogeneity and genetic mutations is generally weaker, certain morphological characteristics are strongly associated with specific genetic alterations. For instance, He et al. showed that AEBP1 expression is associated with smaller nuclear size and lower nuclear density (P < 0.01) [115]. Acinar and papillary growth patterns are indicative of EGFR mutations, whereas solid growth patterns and large nuclei suggest TP53 mutations [79, 116]. These findings highlight the importance of temporal changes in morphological features as reflections of tumor growth patterns driven by specific genetic mutations.

Links between immune heterogeneity and biological mechanisms

Tumor heterogeneity is often associated with immune-related genes. Thorsson et al., through immunogenomic analysis, identified that TGF-β dominance is associated with immune characteristics characterized by a balanced distribution of both type I and type II T cells [117, 118]. Radiogenomics integrates imaging data with immune-related gene expressions, such as PD-L1 and other immune checkpoint genes, to predict tumor responsiveness to immunotherapy by tracking immune cell dynamics over time [119, 120]. This approach provides a comprehensive framework for assessing tumor immunoreactivity, offering critical insights into tumor-immune dynamics.

Radiogenomics integrates spatial heterogeneity features derived from radiomics with genomic analyses, providing deeper insights into the molecular mechanisms underlying tumor growth, metabolism, and immune response. Structural heterogeneity reveals variations in cellular density and proliferation linked to mutations in cell cycle-regulating genes, while vascular heterogeneity, reflected in blood flow and oxygen supply variations, correlates with angiogenesis-related mutations. Metabolic heterogeneity exposes abnormalities in tumor metabolic pathways, particularly involving the PI3K/AKT/mTOR signaling cascade. Immune microenvironment heterogeneity, assessed through advanced imaging and its association with immune-related genes, enhances the precision of evaluating tumor responses to immunotherapy.

Spatial-Temporal heterogeneity and radiogenomics

Radiogenomics integrates spatial-temporal heterogeneity captured through radiomics with genomic data, enabling a deeper mechanistic analysis of their interaction. By incorporating imaging features from different types of data—such as cellular density, blood supply, metabolic states, and the immune microenvironment—genomics establishes connections with tumor genetic mutations and molecular characteristics. The combined analysis of genomic data and radiomics across multiple temporal phases provides a comprehensive understanding of the relationship between tumor microenvironment changes and genetic abnormalities. Cellular density, texture, and signal intensity within tumors are closely associated with changes in gene expression. Changes in cell density induce mechanical deformation of cells and nuclei, activating protective mechanisms and leading to DNA damage and nuclear envelope rupture, which release cytoplasmic DNA. This triggers the cGAS-STING signaling pathway, altering gene expression and promoting malignant traits such as epithelial-mesenchymal plasticity and chemotherapy resistance. These findings highlight that cell density fluctuations influence cancer invasiveness through gene regulation [121]. Subtle grayscale texture variations in foundational imaging features may correlate with genetic and phenotypic differences [122, 123], particularly in genes regulating cell proliferation and apoptosis, such as TP53 and RB1 [124]. Over time, these spatial variations evolve, driving distinct tumor biological behaviors at different stages.

Zhou et [125]. constructed an imaging-genomics model by integrating the variant allele frequency (VAF) of the REL and MED23 genes from genomic data with imaging model features. In the validation cohort, the model achieved an AUC of 0.93, significantly outperforming the imaging model alone (AUC = 0.86). Kaplan-Meier analysis showed that the disease-free survival (DFS) in the pCR prediction group was significantly better than in the non-pCR group (P = 0.034, 0.001, and 0.019), highlighting the potential of the imaging-genomics model in prognostic assessment. However, their study only established the model by incorporating gene-related features, without investigating the relationship between imaging and genomic data. Huang et [126]. explored the relationship between spatial habitat radiomics features and the tumor immune microenvironment, revealing the biological basis of the multimodal model through single-cell RNA-seq analysis. Using k-means clustering and the Calinski-Harabasz index, tumors were divided into three subregions: the high metabolic subregion (Region I), the junction subregion (Region II), and the marginal subregion (Region III). Heatmap analysis showed significant correlations between B cells and multiple imaging features (such as Rad5, Rad6, Rad9, Rad10), especially in patients with high Clin-SHR scores, where B cell infiltration significantly increased. B cell proportions rose from 0.28 to 1.49% in the low Clin-SHR group to 14.55–42.65% in the high Clin-SHR group. Gene Ontology enrichment analysis revealed that immune response-related pathways, including the B cell receptor signaling pathway, were significantly enriched in the high Clin-SHR group, emphasizing the role of B cells in the tumor immune microenvironment. The study found that patients with high Clin-SHR scores exhibited higher B cell infiltration and better treatment responses, suggesting that a B cell-driven immune microenvironment may play a pivotal role in tumor response to NAC. This approach offers a robust framework for evaluating tumor biological alterations before and after NAC, shedding light on the dynamic processes of tumor adaptation and response.

This integrated framework of radiogenomics offers a holistic molecular and imaging-based approach for personalized treatment. By combining insights from structural, vascular, metabolic, and immune heterogeneities, it enables more accurate predictions of therapeutic responses and supports the development of optimized treatment strategies, ultimately improving patient outcomes.

Multiomics data and potential interconnections

Radiogenomics has found significant application in using imaging to predict the mutation status of genes. Previous research has demonstrated its strong performance in assessing correlations between imaging features and the molecular subtypes of breast cancer [127,128,129,130]. Moreover, radiogenomics has shown remarkable success in evaluating relationships between imaging features and genetic mutations [131,132,133,134,135,136]. For instance, Ji et al. identified associations between features extracted from DCE-MRI and the molecular classification of breast cancer. This underscores the pivotal role of radiogenomics in enhancing the accuracy of breast cancer diagnosis and classification [137].

As radiogenomics continues to evolve, a key research focus has emerged: decoding the biological mechanisms underlying newly identified imaging biomarkers. Establishing robust associations between imaging and genomic features is essential to uncovering the processes that drive these biomarkers. For example, Lai et [138]. employed univariate logistic regression to analyze correlations between differential gene expression levels and axillary lymph node status, excluding highly correlated genes and ultimately identifying 16 potential predictive genes. Similarly, Yu et [139]. applied t-tests to identify differentially expressed genes linked to imaging features, uncovering connections between MRI radiomics and tumor microenvironment characteristics. These included immune cell profiles, long non-coding RNAs, and methylation site types, illustrating the complex interplay between imaging phenotypes and molecular biology. In another study, Shaveta et [140]. investigated the relationships between gene expression scores, DCE-MRI parameters such as Ktrans, clinical variables, and immunohistochemistry (IHC) parameter scores. Using Spearman analysis for continuous variables and Kruskal-Wallis tests for categorical variables, they identified significant differences in gene expression between responders and non-responders. Among responders, genes involved in angiogenesis and extracellular matrix pathways, such as VEGF and ECM-related genes, were notably downregulated, indicating that effective treatment reduced angiogenesis. Conversely, non-responders exhibited activation of axon guidance and mTOR signaling pathways, suggesting potential mechanisms underlying treatment resistance. These findings underscore the ability of Ktrans changes to reflect not only treatment efficacy but also the molecular mechanisms at play. By linking imaging parameters with gene expression profiles, the study highlights the potential of radiogenomics to provide insights into the biological processes driving tumor responses, paving the way for more targeted and effective therapeutic interventions. Fan et [141]. conducted a genomic analysis to identify the top 2,000 most variable genes, excluding those with low expression or minimal changes, to investigate their association with treatment response. Functional roles of these significant genes and their potential impact on treatment outcomes were further analyzed using Gene Set Enrichment Analysis and KEGG pathway evaluations. Regression models established clear relationships between gene expression patterns and tumor shrinkage, offering valuable insights into treatment dynamics.

Addressing gaps and future directions in radiogenomics

The exploration of metabolism, morphology, structure, immunity, and spatial-temporal heterogeneity has gradually become a focal point of attention among researchers. However, current methodologies primarily focus on whole-tumor imaging features, often overlooking the spatial-temporal heterogeneity within tumors driven by genomic variations. To address this gap, integrating multidimensional data from spatial-temporal radiomics, genomics, and pathology could allow for a more comprehensive evaluation of the effects of NAC. This systematic framework for genomic-radiomic integration not only enhances the application of multimodal data in precision medicine but also fosters innovation by providing deeper insights into the biological and clinical implications of integrated data.

However, the existing high-quality studies typically establish only one-dimensional relationships between macro-level imaging features and micro-level pathology or genetics. A critical gap remains: while genes are often used to explain outcomes, the intermediate processes that lead to these outcomes are still poorly understood and remain opaque.

In two other studies [142, 143], researchers conducted reverse validation of radiogenomics using animal models, further strengthening the rigor and reliability of the findings. Current investigations in this domain are sparse, underscoring the tremendous opportunities for progress in this promising yet largely untapped field. The integration of spatial-temporal heterogeneity with radiogenomics, coupled with validation through animal experiments, presents a promising approach to address this challenge. By systematically interpreting the temporal and spatial heterogeneity within tumor imaging, this review could help explain the progression from underlying biological processes to clinical outcomes, thereby enhancing the interpretability and predictive power of tumor models.

Spatial-temporal radiogenomics: opportunities and challenges

Spatial-temporal radiogenomics uniquely bridges the gap between imaging changes and biological mechanisms, granting imaging features a deeper biological interpretability. Tumor development is inherently dynamic, with breast cancer undergoing continuous changes in imaging characteristics throughout its progression—from diagnosis to treatment. These changes encompass alterations in shape, texture, size, contour, and heterogeneity, reflecting the evolving biology of the tumor.

Imaging changes observed before and after NAC often correlate with therapeutic efficacy, which is heavily influenced by variations in gene expression (Fig. 2). Tumors responding to treatment may exhibit reductions in size or heterogeneity, while non-responders may display distinct patterns of resistance-driven changes. Despite these associations, effectively integrating the spatial-temporal dynamics of imaging features with genomic information remains a critical challenge.

Addressing this question requires the development of advanced computational frameworks capable of capturing the interplay between imaging phenotypes and underlying genomic variations over time. Such frameworks could provide deeper insights into treatment response mechanisms, enhancing the precision of radiogenomic analysis and supporting the design of personalized therapeutic strategies. Future research must focus on the integration of longitudinal imaging and multi-omics data to unravel these complex relationships and maximize the clinical utility of radiogenomics.

The main challenges facing the integration of radiogenomics in clinical practice are as follows:

The challenges spatio-temporal radiogenomics

High cost of acquiring paired genomic data

Obtaining genomic data paired with imaging remains prohibitively expensive. Consequently, most radiogenomics studies are limited to single-center investigations with small sample sizes, often lacking adequate external validation. To address this limitation, future research should emphasize standardized, large-scale, prospective studies involving multi-center patient cohorts. This approach would improve the generalizability of findings and establish a more robust foundation for clinical applications.

Time-consuming and subjective nature of radiomics research

Current radiomics studies frequently depend on manual image segmentation, which is not only time-intensive but also susceptible to operator variability, potentially undermining the reproducibility and validity of models. To overcome this challenge, future research must focus on the development of automated, efficient, and reliable segmentation techniques. Such advancements would significantly enhance the clinical applicability of radiomics by improving accuracy, reducing variability, and increasing efficiency.

Limited exploration of biological mechanisms

​Many radiomics and radiogenomics studies have primarily focused on correlating imaging features with genomic data, often lacking a deeper exploration into the biological mechanisms driving these associations. This limitation hinders our understanding of how imaging phenotypes reflect underlying tumor biology. To address this gap, future research should integrate imaging data with functional biological studies, utilizing animal models to validate radiomic features and elucidate their biological significance. For instance, preclinical studies using murine models have demonstrated that radiomic features derived from imaging modalities like MRI and PET can correlate with gene expression profiles, providing insights into tumor heterogeneity and response to therapy. By combining imaging data with molecular and histological analyses in animal models, researchers can identify specific biological processes, such as angiogenesis or immune cell infiltration, that correspond to radiomic features. This integrative approach not only enhances the interpretability of radiomic data but also facilitates the development of imaging biomarkers that are biologically relevant and clinically applicable. Ultimately, bridging the gap between imaging, genomics, and functional biology through the use of animal models will pave the way for more precise and personalized therapeutic strategies in oncology [144].

Addressing these challenges will require a collaborative and multidisciplinary effort, combining advancements in imaging, genomics, computational biology, and clinical research. By overcoming these barriers, radiogenomics can fulfill its potential as a transformative tool in precision medicine, paving the way for deeper insights into cancer biology and more effective patient care.

Future outlook

Radiomics allows for the high-throughput extraction of multidimensional features from tumors, while genomics reveals the molecular characteristics and genetic mutation patterns of the tumor. Technological innovations are expected to bring breakthroughs in this field. Spatial transcriptomics enables the precise localization of gene expression changes within tumor tissues and provides spatial references for imaging data, greatly enhancing our understanding of the tumor microenvironment and treatment responses. Single-cell multi-omics is promising for revealing tumor cell heterogeneity, which could help predict the early emergence of drug-resistant cells. When combined with dynamic MRI monitoring, it allows for more precise therapeutic interventions. Additionally, the integration of explainable deep learning and GNN enables the deep fusion of multimodal imaging and genomic data on a single platform. Through attention mechanisms, the model’s interpretability is improved, making it easier for clinicians to trust AI-driven predictions. To promote this process, federated learning holds significant promise for multi-center data sharing, enabling large-scale data collaborative modeling while protecting patient privacy, thus enhancing the generalization capabilities of models.

In the future, the combination of digital twin technology and virtual clinical trials will make treatment plan selection more scientific. By constructing digital models of patients, different treatment strategies can be simulated in a virtual environment to predict drug responses and adverse effects, optimizing personalized treatment plans. Real-time liquid biopsy, in conjunction with radiomics, offers a new approach for dynamic tumor monitoring, allowing for the timely detection of disease progression or signs of resistance, thus supporting the timely adjustment of NAC treatment.

Looking ahead, the integration of radiomics, pathology, and genomics to model and analyze patient response is poised to become a cornerstone of cancer research and clinical practice. These complementary approaches address distinct informational gaps—particularly by illuminating the underlying mechanisms driving imaging changes—and thereby accelerate the clinical translation of AI models. However, despite the promise that AI may one day outperform radiologists, several challenges must be acknowledged. First, AI performance critically depends on the quality and diversity of training data: while models excel on common lesions, they can misdiagnose rare cases, necessitating rigorous validation to ensure broad applicability. Second, radiologists incorporate not only imaging data but also clinical context and patient history—judgment that AI cannot yet fully replicate in complex cases. Third, AI’s “black-box” nature remains a barrier: clinicians require transparent, interpretable decision processes before trusting AI outputs. Therefore, the most effective approach currently is to use AI as a decision-support tool, rather than replacing human expertise [145].

To bolster AI credibility, we recommend training on representative, diverse datasets [146]. Commonly, data are split 80∶20 into training and validation sets—or 80∶10∶10 when external validation is available—but limited datasets may warrant 70∶30 splits [147] or cross-validation strategies [33, 148]. Small, imbalanced datasets significantly increase the risk of overfitting [149], which can be detected by a large discrepancy between training and validation errors [150]. To mitigate overfitting, methods such as cross-validation, dataset augmentation, and regularization should be employed [150, 151]. Finally, evaluating model performance across multiple, independent datasets provides a more comprehensive assessment, thereby enhancing trustworthiness and promoting generalizability in both clinical and real-world settings [152].

Conclusion

This review analyzes the role of temporal and spatial heterogeneity in artificial intelligence-assisted prediction of neoadjuvant chemotherapy efficacy. We summarize the current research methods for analyzing time series and constructing tumor subregions, focusing on the temporal heterogeneity of imaging changes before and after neoadjuvant treatment and their associated biological mechanisms. Furthermore, we discuss the spatial heterogeneity and its biological significance revealed by research hotspots such as tumor subregion analysis. Based on this, we explore the potential of integrating spatial-temporal heterogeneity with imaging genomics to enhance the interpretability of radiomics models. In the future, a deeper exploration of the biological mechanisms underlying pre- and post-treatment imaging changes and the biological significance of imaging variations in patients with different treatment responses will contribute to more precise individualized treatment and promote the clinical application of artificial intelligence models.