Unveiling the potential of artificial intelligence in revolutionizing disease diagnosis and prediction: a comprehensive review of machine learning and deep learning approaches

Abstract

The rapid advancement of Machine Learning (ML) and Deep Learning (DL) technologies has revolutionized healthcare, particularly in the domains of disease prediction and diagnosis. This study provides a comprehensive review of ML and DL applications across sixteen diverse diseases, synthesizing findings from research conducted between 2015 and 2024. We explore these technologies’ methodologies, effectiveness, and clinical outcomes, highlighting their transformative potential in healthcare settings. Although ML and DL demonstrate remarkable accuracy and efficiency in disease prediction and diagnosis, challenges including quality of data, interpretability of models, and their integration into clinical workflows remain significant barriers. By evaluating advanced approaches and their outcomes, this review not only underscores the current capabilities of ML and DL but also identifies key areas for future research. Ultimately, this work aims to serve as a roadmap for advancing healthcare practices, enhancing clinical decision making, and strengthening patient outcomes through the effective and responsible implementation of AI-driven technologies.

Graphical Abstract

Introduction

Healthcare is entering a transformative phase, where the vast volume of medical data holds significant promise and challenges for advancing healthcare research. The healthcare industry generates enormous amounts of data through Electronic Health Records (EHRs), medical imaging, genetic information, and clinical documentation. However, examining the relationships between different pieces of data is important for creating reliable medical tools through data-driven methodologies. In today’s rapidly evolving healthcare environment, leveraging these methodologies has become a powerful tool to revolutionize healthcare delivery. integrating advanced analytics, artificial intelligence (AI), and specifically machine learning (ML) and deep learning (dL) enables healthcare professionals to make informed, data-driven decisions, improve patient outcomes, streamline clinical workflows, and ultimately transform healthcare systems to deliver higher-quality care and improved public health outcomes [1,2,3,4].

Recently, ML and DL have emerged as transformative forces within healthcare, with applications spanning various critical areas, such as medical imaging for accurate diagnosis, personalized treatment recommendations, drug discovery, and improving operational efficiency in healthcare facilities. Moreover, wearable devices and predictive analytics have facilitated remote monitoring of patients and early disease detection [5,6,7] (Fig. 1).

Fig. 1

Various applications of ML and DL in healthcare

Although ML and DL have demonstrated remarkable potential across healthcare domains, their significance in early disease diagnosis and prediction has garnered particular attention due to their ability to revolutionize diagnostic processes [8, 9]. Disease diagnosis and prediction involve assessing symptoms, conducting tests, and analyzing patient data to predict the likelihood of a disease. ML algorithms are particularly valuable in predicting the risk of developing certain diseases based on the medical history, genetic information, and other relevant factors. However, challenges remain in feature selection and data analysis, especially when ground truth data is needed to identify anomalies effectively in medical data [10,11,12].

On the other hand, DL, a subset of ML, excels in modeling complex relationships and extracting high-level features from raw data using multilayer neural networks. DL models are especially effective in dealing with unstructured or complex data, making them highly suitable for tasks, such as medical imaging and disease prediction. Despite this, DL approaches have not been widely tested across a diverse range of medical conditions that could benefit from their advanced capabilities. A major difference between ML and DL is the level of feature engineering required, with DL bypassing the need for manual extraction thanks to its ability to learn hierarchical representations from data. Although DL is particularly effective in sophisticated pattern recognition and large-scale data processing, ML methods offer flexibility and interpretability, making them applicable to a broader range of healthcare tasks [1, 13,14,15].

The potential of ML and DL in disease prediction and diagnosis is profound, not only enhancing the accuracy and efficiency of healthcare delivery but also advancing precision medicine and population health management. To accelerate progress, researchers must address challenges, such as data sparsity, noise, heterogeneity, and time dependency, as well as develop methods to better integrate DL into clinical workflows and decision support systems [16,17,18,19].

• –Motivation

This paper is motivated by the increasing significance of ML and DL in healthcare. Healthcare organizations and academic institutions are swiftly recognizing that these technologies have the potential to refine patient outcomes, diagnoses, and healthcare delivery. The availability of extensive healthcare data and advancements in computing capabilities have also facilitated the integration of ML and DL across various healthcare domains, especially early disease diagnosis and prediction. Accordingly, we decided to discuss recent and upcoming uses of ML and DL in early disease diagnosis and prediction as well as emphasizing the critical elements that will have a substantial impact on healthcare. It is believed that a thorough examination of the utilization of these methods in healthcare will offer insights into their potential benefits and influence on the industry.

• –Objectives

The advancements in health systems enabled by ML and DL have facilitated the transformation of traditional clinical diagnostics, leading to refined patient outcomes and decreased healthcare costs. Although ML and DL offer promising avenues for forecasting medical information for disease diagnosis and prediction, there remain gaps in standard coverage. The key contributions of this research study include addressing the challenges and proposing innovative approaches to enhance disease prediction and diagnosis in healthcare settings. The objectives of this study can be summarized as follows:

• 1. To review and analyze the existing literature and research studies on the use of ML and DL for early disease diagnosis and prediction.

• 2. To identify the different types of diseases for which ML and DL have been applied for early diagnosis and prediction.

• 3. To evaluate the performance metrics, such as accuracy, sensitivity, and specificity, of ML and DL models in early disease diagnosis and prediction compared to traditional methods.

• 4. To assess the impact of ML and DL on the early detection of diseases in terms of patient outcomes, treatment efficacy, and healthcare cost savings.

• 5. To investigate the challenges and limitations related to implementing ML and DL in early disease diagnosis and prediction, such as data availability, model interpretability, and ethical considerations.

• 6. To provide insights and recommendations for the direction of future research and practical applications of ML and DL in improving early disease diagnosis and prediction in clinical practice.

Noteworthy, this review distinguishes itself from the existing surveys through several unique contributions. First, we provide a comprehensive analysis of sixteen diverse diseases, offering a broader and more holistic understanding of ML and DL applications across various healthcare contexts, unlike surveys that focus on a limited set of diseases or specific domains. Second, our study specifically examines research from 2015 to 2024, ensuring coverage of the most recent advancements and trends in ML and DL for disease diagnosis and prediction. Third, we synthesize not only the technical methodologies but also critically evaluate their clinical outcomes and effectiveness, bridging the gap between algorithmic innovation and real-world impact. In addition, we systematically identify key challenges, such as quality of data, interpretabilityof models, and clinical workflow integration, while proposing actionable future research directions often overlooked in similar reviews. Finally, our work emphasizes the responsible and effective implementation of AI-driven technologies, providing a roadmap for advancing healthcare practices, enhancing clinical decision making, and improving patient outcomes—a perspective rarely addressed in the existing surveys.

The remainder of this paper is structured as follows: Sect. “Background” offers an overview of the foundational concepts of ML and DL, along with a comparative analysis of them. The details of various data types that can be employed for disease diagnosis and prediction are mentioned in Sect. “Data types”. Sect. “Toward disease prediction and diagnosis” summarizes research on disease diagnosis and prediction using ML and DL as well as offering a detailed comparative analysis and covering the key implications and limitations of the study. Sect. “Challenges and open issues” also addresses challenges and unresolved issues within the field. Finally, Sect. “Conclusion and future works” presents the conclusions drawn regarding disease diagnosis and prediction from the study.

Background

This section covers the foundational concepts necessary for understanding this review. In both ML and DL, the process typically begins with clearly defining the problem and gathering relevant data. Although the data are collected, it must undergo preprocessing and cleaning to guarantee its quality and reliability. Then, features are selected or engineered from the raw data to be used for training the model. The next step involves choosing an appropriate model or algorithm, followed by training and evaluating the model using a validation set. Based on the evaluation results, the model is fine-tuned to improve its performance. Once a satisfactory model is obtained, it can be deployed to make predictions on new data. For DL, additional considerations specific to neural networks, such as selecting the right architecture and dealing with challenges like overfitting, also come into play [16, 20]. Steps in developing models using ML and DL are shown in Fig. 2.

Fig. 2

Steps in developing models using ML and DL

Machine learning

ML has become a significant tool with various applications in healthcare. By leveraging algorithms and statistical models to analyze and interpret data, ML holds the potential to revolutionize healthcare, particularly in the areas of disease diagnosis and prediction. The three main categories of ML algorithms are supervised learning, unsupervised learning, and reinforcement learning.[21]. The classification of ML models is provided in Fig. 3.

Fig. 3

Classification of machine learning models

In supervised learning, the algorithm learns to make predictions or decisions using labeled data. During the training phase, both the input data (features) and their corresponding output labels are provided. The algorithm learns the relationship between input and output by minimizing a predefined loss function. Common techniques in supervised learning include Logistic Regression (LR), Decision Trees (DT), Support Vector Machine (SVM), and Random Forests (RF). In contrast, unsupervised learning works with unlabeled data, requiring the algorithm to identify patterns and structures without explicit instructions. Typical tasks in unsupervised learning include clustering, dimensionality reduction, and association rule learning. Clustering algorithms including K-means and hierarchical clustering, organize similar data points into groups, whereas dimensionality reduction methods, like Principal Component Analysis (PCA), work to reduce the number of features in a dataset while preserving essential information. Reinforcement learning is another form of ML where an agent learns to make decisions by interacting with its environment. The agent receives feedback in the form of rewards or penalties depending on its actions, aiming to maximize cumulative rewards over time. Notable reinforcement learning algorithms include Q-learning, Deep Q-Networks (DQN), and policy gradients. These different types of ML algorithms are designed for different learning scenarios and tasks, enabling a broad range of applications across sectors, particularly in healthcare [22, 23].

In summary, ML plays a significant role in disease diagnosis and prediction by leveraging algorithms to analyze large and diverse healthcare datasets, encompassing patient records, genetic information, medical imaging, and more. Through supervised learning, ML models can be trained to recognize patterns and associations indicative of various diseases, allowing for accurate diagnosis and risk assessment. Furthermore, unsupervised learning techniques can uncover hidden structures within medical data, potentially revealing novel insights into disease mechanisms and patient subgroups. By integrating ML into clinical workflows, healthcare practitioners can benefit from predictive models that aid in early disease detection, personalized treatment planning, and prognostic assessments, ultimately leading to enhanced patient outcomes and the progression of precision medicine [23, 24].

Deep learning

DL aims to mimic the human brain’s ability to process data and make sense of the world. At its core, DL involves training complex neural networks—inspired by the interconnected structure of neurons in the brain—to learn from large amounts of data and make accurate predictions or decisions without explicit programming. The tremendous success of DL in recent years can be attributed to its capacity to automatically discover intricate patterns and representations within data, ranging from images and speech to text and sensor readings. This has resulted in significant advancements across multiple domains, including healthcare [2].

In recent times, DL has gained widespread acceptance in various healthcare sectors, encompassing disease diagnosis and forecasting. Techniques such as Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), and Recurrent Neural Networks (RNNs) have been applied for disease prognosis and diagnosis. Figure 4 illustrates the representation of DL models utilized in this context [23].

Fig. 4

Classification of deep learning models

As can be seen, a multitude of DL structures and techniques are commonly applied for disease prognosis and diagnosis. CNN architectures stand out as one of the most frequently utilized designs, particularly beneficial for processing image-based data such as medical imaging. CNNs can leverage different levels of abstraction to extract crucial information from images, aiding in disease diagnosis and prediction. RNNs represent another commonly employed model for disease diagnosis, particularly effective for analyzing time-series data like Electrocardiogram (ECG) data. RNNs excel in capturing temporal dependencies within the data, which can be utilized for disease diagnosis and prediction. Apart from CNNs and RNNs, various other DL architectures and methods are frequently utilized for disease detection and prediction, including Generative Adversarial Networks (GANs), Deep Belief Networks (DBNs), and auto encoders [23, 25].

DL holds immense potential for disease diagnosis and prediction, especially in the analysis of intricate medical data, such as imaging scans, genetic information, and patient records. With its capability to automatically extract intricate patterns and representations from raw data, DL models, especially CNNs and RNNs, can effectively identify subtle biomarkers, anomalies, and disease indicators from medical images, genetic sequences, and clinical notes. By learning from large, diverse datasets, DL models can uncover latent patterns that may elude human experts, leading to earlier and more accurate disease diagnoses, as well as personalized risk assessments for patients. The application of DL in healthcare not only enhances diagnostic accuracy but also holds promise for predicting disease progression, and treatment outcomes, and even contributing to the discovery of novel therapeutic targets, thus playing a pivotal role in advancing precision medicine and improving patient care [3].

Transfer learning

Transfer learning is a ML approach in which a model developed for one task is adapted or reused for a different but related task. In this approach, the knowledge gained while solving the source task is leveraged to help solve the target task, typically requiring less data and time compared to training from scratch. Transfer learning has gained widespread attention and application in various domains, especially in the field of DL [26].

The main idea behind transfer learning is that features learned from one problem domain can be useful for solving a different but related problem. By leveraging a pretrained model as a foundation, practitioners can refine or adjust the model for their specific needs, particularly when the target dataset is limited or computational resources for training are constrained. In the context of DL, transfer learning often involves using a pretrained neural network, such as those trained for image classification tasks on large-scale datasets like ImageNet, and then adapting it for a different task, such as object detection or medical image analysis. Transfer learning offers several advantages, including reduced data requirements, faster training, and improved generalization. Accordingly, transfer learning has proved to be a valuable tool for practitioners and researchers looking to develop effective ML models with constrained resources especially in healthcare [26, 27].

Transfer learning can be also a powerful tool for disease diagnosis and prediction, especially in medical imaging tasks such as identifying tumors from medical scans. By leveraging pretrained DL models that have learned rich visual features from large-scale image datasets, medical practitioners and researchers can adapt these models to new medical imaging datasets with relatively small amounts of labeled data. This approach enables the development of accurate diagnostic and predictive models for various diseases, potentially leading to earlier detection, personalized treatment plans, and refined patient outcomes. In addition, transfer learning can expedite the deployment of reliable disease diagnosis and prediction systems by decreasing the need for computational resources and training data, ultimately contributing to advancements in healthcare and medical research [28].

Ensemble learning

Ensemble learning is a powerful ML technique that involves combining multiple individual models to create a more robust predictive model. By using the diversity of multiple models’ predictions, ensemble methods aim to improve predictive accuracy, generalization, and robustness over single models. Ensemble learning can take various forms, including but not limited to bagging, boosting, and stacking, each with its unique approach to combining models. This approach is widely used in diverse areas of ML, including classification, regression, and anomaly detection, and has been instrumental in winning various ML competitions and improving the overall performance of predictive models. By harnessing the collective wisdom of multiple models, ensemble learning offers a powerful and versatile approach to tackling complex real-world challenges in predictive modeling and decision making [29].

Ensemble learning can significantly enhance disease diagnosis and prediction by integrating the expertise of multiple predictive models, each with its strengths and biases. In the context of healthcare, ensemble methods can amalgamate diverse data sources, like medical imaging, genetic markers, clinical data, and patient histories, to build more accurate and reliable diagnostic and predictive models. By combining the outputs of multiple models, ensemble learning can mitigate individual model weaknesses, improve diagnostic precision, and provide more robust predictions, ultimately leading to better-informed clinical decision making. In addition, ensemble learning’s ability to capture complex interactions within heterogeneous healthcare data contributes to the advancement of personalized medicine, aiding in early disease detection, treatment planning, and patient outcomes. As a result, ensemble learning serves as a valuable tool for improving the effectiveness and reliability of disease diagnosis and prediction in clinical practice [23, 29].

Comparison

ML, DL, transfer learning, and ensemble-based models are all fundamental techniques in the field of AI and have distinct attributes that make them suitable for different types of problems and data. There are significant distinctions between traditional ML and DL that are summarized in Table 1. In traditional ML workflows, there is a manual process of feature extraction or engineering, followed by the utilization of ML algorithms with relatively shallow structures, ultimately leading to the desired output. On the other hand, in DL workflows, an artificial neural network (ANN) is employed, which is capable of integrating feature extraction and classification within a single step of its algorithm, enabling an end-to-end learning process, as depicted in Fig. 5. As a result, DL necessitates less domain-specific knowledge to address the given problem. However, comprehending DL can be more challenging due to its algorithms being predominantly self-directed, often described as “black box” systems. In comparison, traditional ML is relatively straightforward to train and test, but its performance is contingent upon the quality of its features and becomes constrained as the volume of data increases, as indicated in Fig. 6. In addition, these relatively shallow models are less efficient, demanding a substantial number of computations and significant maintenance, notably reliant on considerable human effort for data labeling. Conversely, the performance of DL can steadily advance as the volume of data grows or as the network’s capacity increases. Although DL is capable of learning representations of high-level features, it requires a significant amount of data for training and can involve substantial computational costs. It is worth noting that, despite the growing availability of data, human performance remains consistent or may even decrease due to factors such as fatigue.

Table 1 ML versus DL

Fig. 5

Workflow of ML versus DL

Fig. 6

Performance comparison of DL, ML, and human [23]

Table 2 also indicates a brief comparison of the characteristics, applicability, and use cases of ML, DL, transfer learning, and ensemble-based models. When considering the diverse array of ML and DL tools, each of them has specific applications with their respective advantages and drawbacks in healthcare. These components are detailed in Table 3.

Table 2 Brief comparison of the characteristics, applicability, and use cases of ML, DL, transfer learning, and ensemble-based models

Table 3 Summary of various kinds of learning and their application in healthcare

Data types

Datasets in healthcare encompass a wide range of data types and formats, covering various aspects of healthcare and medical research. These datasets are crucial for training and evaluating ML and DL models, conducting epidemiological studies, and advancing medical knowledge [30]. Some common types of datasets in this field include:

• Electronic health records (EHR): EHR datasets contain comprehensive records of patient health information, including demographics, medical history, diagnoses, medications, laboratory test results, and clinical notes. These datasets are valuable for clinical research, predictive modeling, and population health management.

• Medical imaging datasets: Medical imaging datasets consist of images from modalities, such as X-ray, computed tomography (CT), magnetic resonance imaging (MRI), ultrasound, and positron emission tomography (PET). These datasets are used for developing and evaluating algorithms for image segmentation, classification, detection, and reconstruction in fields like radiology, pathology, and cardiology.

• Genomic datasets: Genomic datasets contain genetic information, including DNA sequences, gene expression profiles, single nucleotide polymorphisms (SNPs), and epigenetic modifications. These datasets are utilized for studying genetic diseases, population genetics, personalized medicine, and drug discovery.

• Drug datasets: Drug datasets include information on pharmaceuticals, such as chemical structures, pharmacokinetics, pharmacodynamics, indications, adverse effects, and interactions. These datasets are employed for drug repurposing, target identification, toxicity prediction, and pharmacovigilance.

• Clinical trials datasets: Clinical trial datasets comprise data from randomized controlled trials (RCTs) and observational studies, including patient demographics, treatment regimens, outcomes, and adverse events. These datasets are critical for evaluating the safety and efficacy of interventions, assessing treatment effectiveness, and informing clinical guidelines.

• Public health datasets: Public health datasets contain information on disease surveillance, outbreaks, epidemiological studies, population demographics, environmental factors, and social determinants of health. These datasets are used for disease prevention, health policy development, and public health research.

• Healthcare claims and billing datasets: Healthcare claims datasets include information on medical procedures, diagnoses, treatments, and reimbursements obtained from insurance claims and billing records. These datasets are utilized for healthcare utilization analysis, cost-effectiveness studies, and health services research.

• Physiological signal datasets: Physiological signal datasets comprise data from biosensors, wearable devices, and physiological monitors, such as electrocardiography (ECG), electromyography (EMG), electroencephalography (EEG), and respiratory rate monitors. These datasets are valuable for monitoring patient health status, detecting anomalies, and assessing physiological responses.

These are just a few examples of the diverse range of medical datasets available. Each dataset serves specific purposes and contributes to various areas of medical research, clinical practice, and healthcare delivery.

Toward disease prediction and diagnosis

In recent years, there has been a significant increase in research and development efforts aimed at utilizing ML and DL for disease prediction and diagnosis. By examining large volumes of medical data, these algorithms can identify patterns and trends that might be overlooked by human healthcare professionals. This enables earlier detection of diseases, personalized treatment plans, and improved patient outcomes. This section encompassed a comprehensive review of studies conducted between 2015 and 2024 that explored the application of ML and DL in a diverse array of diseases for diagnostic and predictive purposes. This inclusive approach allowed us to gain insights into the vast potential of these technologies across various medical domains, ranging from cancer and cardiovascular diseases to infectious diseases and neurological disorders. The breadth of diseases covered and the advancements observed reaffirm the continued growth and innovation within this exciting field of research.

In conducting this review, we utilized a range of scientific catalogs and databases, including PubMed, IEEE Xplore, SpringerLink, ScienceDirect, and Google Scholar, which are widely recognized for their comprehensive coverage of peer-reviewed research in healthcare, machine learning, and deep learning. Additionally, we included proceedings from top-tier conferences such as NeurIPS, ICML, CVPR, and ACM SIGKDD, as these are leading venues for cutting-edge advancements in AI and ML applications in healthcare. These sources were chosen for their relevance, reliability, and broad coverage of both clinical and technical literature, ensuring that our review captures a diverse and up-to-date range of studies. Searches were conducted using specific keywords and Boolean operators to identify studies focusing on ML, DL, and their applications in disease diagnosis and prediction besides the names of diseases that are covered in this study. To further enhance clarity, we have included a flowchart (Fig. 7) illustrating the process of study selection, from initial database searches to the final inclusion of papers.

Fig. 7

Flow diagram of the paper selection process

Studies were included if they met the following criteria: (1) publication between 2015 and 2024 to ensure coverage of recent advancements; (2) peer-reviewed status, including journal articles, conference papers, and reputable reviews, to maintain academic rigor; (3) explicit focus on the application of ML or DL for diagnosis and predictionof 16 various disease that are covered in this study; (4) demonstration of significant clinical outcomes, technical innovation, or practical applicability in healthcare settings; and (5) availability of publicly accessible datasets or detailed descriptions of data sources to ensure reproducibility and transparency.

Studies were excluded based on the following criteria: (1) lack of relevance to ML or DL applications in disease diagnosis or prediction; (2) nonpeer-reviewed sources such as editorials, opinion pieces, or nonacademic articles; (3) insufficient methodological or clinical detail, which could hinder the reproducibility or validation of findings; (4) duplicate or overlapping studies to avoid redundancy; and (5) non-English publications to ensure consistency in interpretation and avoid potential translation biases.

Disease-focused review of machine learning and deep learning

Cardiovascular disease

ML and DL play an important role in cardiovascular disease prediction and diagnosis by offering several key advantages. These advanced algorithms can efficiently analyze vast amounts of patient data, including genetic information, medical imaging, clinical records, and lifestyle factors, to specify patterns and nuances that may elude traditional diagnostic methods. By leveraging these insights, healthcare providers can predict the likelihood of cardiovascular events with greater accuracy, enabling proactive interventions and personalized treatment strategies. Additionally, machine learning can help streamline the diagnostic process, leading to quicker and more precise identification of cardiovascular diseases, such as arrhythmias, heart failure, and atherosclerosis. The integration of AI in cardiovascular care not only improves diagnostic accuracy but also empowers healthcare professionals to deliver targeted interventions and improve patient outcomes in a timely manner. Embracing ML and DL in cardiovascular disease management represents a significant step towards more effective healthcare practices and better heart health for individuals worldwide. A summary of the existing works related to ML and DL techniques for cardiovascular disease prediction and diagnosis is provided in Table 4.

Table 4 A summary of the existing works related to ML and DL techniques for cardiovascular disease prediction and diagnosis

Brain tumor

In brain tumor management, ML and DL play a critical role in improving various aspects of patient care and represent a groundbreaking approach in neuro-oncology that holds tremendous promise for enhancing patient care and outcomes. Over the past decade, a burgeoning body of research has explored the application of ML and DL in analyzing complex neuroimaging data, genetic markers, and clinical variables to revolutionize the detection and characterization of brain tumors. This section summarizes conducted studies that delve into the intersection of machine learning, deep learning, and brain tumor management which can provide insights into the current landscape of brain tumor prediction and diagnosis, highlighting the advancements and challenges in leveraging cutting-edge artificial intelligence tools in the realm of neuro-oncology. A summary of the existing works related to ML and DL techniques for brain tumor prediction and diagnosis is provided in Table 5.

Table 5 A summary of the existing works related to ML and DL techniques for brain tumor prediction and diagnosis

Diabetes

The utilization of ML and DL in diabetes prediction and diagnosis has opened up new horizons in the realm of personalized healthcare and disease management. With the prevalence of diabetes on the rise globally, there is an urgent need for innovative approaches to identify at-risk individuals, facilitate early intervention, and optimize treatment strategies. Over the past decade, a growing body of research has leveraged ML and DL to analyze diverse data sources, including clinical parameters, genetic information, and lifestyle factors, to enhance the accuracy and efficiency of diabetes prediction models. This section explores the extensive body of literature focusing on the impact of ML and DL in improving the early detection of diabetes patients. A summary of the existing works related to ML and DL techniques for diabetes prediction and diagnosis is provided in Table 6.

Table 6 A summary of existing works related to ML and DL techniques for diabetes prediction and diagnosis

Alzheimer’s disease

The use of ML and DL in Alzheimer’s prediction and diagnosis marks a significant advancement in the field of neurodegenerative diseases, offering hope for early intervention and improved patient outcomes. With the prevalence of Alzheimer’s disease expected to rise exponentially in the coming years, there is a pressing need for innovative approaches to accurately detect and monitor this debilitating condition. Over the past decade, researchers have increasingly turned to artificial intelligence to analyze complex biological markers, neuroimaging data, and clinical variables to develop predictive models and diagnostic tools. A summary of the existing works related to ML and DL techniques for Alzheimer’s disease prediction and diagnosis is provided in Table 7.

Table 7 A summary of existing works related to ML and DL techniques for Alzheimer’s disease prediction and diagnosis

Parkinson’s disease

ML and DL in Parkinson’s disease prediction and diagnosis represent a significant advancement in the field of neurodegenerative disorders, promising earlier detection and tailored management strategies. As Parkinson’s disease poses a growing burden on global healthcare systems, there is an increasing demand for innovative approaches to accurately identify and monitor the progression of this complex condition. Over the past decade, researchers have turned to artificial intelligence to analyze diverse datasets encompassing clinical assessments, neuroimaging scans, and genetic profiles, aiming to develop robust predictive models and diagnostic tools. A summary of the existing works related to ML and DL techniques for Parkinson’s disease prediction and diagnosis is provided in Table 8.

Table 8 A summary of existing works related to ML and DL techniques for Parkinson’s disease prediction and diagnosis

Gastrointestinal disease

ML and DL have also been extensively used in gastrointestinal disease prediction and diagnosis and represent a groundbreaking frontier in gastroenterology, offering new avenues for enhanced patient care and disease management. Gastrointestinal disorders encompass a wide spectrum of conditions, ranging from inflammatory bowel diseases to gastrointestinal cancers, presenting unique challenges for accurate diagnosis and treatment. Over the past decade, researchers have increasingly turned to artificial intelligence to analyze diverse datasets, including clinical symptoms, endoscopic imaging, histopathological findings, and genetic markers, with the aim of developing predictive models and diagnostic tools tailored to individual patients. A summary of the existing works related to ML and DL techniques for gastrointestinal disease prediction and diagnosis is provided in Table 9.

Table 9  A summary of existing works related to ML and DL techniques for gastrointestinal disease prediction and diagnosis

Kidney disease

The utilization of ML and DL in kidney disease prediction and diagnosis marks a significant advancement in nephrology, offering promising avenues for early detection and personalized treatment strategies. Chronic kidney disease (CKD) and related complications impose a substantial burden on global healthcare systems, underscoring the critical need for innovative approaches to improve patient outcomes. Over the past decade, researchers have increasingly turned to artificial intelligence to analyze diverse datasets, including clinical parameters, biomarkers, medical imaging, and genetic profiles, to develop predictive models and diagnostic tools for various kidney-related conditions. A summary of the existing works related to ML and DL techniques for kidney disease prediction and diagnosis is provided in Table 10.

Table 10 A summary of existing works related to ML and DL techniques for kidney disease prediction and diagnosis

Lung disease

The utilization of ML and DL in lung disease prediction and diagnosis represents a groundbreaking stride in respiratory healthcare, offering a potential paradigm shift towards earlier detection and more precise management strategies. Lung diseases, ranging from Chronic Obstructive Pulmonary Disease (COPD) to lung cancer, present significant challenges to public health globally, necessitating innovative approaches to enhance diagnosis and treatment outcomes. Over recent years, the advent of artificial intelligence has empowered researchers and clinicians to analyze diverse datasets encompassing clinical data, radiological imaging, and genetic information, to develop robust predictive models and diagnostic tools. A summary of the existing works related to ML and DL techniques for lung disease prediction and diagnosis is provided in Table 11.

Table 11 A summary of existing works related to ML and DL techniques for lung disease prediction and diagnosis

Liver disease

The employment of ML and DL in liver disease prediction and diagnosis heralds a promising era in hepatology, offering novel opportunities for early detection and tailored treatment strategies. Liver diseases, spanning from fatty liver disease to hepatocellular carcinoma, pose significant health challenges globally, necessitating innovative approaches to improve patient outcomes. In recent years, artificial intelligence has emerged as a powerful tool for analyzing complex datasets comprising clinical parameters, imaging modalities, biomarkers, and genetic profiles, with the goal of developing accurate predictive models and diagnostic algorithms. A summary of the existing works related to ML and DL techniques for lung disease prediction and diagnosis is provided in Table 12.

Table 12 A summary of existing works related to ML and DL techniques for liver disease prediction and diagnosis

Hepatitis

In the realm of hepatitis management, ML and DL serve as indispensable tools across various pivotal aspects. They enable swift and precise diagnosis by meticulously analyzing patient data, facilitating timely intervention and treatment initiation crucial for impeding disease progression. Furthermore, ML and DL optimize treatment strategies by tailoring therapy to individual patient characteristics, ensuring efficacy while minimizing adverse effects. They also provide invaluable prognostic insights, aiding clinicians in formulating treatment plans and implementing preventive measures. Moreover, ML and DL contribute to public health surveillance by detecting and predicting outbreaks, and informing targeted interventions to mitigate disease spread. A summary of the existing works related to ML and DL techniques for hepatitis prediction and diagnosis is provided in Table 13.

Table 13 A summary of existing works related to ML and DL techniques for hepatitis prediction and diagnosis

Dental disease

Dental diseases encompass a range of conditions affecting the teeth, gums, and oral cavity, including cavities, gum disease, oral infections, and oral cancer. ML and DL are increasingly being integrated into dentistry to improve the diagnosis, treatment, and prevention of these diseases. ML and DL help analyze various data sources, such as dental images, patient records, and genetic information, to assist in early detection and accurate diagnosis of dental diseases. These algorithms can detect subtle abnormalities, predict disease progression, and help personalize treatment plans based on the individual patient characteristics. Additionally, ML and DL can aid in dental imaging analysis, orthodontic treatment planning, and even robotic-assisted dental surgeries, enhancing precision and efficiency in dental procedures. A summary of the existing works related to ML and DL techniques for hepatitis prediction and diagnosis is provided in Table 14.

Table 14 A summary of existing works related to ML and DL techniques for dental disease prediction and diagnosis

Ophthalmic disease

Ophthalmic diseases encompass a wide range of conditions affecting the eyes, from common refractive errors like myopia and astigmatism to more serious conditions such as glaucoma, macular degeneration, and diabetic retinopathy. ML and DL are increasingly being integrated into ophthalmic care to improve the diagnosis, treatment, and management of these diseases. ML and DL help analyze various data sources including medical images, patient records, and genetic information to assist in early detection and accurate diagnosis of ophthalmic diseases. These algorithms can also predict disease progression, assess treatment efficacy, and personalize treatment plans based on the individual patient characteristics. In addition, AI-powered teleophthalmology platforms enable remote screening and monitoring, expanding access to eye care, especially in underserved areas. As ML and DL continue to advance, their role in ophthalmic disease management is expected to grow, contributing to improved patient outcomes and vision health globally. A summary of the existing works related to ML and DL techniques for hepatitis prediction and diagnosis is provided in Table 15.

Table 15 A summary of existing works related to ML and DL techniques for ophthalmic disease prediction and diagnosis

Skin disease

ML and DL are advancing skin disease management across several critical domains. They analyze medical images to aid in precise diagnosis and classification of various skin conditions, supporting dermatologists in accurately identifying diseases from melanoma to eczema. Leveraging patient data and clinical guidelines, ML and DL can recommend personalized treatment plans, considering factors such as disease severity and patient preferences. In addition, they enable monitoring and tracking of disease progression, detecting subtle changes and facilitating timely interventions. Telemedicine platforms powered by ML and DL extend dermatological care to remote or underserved areas, allowing for remote consultations and diagnosis based on the uploaded skin lesion images. Furthermore, Ml and DL contribute to skin disease research and drug development by analyzing biological data, expediting the discovery of novel treatments and improving patient outcomes. A summary of the existing works related to ML and DL techniques for hepatitis prediction and diagnosis is provided in Table 16.

Table 16 A summary of existing works related to ML and DL techniques for skin disease prediction and diagnosis

Plastic surgery

In plastic surgery, ML and DL are pivotal in revolutionizing various aspects of patient care. They facilitate preoperative planning by allowing surgeons to simulate surgical outcomes and analyze anatomical features, ensuring optimal results and patient satisfaction. Additionally, ML and DL enable precise measurements and assessments of facial and body features, aiding surgeons in tailoring procedures to individual patient characteristics. Moreover, they can be used to predict and optimize scar formation, as well as ensuring timely interventions and improved postoperative outcomes. A summary of the existing works related to ML and DL techniques used in plastic surgery is provided in Table 17.

Table 17 A summary of existing works related to ML and DL techniques in plastic surgery

Mental illnesses

The Utilization of ML and DL in mental disease prediction and diagnosis marks a pivotal advancement in psychiatry, offering new avenues for early intervention and personalized treatment strategies. Mental disorders, ranging from depression and anxiety to schizophrenia and bipolar disorder, present complex diagnostic challenges and profound impacts on individuals’ well-being and society at large. In recent years, there has been a growing recognition of the potential of artificial intelligence to analyze diverse datasets, including behavioral patterns, genetic markers, neuroimaging scans, and electronic health records, to develop predictive models and diagnostic tools. A summary of the existing works related to ML and DL techniques for mental disease prediction and diagnosis is provided in Table 18.

Table 18 A summary of existing works related to ML and DL techniques in mental illness prediction and diagnosis

Thrombosis

The utilization of ML and DL in thrombosis prediction and diagnosis represents a groundbreaking frontier in cardiovascular medicine, offering transformative prospects for early risk assessment and targeted therapeutic interventions. Thrombotic disorders, including deep vein thrombosis and pulmonary embolism, present substantial challenges in both acute management and long-term prevention, underscoring the critical need for innovative approaches to enhance patient care. In recent years, the application of ML and DL have gained momentum as a powerful tool for analyzing diverse data sources, such as clinical indicators, imaging studies, genetic markers, and lifestyle factors, with the aim of developing accurate predictive models and diagnostic algorithms. A summary of the existing works related to ML and DL techniques for thrombosis prediction and diagnosis is provided in Table 19.

Table 19 A summary of existing works related to ML and DL techniques in thrombosis prediction and diagnosis

Real-life applications

The integration of ML and DL into healthcare has led to transformative real-life applications across the 16 diseases covered in this review. In cardiovascular diseases, ML models analyze EHRs and wearable data to predict heart disease and detect thrombosis, enabling early interventions. For instance, ML algorithms have been integrated into platforms like the Framingham Heart Study to enhance cardiovascular risk prediction, improving preventive care [189]. In brain tumor diagnosis, DL models, particularly CNNs, segment and classify tumors in MRI and CT scans with high accuracy, assisting radiologists in diagnosis and treatment planning [190]. For diabetes management, ML algorithms predict blood sugar levels using data from glucose monitors and wearable devices, enabling personalized care through platforms like Livongo and DreaMed Diabetes [191]. In Alzheimer’s disease, DL models analyze MRI and PET scans to detect early signs of neurodegeneration, with tools like NeuroLex Laboratories aiding in early diagnosis and monitoring [192]. Similarly, for Parkinson’s disease, ML algorithms analyze gait patterns, voice recordings, and wearable sensor data to diagnose and monitor disease progression, improving patient management [193].

In gastrointestinal diseases, DL models assist in detecting colorectal cancer from endoscopic images, with studies demonstrating high accuracy in polyp detection and classification [194]. For kidney diseases, ML algorithms predict the progression of chronic kidney disease (CKD) using patient data, enabling early interventions and personalized treatment plans [195]. In lung diseases, DL models analyze chest X-rays and CT scans to detect lung cancer and pneumonia, with tools like Zebra Medical Vision improving diagnostic accuracy [196]. For liver diseases, DL aids in diagnosing liver cancer and cirrhosis through medical imaging, while ML optimizes treatment plans for conditions like hepatitis B and C [197]. In dental diseases, DL models analyze X-rays to detect cavities, periodontal disease, and other conditions, improving diagnostic precision [133]. For skin diseases, DL models like Google’s DeepMind detect melanoma and other dermatological conditions from images, enabling early treatment and reducing diagnostic errors [198].

In plastic surgery, ML algorithms predict surgical outcomes and optimize treatment plans, improving patient satisfaction and reducing complications [199]. For mental health disorders, ML analyzes behavioral and physiological data from wearable devices and EHRs to predict depression, anxiety, and other conditions, enabling early interventions [200]. In thrombosis, DL models analyze medical imaging data, such as CT scans, to detect blood clots and predict the risk of complications, improving patient outcomes [201]. Finally, in ophthalmic diseases, ML and DL have revolutionized the diagnosis and management of conditions like diabetic retinopathy, age-related macular degeneration (AMD), and glaucoma. For example, Google’s DeepMind developed an AI system that analyzes retinal images to detect diabetic retinopathy with a level of accuracy comparable to human experts, enabling early intervention and preventing vision loss [140]. Similarly, DL models have been used to diagnose AMD by analyzing optical coherence tomography (OCT) scans, with studies demonstrating high sensitivity and specificity in detecting early-stage AMD [202]. For glaucoma, ML algorithms analyze visual field tests and retinal nerve fiber layer (RNFL) thickness measurements to predict disease progression and guide treatment decisions [203].

Discussion

The rapid advancement of ML and DL technologies has revolutionized disease diagnosis and prediction across a wide range of medical conditions. This review has systematically examined the applications of ML and DL in 16 diverse diseases, highlighting their methodologies, effectiveness, and clinical outcomes. In this section, we compare the various ML and DL approaches reviewed in this study, discuss their strengths and limitations, and identify key areas for future research.

Generally, traditional ML algorithms, such as LR, SVM, and RF, are computationally efficient, interpretable, and well-suited for structured data. For example, LR has been widely used in cardiovascular disease prediction due to its simplicity and ability to handle risk factor analysis [7, 14]. Similarly, RF has shown high accuracy in predicting chronic kidney disease (CKD) progression by analyzing EHRs [97]. Traditional ML models often require extensive feature engineering and may struggle with unstructured data, such as medical images or free-text clinical notes. Their performance is also highly dependent on the quality and quantity of the input data.

On the other hand, DL models, particularly CNN and RNN, excel in handling unstructured data, such as medical images, time-series data, and natural language. For instance, CNNs have achieved state-of-the-art performance in diagnosing brain tumors from MRI scans [50] and detecting diabetic retinopathy from retinal images [54]. RNNs, including LSTM networks, have been effective in predicting disease progression in conditions like Parkinson’s disease using wearable sensor data [79]. However, DL models are computationally intensive, require large amounts of labeled data, and are often considered “black boxes” due to their lack of interpretability. These challenges can hinder their adoption in clinical settings, where transparency and explainability are critical. To provide a better comparison, the advantages and disadvantages of the prominent models introduced in previous sections are summarized in Table 20.

Table 20 Advantages and disadvantages of ML and DL models

Key implications

The findings from this comprehensive review highlight the transformative role of ML and DL in disease diagnosis and prediction. The implications of these advancements are profound, offering significant opportunities for healthcare systems and patient care. In this section, we explicitly discuss the broader implications of our findings for researchers, clinicians, and policymakers, aligning them with the study’s focus on the applications of ML and DL in disease diagnosis and prediction.

• Advancing healthcare practices: Our review underscores the transformative potential of ML and DL in enhancing diagnostic precision, enabling early detection of diseases, and supporting the development of personalized treatment strategies. These innovations have the potential to greatly improve healthcare delivery and patient outcomes.

• Bridging the gap between research and clinical practice: By synthesizing state-of-the-art methodologies and their clinical outcomes, this study provides a roadmap for integrating ML and DL technologies into clinical workflows. This can help bridge the gap between research innovations and real-world healthcare applications.

• Addressing challenges for future research: We pinpoint critical challenges, including data quality, model interpretability, and seamless integration into clinical workflows, that need to be resolved to fully harness the advantages of AI-driven healthcare solutions. These insights can inform future research initiatives aimed at overcoming these obstacles.

• Policy and ethical considerations: Our findings emphasize the necessity for strong regulatory frameworks and ethical guidelines to ensure the responsible deployment of ML and DL technologies in healthcare. This involves tackling concerns related to data privacy, algorithmic bias, and obtaining patient consent.

• Empowering clinicians and researchers: By offering a thorough review of ML and DL applications across 16 diseases, this study acts as a valuable resource for clinicians and researchers aiming to utilize AI technologies in their work. It also underscores the significance of interdisciplinary collaboration among computer scientists, clinicians, and healthcare providers.

Limitations of the study

Although this review provides a comprehensive analysis of the applications of ML and DL in disease diagnosis and prediction, it is not without limitations. These limitations highlight areas for improvement and future research:

• Scope of diseases: Although this review covers 16 diverse diseases, it does not encompass all medical conditions where ML and DL have been applied. As a result, the findings may not be fully generalizable to diseases outside the scope of this study.

• Temporal constraints: The review focuses on studies published between 2015 and 2024 to capture recent advancements. However, the rapid pace of innovation in AI means that some cutting-edge developments may not have been included, potentially limiting the timeliness of the review.

• Data heterogeneity: The studies reviewed utilized datasets with varying quality, size, and annotation standards. This heterogeneity can affect the comparability of results and the generalizability of the findings across different healthcare settings.

• Model interpretability: Although DL models have demonstrated remarkable accuracy in disease diagnosis and prediction, their “black-box” nature remains a significant limitation. The lack of interpretability can hinder their adoption in clinical settings, where transparency and explainability are critical.

• Clinical integration challenges: Although this review highlights the potential of ML and DL in healthcare, it does not extensively address the practical challenges of integrating these technologies into clinical workflows. Issues such as regulatory hurdles, cost, resistance to change, and the need for interdisciplinary collaboration require further exploration.

• Bias in literature selection: Despite efforts to include a wide range of studies, there may be inherent biases in the selection of literature, such as the preference for high-impact journals or studies with positive results. This could affect the representativeness of the findings.

• Focus on technical aspects: The review primarily focuses on the technical and methodological aspects of ML and DL applications, with limited discussion on ethical, legal, and social implications (ELSI). These considerations are crucial for the responsible deployment of AI in healthcare.

By recognizing these limitations, we strive to present a balanced view of the current capabilities of ML and DL in disease diagnosis and prediction. Tackling these limitations in future research will be crucial for progressing the field and unlocking the full potential of AI-driven healthcare solutions.

Challenges and open issues

Navigating the integration of ML and DL in healthcare comes with a myriad of challenges and open issues that require careful consideration. Although these technologies hold immense promise for transforming the industry, concerns such as data quality and quantity, interpretability and trust in models, generalization and bias, regulatory and legal hurdles, computational resource requirements, integration with clinical workflows, and ethical considerations loom large. Tackling these challenges is essential to harness the full potential of AI in healthcare, ensuring that the benefits of these innovations are realized while upholding patient privacy, trust, fairness, and ethical standards. Addressing these complexities will be pivotal in shaping the future of healthcare delivery, ultimately leading to more efficient, accurate, and patient-centric care practices. The existing key challenges that pose significant complexities and obstacles to the application of ML and Dl in disease diagnosis and prediction can be summarized as follows (Fig. 8):

• Data accessibility

Fig. 8

Challenges in disease diagnosis and prediction

In the realm of disease diagnosis and prediction, the scarcity of data poses a significant challenge, particularly pronounced in the context of rare diseases with limited patient populations. It becomes arduous to develop accurate diagnostic and predictive models in the absence of sufficient data.

• Data quality

Even when the data are available, its quality is not always assured. Factors like inaccurate or incomplete records, or poorly structured data, can compromise data quality. Unreliable data may lead to erroneous diagnoses and forecasts.

• Data bias

Another concern is the presence of bias in the data. Data bias occurs when the dataset used to train a diagnostic or predictive model does not accurately represent the population for which the model is intended. Data bias can result in models that are unfair or inaccurate.

• Model complexity

Predictive and diagnostic models can be highly intricate, rendering them more challenging to comprehend, interpret, and apply in clinical settings.

• Ethical considerations

Addressing ethical concerns related to patient privacy, consent, algorithmic bias, and transparency is essential to maintain trust and ensure the responsible use of AI in healthcare.

• Cost

The development and implementation of diagnostic and predictive models can incur significant costs, potentially hindering their widespread adoption, especially in resource-constrained settings.

• Integration with clinical workflow

Seamlessly integrating machine learning solutions into the existing clinical workflows and ensuring adoption by healthcare professionals remain key challenges to realizing the full potential of these technologies in healthcare settings.

Despite these challenges, ML and DL hold immense potential for improving disease prediction and diagnosis. They can sift through large volumes of patient data to uncover patterns that may escape human clinicians’ notice. This can aid in identifying individuals at risk of specific diseases and lead to more precise and timely diagnoses. Future research in disease diagnosis and prediction using ML and DL will be focused on overcoming current challenges and limitations. This may involve the advancement of new DL capable of handling diverse data modalities like medical images and electronic health records. Additionally, there will be a focus on enhancing explainable ML and DL that offer insights into the model’s decision-making process.

Further exploration in the application of ML and DL to disease diagnosis and prediction is warranted. Promising avenues include the utilization of transfer learning to leverage pretrained models for improving performance on smaller datasets and the integration of deep learning with technologies like blockchain and the Internet of Things (IoT) to fortify healthcare systems’ robustness and security. The existing potential of using ML and Dl in disease diagnosis and prediction can be summarized as follows (Fig. 9):

• Early detection

Fig. 9

Potentials in disease diagnosis and prediction

ML and DL algorithms can analyze large volumes of diverse patient data, including medical images, genetic information, and clinical records, to identify subtle patterns and biomarkers indicative of early-stage disease. This early detection capability holds promise for proactive intervention and improved patient outcomes.

• Personalized medicine

By leveraging patient-specific data, ML and DL models can contribute to tailoring treatment strategies and clinical decision making based on the individual characteristics, genetic profiles, and disease risks, thus supporting the shift towards personalized medicine.

• Enhanced diagnostic accuracy

The use of ML algorithms in medical imaging analysis, such as in radiology and pathology, has shown potential in improving diagnostic accuracy and reducing misinterpretation, aiding healthcare providers in making more informed clinical decisions.

• Prognostic insights

ML and DL algorithms can integrate and analyze diverse data sources to predict disease progression, treatment responses, and patient prognosis, providing valuable insights for optimizing care pathways and resource allocation.

• Patient risk stratification

ML models can assist in stratifying patients based on their risk of developing specific diseases or experiencing adverse events, allowing for more targeted interventions and preventive measures.

• Public health surveillance

ML-based disease prediction models have the potential to bolster public health efforts by forecasting disease outbreaks, identifying at-risk populations, and optimizing resource allocation for healthcare services.

• Continuous learning and adaptation

ML and DL models have the potential to continuously learn and adapt based on the incoming data and evolving patient profiles, presenting opportunities for dynamic and responsive healthcare decision-support systems.

Conclusion and future works

The past decade has witnessed a transformative shift in disease prediction and diagnosis, driven by rapid advancements in ML and DL techniques. This review, encompassing studies from 2015 to 2024 across sixteen diverse diseases, highlights the immense potential of ML and DL in revolutionizing healthcare. These technologies have demonstrated remarkable success in identifying complex patterns in medical data, enabling accurate disease prediction, early detection, and personalized treatment strategies. By facilitating timely interventions and optimizing resource allocation, ML and DL have significantly enhanced clinical decision making and patient outcomes. Although this study focuses on the key diseases, future research could expand to other areas, such as additional cancer types, nephrology diseases, and surgical applications.

Despite these advancements, several challenges must be addressed to fully realize the potential of ML and DL in healthcare. Key considerations for future research include interdisciplinary collaboration to bridge the gap between data scientists and healthcare practitioners, improving data quality and standardization to build robust predictive frameworks, and developing transparent and interpretable models to foster trust and acceptance. In addition, ethical and regulatory frameworks must be established to ensure patient privacy, equity, and transparency. Rigorous real-world validation of predictive models is essential to assess their clinical effectiveness and generalizability, while educational initiatives are needed to equip healthcare professionals with the skills to effectively integrate these technologies into practice.

In conclusion, the integration of ML and DL into healthcare holds immense promise for improving disease diagnosis and prediction. As we navigate this evolving landscape, sustained attention to ethical, practical, and translational dimensions will be critical to ensure the successful adoption of these technologies. By addressing these challenges, we can pave the way for personalized, evidence-based healthcare delivery that benefits patients and clinicians alike, ultimately transforming the future of medicine.