Optimizing genetic prediction: Define-by-run DL approach in DNA sequencing

1 Introduction

As personalized medicine expands, genetic testing has been progressively recognized as a cornerstone of contemporary healthcare practices [1]. Among these, DNA sequencing, a process determining the precise order of nucleotides within a DNA molecule, is particularly interesting [2]. This technology allows for an in-depth understanding of genetic makeup and the roles specific genes or genetic variants may play in health and disease. It presents an unrivaled opportunity to evaluate an individual's risk of developing genetic disorders [3]. While the individuals concerned may not manifest symptoms, they may have inherited a genetic predisposition towards certain conditions, making DNA sequencing especially critical for this demographic [4]. Traditional diagnostic methodologies often demonstrate limitations, particularly their scope to deliver consistent accuracy and predictive capability [5]. Traditional diagnostic methods encompass a range of medical testing procedures used to identify diseases and conditions. These methods might include physical examinations, imaging tests like X-rays or MRIs, laboratory tests such as blood and urine tests, and biopsies. While these methods are fundamental and effective for diagnosing a wide range of diseases, they may have limitations when predicting genetic disorders or understanding an individual's genetic predisposition to certain conditions. Unlike DNA sequencing, traditional diagnostic methods may not provide a deep insight into the genetic makeup of individuals and might lack the predictive power to ascertain the risks of genetic disorders at an early or pre-symptomatic stage. In this light, the emerging field of personalized medicine, fueled by advancements in genetic testing and deep learning (DL), aims to transcend these limitations by offering a more precise, proactive, and personalized approach to disease diagnosis, prediction, and management. Especially in the emerging field of personalized medicine, genetics has become an integral component of modern healthcare practices [6].

Genetic testing is at the core of these advancements, a technique that has changed our ability to predict, identify, and treat a vast array of diseases [7]. The development of DNA sequencing, which determines the exact sequence of nucleotides in a DNA molecule, has been of immense assistance to genetic testing [8]. Using this technology, medical practitioners may know an individual's genetic makeup and forecast their likelihood of developing certain genetic diseases [9]. Despite the significance of these breakthroughs, conventional diagnostic procedures struggle to consistently provide high levels of accuracy and predictive power, signaling the need for a new technique [10].

DL, a sophisticated branch of machine learning (ML) [11,12], is rapidly gaining traction across various scientific domains, including genomics and personalized medicine [13]. The ability of DL algorithms to harness vast, diverse datasets enables them to uncover complex patterns and correlations that often remain undetected by conventional statistical methodologies [14,15]. Numerous genomic researchers have effectively used DL, identifying unique genetic variations linked to many disorders [16]. The typical training of DL models includes specifying preset hyperparameters, which may limit the models' adaptability to data. Table 1 summarizes the most recent published research on DNA sequences for diagnosis using ML techniques with their limitations.

Recent works in the domain of genetic testing and DNA sequence analysis have showcased a variety of computational and ML methodologies. For instance, a study by Ali et al. [17] proposed a computational method for accurately identifying DNA-binding proteins using a combination of split amino acid composition and position specific scoring matrix for feature extraction. Although effective, this model can become computationally intensive and slow for large-scale applications due to its use of several feature extraction and selection methods. On the other hand, research by Alakus and Baykara [18] employed a DL model, specifically a Bi-directional Long Short-Term Memory (BiLSTM) model, for DNA sequence classification. However, solely relying on the BiLSTM model and raw DNA sequences without various feature extraction operations might limit the performance.

Furthermore, a novel DL model named LegNet was introduced by Penzar et al. [19], showcasing its prowess in predicting promoter expression from DNA sequences. Despite its effectiveness, the training of this model is computationally intensive, requiring specific hardware and software configurations. In a different stride, the paper by Zhang et al. [20] unveiled FunGeneTyper, a DL framework aimed at protein-coding gene function prediction, yet lacked a detailed discussion on computational efficiency. Finally, research by Gunasekaran et al. [21] explored the use of various DL models like convolutional neural network (CNN), CNN-LSTM, and CNN-BiLSTM for DNA sequence classification. However, the manual tuning of hyperparameters in these models posed significant limitations.

As highlighted by these studies, the challenges posed by manual tuning of hyperparameters underscore the necessity for more efficient, automated methods in hyperparameter optimization, leading to the exploration of automated methods like grid search, random search, and Bayesian optimization to surmount the limitations of manual tuning, providing a segue into the define-by-run DL methodology employed in this research.

It can be concluded that the previous works are based on manual hyperparameter tuning.

Manual tuning of hyperparameters in DL models can have some following limitations:

1. Manual tuning involves a considerable amount of time and effort. Each set of hyperparameters must be separately tested, with the process repeated until satisfactory results are obtained [22].

2. Manual tuning requires a deep understanding of the model and how each hyperparameter affects it, which can be hard for people who do not have much experience in the field [23].

3. No guarantees of the optimal solution; even with much time and effort, discovering the optimum set of hyperparameters is not a certainty. When the search field is wide, it is simple to overlook the ideal answer [24].

4. Manual tuning is not scalable for models with a large number of hyperparameters. As hyperparameters rise, the tuning procedure becomes exponentially more difficult [25].

5. Manual tuning can induce bias, as the tuner's prior experiences or presumptions can impact decisions [26].

6. When manually tuning hyperparameters, there is a potential of overfitting the model to the training data, which might result in poor generalization performance on unseen data [27].

Automated methods like grid search, random search, and Bayesian optimization aim to overcome these limitations by systematically exploring the hyperparameter space and finding the optimal set in a more efficient and less biased manner [28].

A more adaptable method, defined-by-run DL, has been created [29]. The models based on define-by-run are characterized by their ability to dynamically and iteratively optimize model hyperparameters, which gives them a significant edge over traditional models in terms of performance and adaptability [25]. This approach allows the model to tune its hyperparameters repeatedly and dynamically based on the data, offering an advantage over conventional models. These hyperparameters include the number of layers, the number of neurons in each layer, the learning rate, and the batch size, among others. This adaptability enables the model to learn from and adapt to the data more effectively, hence enhancing the model's performance and precision.

This research employs a define-by-run DL model reinforced by applying the Tree-structured Parzen Estimator (TPE) algorithm to refine the application of DNA sequencing in disease diagnosis and prediction [30]. The TPE algorithm addresses hyperparameter optimization by building a probabilistic model of the objective function's conditional distributions, which enables more efficient hyperparameter space searches. This strategy has significantly enhanced the optimization procedure and overall performance of DL models. The model's distinguishing feature is its capacity for iterative optimization of critical hyperparameters throughout the learning process, enabling it to adjust dynamically to the data, thereby ensuring superior accuracy [31]. By leveraging a diverse dataset comprising DNA sequences from control and patient cohorts, the study aims to ascertain the efficacy of this innovative approach in enhancing the precision of disease prediction and diagnosis [32]. This research strives to contribute to the ongoing discourse on the transformative potential of DL in genetic disorder diagnosis and management [33].

The pivotal dataset encompasses DNA sequences from two disparate cohorts: individuals previously diagnosed with breast cancer and a control group comprising healthy subjects. The goal is to scrutinize the genetic distinctions, potentially revealing genetic markers or mutations correlative or causative of breast cancer. This DL model is architected to adapt during the learning voyage dynamically, iteratively optimizing critical hyperparameters to enhance its diagnostic acumen. The significance of this endeavor is profound, as early detection of breast cancer is paramount in elevating the success rate of therapeutic interventions and ameliorating patient prognoses. By leveraging DNA sequencing to unveil the genetic underpinnings of breast cancer, this investigative effort aims not only to augment diagnostic precision but also to furnish invaluable foresight, enabling timely medical and lifestyle interventions. The predictive prowess of this model, as demonstrated by its remarkable performance metrics, heralds a substantial stride towards the early discovery of breast cancer, thus potentially contributing to the broader objectives of personalized medicine and genetic counseling in the oncology domain.

The main contribution of this study can be summarized as follows:

1. This is the first define-by-run DL model created to categorize DNA sequences to the best of our knowledge.

2. Applying the define-by-run method in genomics provides a valuable case study for using this approach in other fields. It contributes to the broader field of ML by demonstrating the effectiveness of dynamic, data-driven model optimization.

3. The proposed define-by-run method's ability to dynamically adjust to data makes it a scalable solution for DNA sequence classification. As genomic datasets grow in size and complexity, this method's scalability becomes increasingly important.

This work is organized as follows: Section 2 introduces the dataset and the premise underlying the proposed approaches. Section 3 describes the results and discussion of the experiments. The conclusion of the study is given in Section 4.

2 Materials and methods

Algorithm 1 outlines a DL method for DNA sequence classification. It begins with the loading and preprocessing of DNA sequences into numerical vectors. A DL model is then defined and trained using a TPE for hyperparameter optimization. The motivation behind employing a “define-by-run” DL approach emanates from its inherent capability to dynamically and iteratively optimize critical model hyperparameters during the learning process. This dynamic adaptation is envisioned to enhance the model's accuracy and predictive power, crucial for the precise diagnosis and prognosis of breast cancer through DNA sequencing.

Furthermore, unlike conventional DL models, which necessitate preset hyperparameters, the “define-by-run” DL model allows for a more flexible, data-driven optimization of hyperparameters such as layer count, neuron count per layer, learning rate, and batch size. This adaptability is anticipated to enable the model to learn from and adapt to the data more effectively, amplifying its performance and precision in identifying genetic markers associated with breast cancer. Finally, in the context of DNA sequencing, the “define-by-run” DL model, reinforced by the TPE algorithm, aims to refine the application of DNA sequencing in disease diagnosis and prediction. This approach potentially transcends the limitations of manual hyperparameter tuning, which is often computationally intensive and prone to biases, thus offering a more efficient and robust method for analyzing complex genomic data pertinent to breast cancer.

Various metrics evaluate the model's performance on a test set to evaluate the proposed method. This process results in a model differentiating between control and patient DNA sequences. Figure 1 depicts the workflow associated with the proposed method, while Algorithm 1 explains DNA Sequence Classification with a define-by-run technique.

Figure 1

The workflow of the proposed method.

Algorithm 1: DNA sequence classification with define-by-run

2.1 Data acquisition

This research’s initial phase involved gathering a comprehensive dataset of DNA sequences from both control and breast cancer patient groups, including individuals diagnosed with or having a familial history of genetic disorders and those without such a history. The dataset, sourced from reputable genomics databases, ensured the sequences' validity and the associated metadata's consistency. Additionally, DNA/genomic sequences were collected from “The National Centre for Biotechnology Information” (https://www.ncbi.nlm.nih.gov), a public database of nucleotide sequences. The DNA sequence data were stored in FASTA files and ranged from 8 to 37,970 nucleoids. In terms of diversity, the dataset showcases a wide range of nucleoid lengths, which indicates the varied genetic information contained within the dataset. This diversity is crucial as it likely provides a rich source of genomic data for the DL model to learn from, thereby enhancing the model’s capacity to accurately classify DNA sequences and identify genetic markers associated with breast cancer.

The distribution of each class and the number of samples are shown in Figure 2. Sample DNA sequences from the collection are included in Table 1, along with class labels.

Figure 2

Class distribution. 0 refers to the Control class and 1 to the patient class.

Ethical and privacy concerns are paramount in harnessing DNA sequencing data for disease prediction. These encompass informed consent, data confidentiality, and the potential for genetic discrimination. This study addressed these considerations by adhering to stringent data privacy protocols, anonymizing data to protect individual identities, and ensuring the ethical handling and secure storage of sensitive genomic information. Such measures uphold the ethical integrity of the study while safeguarding the privacy and rights of the individuals whose data were utilized for advancing the understanding and prediction of genetic diseases.

Using a diverse dataset, comprising DNA sequences from breast cancer patients and healthy control groups, is critical for evaluating the model’s performance. Such diversity ensures the model's robustness and generalizability across various genetic backgrounds, aiding in accurately identifying genetic markers associated with breast cancer. It enhances the model's diagnostic accuracy by differentiating cancerous and non-cancerous genetic patterns better. Including a control group provides a baseline for validation and benchmarking, representing real-world population diversity, which is essential for inclusive and unbiased findings. Through a comparative analysis, the model can develop nuanced predictive models, potentially leading to new insights regarding breast cancer's genetic basis, thereby improving the overall effectiveness of the model in real-world clinical settings.

3 Results

This experiment established an initial learning rate of 0.001 for all weights. If validation data loss does not improve after one epoch, the learning rate is decreased by a factor of 0.5 using a lower learning rate with one patient. If, after two epochs, the validation loss of data has not improved, the early halting approach with patient two is utilized to reduce overfitting. The tests were conducted with a 1.7 GHz Intel Core i7 processor, 16 GB of RAM, Windows 11 Professional 64-bit, and a 4 GB NVIDIA GeForce graphics card. Python, the TensorFlow library, and the Optuna module, which generates the optimization algorithm, are used to develop the models.

The TPE algorithm optimizes the hyperparameters of the CNN model using the define-by-run method. The hyperparameters that allowed the model to achieve the best performance were estimated based on the optimization results. The optimal hyperparameters for each dataset are listed in Table 2, along with the search space for hyperparameters. After randomly initializing the hyperparameters, the TPE iterated 50 times to determine the model's recall while evaluating the hyperparameters. Consequently, the optimal CNN model hyperparameters can be determined when the classifier has the highest recall. The recall iteration convergence curve for each dataset utilized in this experiment is depicted in Figure 3.

Figure 3

The recall iteration convergence plot.

The recall iteration convergence plot provides a visual representation of how the recall score of our DL model evolved during the optimization process. The x-axis represents the number of iterations, and the y-axis represents the recall score. Each point on the plot corresponds to the recall score of the model at a particular iteration. At the beginning of the optimization process, we observe that the recall score varies significantly, which is expected as the TPE algorithm explores different sets of hyperparameters to understand the hyperparameter space better. As the iterations progress, we notice that the recall score starts to converge, indicating that the TPE algorithm is beginning to exploit the regions of the hyperparameter space that it has found to yield high recall scores. The fluctuations in the recall score become less pronounced, and the algorithm starts to fine-tune the hyperparameters within these promising regions. By the end of the optimization process, the recall score has converged to a relatively stable value suggesting that the TPE algorithm has successfully identified a set of hyperparameters that yields a high recall score. The final model, trained with these optimized hyperparameters, is expected to perform well on unseen data, particularly in its ability to correctly identify positive cases, as indicated by the high recall score.

Figure 4 shows the importance of each hyperparameter. It is concluded that the most important hyperparameter is the kernel size, followed by the number of convolutional layers

Figure 4

Hyperparameters’ importance concluded by define-by-run and TPE algorithm.

Table 3 illustrates the performance metrics of the define-by-run technique's optimal model and the number of training epochs and training duration per epoch (based on test data).

4 Discussion

Table 4 demonstrates that the accuracy of 0.871 for classifying DNA sequences into two categories – patient and control – is quite high, especially considering the intricacy of genomic data. This suggests that the model is reliable in determining whether a DNA molecule belongs to a patient (someone with a disease) or a control (someone without the disease) approximately 87.1% of the time. The performance measurements indicated by the define-by-run method for the optimal model sequence support this conclusion.

The precision of the model was determined to be 0.872, indicating that of all sequences of the model predicted as patients, 87.2% were indeed from patients. This high precision emphasizes the model’s ability to accurately identify patient sequences and minimize FPs, which is critical in avoiding unnecessary alarm or treatment.

The recall, or the model's sensitivity, was computed as 0.871, demonstrating that the model successfully detected 87.1% of all the actual patient sequences in the dataset. Such sensitivity is pivotal in medical diagnostics, where overlooking a patient's condition can lead to delayed or absent treatment, potentially resulting in severe consequences.

The F1-score, calculated as the harmonic mean of precision and recall, was determined to be 0.872. This score measures the model's balance between precision (minimizing FPs) and recall (minimizing FNs) and is particularly important in contexts where both these aspects are equally critical. An F1-score of 0.872 confirms that the model achieves a good equilibrium between identifying as many patients as possible and maintaining a low rate of false alarms.

An AUC of 0.95 indicates an excellent model. It means there is a 95% chance that the model will be able to distinguish between positive and negative class instances. This high AUC score suggests that the model has a high separability measure and can make highly accurate predictions.

The confusion matrix in Figure 5 offered another perspective to evaluate the model. It showed that most predictions were on the diagonal line, corresponding to correct classifications. This model generated fewer off-diagonal elements, which correspond to misclassifications, compared to previous models, validating the model's robustness.

Figure 5

The confusion matrix for the best-obtained model on test data.

The use of a define-by-run DL technique in combination with the TPE algorithm for hyperparameter optimization is responsible for the significantly high performance of this model. The model dynamically adapted to the training data, optimizing the hyperparameters during the learning process and improving performance. These promising results highlight the potential of using advanced DL techniques in genetic disorder prediction and diagnosis.

Based on the findings of this investigation, the role of DL and AI in genetic disorder diagnosis and management is poised for significant expansion. The define-by-run DL model's demonstrated ability to effectively classify DNA sequences lays a solid foundation for more sophisticated, real-time genetic analysis and personalized medicine. As the technology matures and integrates with clinical workflows, it will likely facilitate quicker, more accurate diagnoses and tailor treatments to individuals’ genetic profiles, potentially transforming the prognosis for many genetic disorders. Additionally, as data volumes grow and algorithms improve, we might evolve towards more holistic, AI-driven approaches that can analyze a wide spectrum of genomic, clinical, and environmental data to provide a more nuanced understanding of genetic disorders and their management. This progression would enhance personalized medicine and contribute to the broader scientific understanding of genetic diseases, potentially uncovering novel therapeutic targets and treatment strategies.

5 Conclusion

Integrating DL and genetic sequencing promises to transform the future of disease diagnosis and prediction. Our study successfully employed a define-by-run DL model, enhanced by TPE hyperparameter optimization, to analyze DNA sequences for disease prediction. The resulting model surpassed previous approaches in accuracy, precision, recall, and F1-score, suggesting that our method can significantly improve the ability to predict genetic disorders.

Notably, the dynamic and iterative nature of the define-by-run approach, coupled with the robustness of TPE for hyperparameter optimization, demonstrates a considerable potential for enhancing DL model performance in genomics and beyond. The remarkable performance of the DL model underscores the potential of leveraging deep learning for precise genetic disease diagnosis, aiding in personalized medicine through early detection and tailored treatment plans, thereby contributing to improved patient outcomes.

As we advance into the era of personalized medicine, these findings underline the promise of DL in harnessing the wealth of information embedded within our DNA. One of the primary challenges in genomic analysis is the quality and availability of genomic data. Ensuring that the data are well-curated, labeled accurately, and sufficiently large to train robust models are essential. This challenge could be addressed by sourcing data from reputable genomic databases or consortia and applying rigorous data preprocessing and cleaning techniques to ensure data quality. Furthermore, the computational demands of DL models, especially when applied to large genomic datasets, can be significant, which was mitigated by employing efficient hardware resources. Further research will be beneficial in exploring more comprehensive datasets, integrating diverse types of genetic and non-genetic data, and adapting our approach to other domains of predictive analytics. Our study contributes significantly to this exciting journey, bringing us closer to unlocking our genetic code’s full potential.

Future studies could explore integrating additional genetic and non-genetic data types, further refining the model’s predictive capability. Also, given the dynamic nature of both the define-by-run approach and TPE optimization, the research could investigate the potential benefits of these methods in other fields of predictive analytics. On the other hand, the research needs to address some limitations, such as incorporating model interpretability techniques, using larger and more diverse datasets, and exploring other methods for hyperparameter optimization.