[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content
Download PDF

A novel prediction model for the prognosis of non-small cell lung cancer with clinical routine laboratory indicators: a machine learning approach

BMC Medical Informatics and Decision Making volume 24, Article number: 344 (2024) Cite this article

Abstract

Background

Lung cancer is characterized by high morbidity and mortality due to the lack of practical early diagnostic and prognostic tools. The present study uses machine learning algorithms to construct a clinical predictive model for non-small cell lung cancer (NSCLC) patients.

Methods

Laboratory indices of the NSCLC patients at their initial visit were collected for quality control and exploratory analysis. By comparing the levels of the above indices between the survival and death groups, the statistically significant indices were selected for subsequent machine learning modeling. Ten machine learning algorithms were then employed to develop the predictive models with survival and recurrence as outcomes, respectively. Moreover, regression models were constructed using the random survival forest algorithm by incorporating the survival time dimension. Finally, critical variables in the optimal model were screened based on the interpretable algorithms to build a decision tree to facilitate clinical application.

Results

682 patients were enrolled according to the inclusion and exclusion criteria. The preliminary comparison results revealed that except for fast blood glucose, CD3+T cell proportion, NK cell proportion, and CA72-4, there were significant statistical differences in other tumor markers, inflammation, metabolism, and immune-related indices between the survival and death groups (p < 0.01). Subsequently, indices with statistical differences were incorporated into machine learning modeling and evaluation. The results showed that among the ten prognostic models constructed using survival status as the outcome, the neural network model obtained the best predictive performance, with accuracy, sensitivity, specificity, AUC, and precision values of 0.993, 0.987, 1.000, 0.994, and 1.000, respectively. The corresponding SHAP16 algorithm revealed that the top five variables in terms of importance were interleukin6 (IL-6), soluble interleukin2 receptor (sIL-2R), cholesterol, CEA, and Cy211, respectively. The random survival forest model also confirmed the critical role of CEA, sIL-2R, and IL-6 in predicting the prognosis of NSCLC patients. A decision tree model with seven cut-off points based on the above three indices was eventually built for clinical application.

Conclusion

The neural network model exhibited ideal predictive performance in the survival status of NSCLC patients, and the decision tree model constructed based on selected important variables was conducive to rapid bedside prognosis assessment and decision-making.

Highlights

There is a lack of highly sensitive, specific, and organ-specific biomarkers to predict the prognosis of lung cancer patients.

Compared with traditional predictive models, the models constructed by machine learning methods have incredibly high predictive accuracy, sensitivity, and specificity.

Both classification and regression algorithms confirmed the significant predictive value of IL-6, sIL-2R, and CEA on the prognosis of lung cancer patients.

A decision tree prognostic model including IL-6, sIL-2R, and CEA with explicit cutoff values was further provided for rapid prognostic assessment and clinical decision-making.

Peer Review reports

Introduction

In the past 20 years, the global mortality rate of lung cancer patients has remained the highest among all malignant tumors [1]. In China, relevant predictions indicated that the mortality rate of lung cancer would continue to rise in the next ten years [2], and the situation of prevention and control is still grim. For clinicians, it is crucial to use specific biomarkers to accurately assess lung cancer patients’ metastasis or prognosis in clinical practice and to give appropriate treatment. Unfortunately, to date, there has been a lack of sensitive and specific biomarkers applied in the clinical prognostic assessment of lung cancer. Traditional tumor markers such as carcinoembryonic antigen (CEA), carbohydrate antigen72-4 (CA72-4), and Cy211 may rise in some patients with recurrent metastasis. However, clinical practice still has uncertainties, with low sensitivity and poor organ specificity [3]. Hence, searching for novel prognostic biomarkers with high sensitivity and specificity is urgent.

In recent years, some new biomarkers such as microRNA(miRNA) [4], exosomal miRNAs [5], DNA methylation [6], and circulating tumor cells (CTCs) [7] have been gradually applied to the monitoring of lung cancer, with improved sensitivity and specificity compared to previous traditional biomarkers. Nevertheless, technical limitations, expensive economic costs, and lack of large-scale clinical validation are potential factors that hinder the clinical application. In addition, several studies have been carried out to explore potential prognostic-related biomarkers of lung cancer based on metabolomics, proteomics, transcriptomics, or multi-omics approaches. Some of these studies have obtained biomarkers with satisfactory specificity and sensitivity, providing ideas for the research on the mechanism of lung cancer development. Unfortunately, the poor accessibility of omics data in actual clinical practice compromised rapid bedside assessment and decision-making.

General blood tests, including blood routine, liver and kidney function, blood glucose and blood lipids, tumor markers, and immune indicators, have been widely carried out in clinical practice as efficacy monitoring and safety evaluation for tumor patients. Among them, there are several markers related to the prognosis of lung cancer patients. For instance, platelet counts [8], lymphocyte/monocyte ratios [9], and platelet/lymphocyte ratios [10] in routine blood tests have been revealed to be associated with the prognosis of lung cancer patients. Some metabolic and immune-related indicators, such as low-density lipoprotein cholesterol [11, 12], interleukin family members [13,14,15], and T-cell subsets [16, 17], have also been suggested to guide the prognostic evaluation of lung cancer. However, as is well known, tumors are highly heterogeneous diseases, and it is difficult for a single-dimensional indicator to reflect the prognostic characteristics fully and objectively. Studies have shown that prognostic models incorporating multiple features may exhibit higher predictive performance [18], especially for external data sets with higher generalization ability. The machine learning algorithm has recently been recognized as an emerging technique in mining the implied knowledge and rules behind high-dimensional data. Therefore, in the present study, we included a total of 25 indices in the four dimensions of blood routine, inflammation, metabolism, and immunity, mined the factors closely related to the recurrence and death of lung cancer patients through multiple machine learning algorithms, and constructed a clinical prognosis model, to assist clinical decision-making.

Materials and methods

Patients

In the present study, we retrospectively included non-small cell lung cancer (NSCLC) patients hospitalized at the Oncology Clinical Medical Center of Shanghai Municipal Hospital of Traditional Chinese Medicine from 2012 to 2020 as subjects. Inclusion criteria for this study were: (1) The diagnosis of lung adenocarcinoma or squamous cell carcinoma was confirmed by definitive histopathological examination; (2) Patients with complete clinical information and available cut-off points for recurrence and mortality; (3) Patients without severe infection, autoimmune diseases, and other comorbidities; (4) Patients were free from drugs known to influence glucose and lipid metabolism during the three months before sampling. In contrast, patients with severe cardiac, hepatic, and renal comorbidities or incomplete medical data (missing data more than 20%) or with multiple primary cancers will be excluded from the study. The present study has been reviewed and approved by the Ethics Committee of Shanghai Municipal Hospital of Traditional Chinese Medicine (approval number: 2023SHL-KY-20-01). The detailed screening process is displayed in Fig. 1.

Data collection

The general information about the patients, such as gender, age, pathological types, clinical stage, and several histories of surgery, radiotherapy, chemotherapy, targeted therapy, and smoking, was retrieved and recorded from the Hospital Information System (HIS). The clinical stage was determined based on the current UICC/AJCC eighth edition TNM staging system. The blood test results of each patient’s initial visit including inflammation-related indices (routine blood test, interleukin, and tumor necrosis factor α), metabolism-related indices (fast blood glucose, triglycerides, and total cholesterol), immune-related indices (T-cell subset proportions, NK cell proportion, and B cell proportion), and lung cancer-related tumor markers (CEA, SCC, CA-125, CA72-4, and Cy211) were accurately recorded through double entered and checked. The primary outcome parameters in the present study included overall survival (OS) and progression-free survival (PFS). We defined OS as the duration from the first visit to either the day of death or the last follow-up day; PFS as the interval between the first visit day to disease progression or death from any cause. The patient’s condition has continued to be followed up utilizing clinical visits or telephone communication, with imaging being performed every six months, and the last follow-up date was December 31, 2021.

Exploratory data analysis (EDA)

The exploratory data analysis approach often serves as the cornerstone of machine learning. Firstly, for missing data in continuous variables, the Multi-variate Imputation by Chained Equations (MICE) R package was utilized to perform the multiple imputation. Considering that some of the laboratory values were characterized by extreme left skewness or contained extreme values, we performed a log transformation of the numerical variables to reduce the impact of outliers on the data distribution. Since several machine learning algorithms assume the normality of features, a Box-Cox transformation approach via the MASS package in R was employed to improve normality. Moreover, to eliminate the potential problem of class imbalance, a hybrid technique of over-sampling and under-sampling via the SMOTE algorithm was adopted to overcome the issue. For continuous variables, results were expressed as mean ± standard deviation. The student t-test or Wilcoxon rank sum test was used to compare the two groups’ means. For categorical variables, results were presented as numbers and percentages.

Machine learning

After data imputation and preprocessing, the data were randomly split into training, validation, and test datasets at 70%, 15%, and 15% to train different machine learning algorithms. Among them, the training dataset was utilized to fit the models, the validation dataset was used for hyperparameter tuning, and the test dataset was used to assess the model performance.

Ten machine learning algorithms, including random forest (RF), K-nearest neighbors (KNN), Naïve Bayes (NB), AdaBoost, XGBoost, support vector machines (SVM), Gradient Boosting machine (GBM), generalized linear model (GLM), multivariate adaptive regression splines (MARS), and neural network (NNET) were applied to implement the classification. Each algorithm has different performance in different applicable scenarios. For instance, KNN is a basic nonparametric classification method that classifies input samples by searching their nearest neighbors in feature space. RF is an integrated learning method that performs classification or regression by creating multiple decision trees and combining their prediction results. Similarly, the core idea of GBM, AdaBoost, and XGBoost is to integrate the training results of several weak classifiers via an iterative algorithm to generate stronger classifiers. Compared to AdaBoost, the regularization and gradient Boosting techniques used by XGBoost lead to higher accuracy and robustness of the prediction results. The SVM algorithm seeks optimal boundaries among different classifications and predicts the classification labels based on the feature vectors. As an extension of a generalized linear model, the GLM does not strictly restrict the assumption of linearity between the predictor and response variable, which makes it possible to reflect the relationship more flexibly and accurately. The MARS can be regarded as a generalization of stepwise linear regression by constructing a weighted sum of hinge functions to form the final prediction model, which is especially suitable for the feature prediction of high-dimensional data.

In addition, we also included overall and progression-free survival information in the study to explore the potential association between blood indices and survival prognosis based on the random survival forest algorithm.

Hyperparameter optimization

Each machine learning algorithm involves various unique hyperparameters that influence the evaluation of the model’s performance. In the present study, hyperparameter tuning was carried out to optimize the hyperparameters and improve the performance of the machine learning models. First, the hyperparameter search space was created, the random search method was defined, and the optimal hyperparameters were identified through internal ten-fold cross-validation. The optimal hyperparameters were then set to the learner for subsequent model construction. The detailed hyperparameter settings for each specific algorithm are displayed in Table S1.

Model evaluation

15% of the total dataset was independently divided as a test set to evaluate the generalization performance of several machine learning models. In the present study, we utilized a confusion matrix, sensitivity, specificity, precision, accuracy, and area under the curve (ROC) to assess the sensitivity and specificity of the predictive models. Moreover, the clinical benefit utility of the model was revealed by decision curve analysis (DCA).

Model interpretability

It is well known that machine learning models yield higher predictive performance than traditional predictive models. However, their ‘black box’ effect feature usually makes them less interpretable. Hence, after obtaining the optimal model in the binary classification task, we calculated the corresponding Shapley value by the SHAP16 algorithm based on the local contribution of each feature. Subsequently, the blood indices with higher weights in the model were selected based on their importance in guiding clinical practice. Similarly, for the model constructed by the randomized survival forest algorithm, we scored and ranked the importance of the variables by utilizing methods of permutation variable importance (VIMP) and tree minimum depth. Moreover, considering that the high number of variables in the model does not aid clinicians’ rapid decision-making, we synthesized and selected the top three features for decision tree construction, thus further optimizing the model’s interpretability and clinical application value.

Data analysis and visualization

All data in the present study were analyzed and visualized using R software (Version 4.3.0, https://www.r-project.org/). The flow chart was realized using Microsoft PowerPoint software (Version 16.0).

Fig. 1
figure 1

Flowchart of case selection and data analysis for the present study

Full size image

Results

Preliminary analysis of patients’ general data and baseline blood indices

According to the inclusion and exclusion criteria, 682 NSCLC patients have finally enrolled in the present study, including 433 patients alive and 249 patients dead, as shown in Fig. 1. The results regarding baseline characteristics comparison indicated significant differences between the two groups in age, gender, pathological classification, clinical stage, history of previous treatment, gene mutation patterns, distant metastasis, and physical performance (p < 0.001). The age of the patients, the proportions of males, squamous carcinoma, advanced stage, and distant metastasis in the death group were higher than those of patients in the survival group. Moreover, the physical scores were worse in the dead group, consistent with clinical common sense. However, no significant difference was seen in the smoking history (p > 0.05), as presented in Table 1.

Furthermore, we selected inflammation, metabolism, immune-related blood indices, and tumor markers at the initial visit and compared them between the two groups. For inflammation-related markers, patients in the death group had significantly higher levels of leukocyte, neutrophil, neutrophil/lymphocyte ratio (NLR), interleukins, and tumor necrosis factor-alpha than patients in the survival group (p < 0.01). Metabolism-related total cholesterol and triglycerides show higher levels in the survival groups (p < 0.01). For immune-related markers, patients in the survival group exhibited higher CD4+T cell proportion and CD4/CD8 ratios and lower CD8+T cell and CD19+ B cell proportions (p < 0.01). In addition, most tumor markers, including CEA, CA-125, SCC, and Cy211, showed higher levels in the death group (p < 0.01). Nevertheless, the two groups had no significant differences regarding fast blood glucose, CD3+T cell proportion, NK cell proportion, and CA72-4 (p > 0.05), as illustrated in Fig. 2.

Table 1 Clinical characteristics of enrolled lung cancer patients (n, %)
Full size table
Fig. 2
figure 2

Comparison of routine blood indices between the survival and death groups at the initial visit of NSCLC patients

Full size image

Machine learning model construction and evaluation

With the preliminary comparison of the blood indices, the variables with statistical significance were incorporated into the machine-learning modeling. Ten machine learning models were subsequently developed, with survival status as an outcome indicator. The models were evaluated using the confusion matrix associated with performance metrics, ROC, DCA, and calibration curves. By verifying the independent test dataset, the results indicated that except for Naïve Bayes and GLM models, the other models’ accuracy, sensitivity, specificity, and area under curves (AUC) were all greater than 90%. Of these, the neural network (NNET) model exhibited one of the best predictive performances, with accuracy, sensitivity, specificity, AUC, and precision values of 0.993, 0.987, 1.000, 0.994, and 1.000, respectively (Fig. 3A; Table 2, and Fig. 4). The decision-curve analysis indicated the good clinical utility of the NNET model (Fig. 3B), while the calibration of the model also exhibited a relatively favorable performance (Fig. 3C). Although the above results confirmed the high predictive performance characteristics of the NNET model, its interpretability and subsequent clinical application are limited due to its ‘black box’ effect. Hence, we employed the SHAP16 algorithm to depict the importance of the variables. The results demonstrated that the top five variables in terms of importance in the NNET model were interleukin6, interleukin2 receptor, cholesterol, CEA, and Cy211, respectively (Fig. 3D).

Similarly, we developed and assessed the machine learning models with relapse as the outcome variable. The results revealed that the NNET model also achieved better predictive performance in the cross-validation with accuracy, sensitivity, specificity, and AUC of 1.000, respectively (Table S2, Figure S1A). Nevertheless, the calibration curve of the NNET model was relatively unsatisfactory (Figure S1B-C). The SHAP16 algorithm presented the ranking of the overall importance of the variables, with levels of CEA, interleukin2 receptor, Scc, cholesterol, and neutrophil/lymphocyte ratio in the top five (Figure S1D).

Fig. 3
figure 3

Construction and evaluation of machine learning models based on survival status as an outcome. (A) Comparison of the area under the ROC curve for each machine learning model; (B) Comparison of the DCA curve for each machine learning model; (C) Calibration curve for the neural network model; (D) SHAP16-based visualization of variable importance for the neural network model

Full size image
Table 2 Machine learning models predict overall survival status based on cancer and immune-related biomarker features
Full size table
Fig. 4
figure 4

Comparison of the accuracy and AUC values of different machine learning models in test dataset prediction using survival status as an outcome indicator

Full size image

Survival time-based random forest model construction

Since machine learning model construction is based only on the dichotomous outcome of survival and relapse, it may lose critical information on survival time and lead to bias. Therefore, we employed the random survival forest algorithm to build a predictive model to utilize the analysis’s survival status and time-to-event data. The overall model performance was assessed with the Brier score, and the methods of VIMP and tree minimum depth were employed to evaluate the importance of the variables. The results demonstrated that interluekin6, interluekin2 receptor, and CEA were the top crucial variables in terms of whether the overall survival or progression-free survival as outcome indicators (Fig. 5A-B, Figure S2), which was consistent with the modeling results according to survival status. The Kaplan-Meier survival analysis suggested that NSCLC patients with high levels of interluekin2 receptor, interluekin6, and CEA all had significantly worse overall survival than those in the low levels group (Fig. 5C). The time-dependent receiver operating characteristic (ROC) curves also indicated that all three indices had a high predictive value for overall survival, with AUC values higher than 0.7 at 1–5 years. Among them, interluekin6 displayed the best predictive property, with AUC values higher than 0.9 at 2–5 years (Fig. 5D).

Fig. 5
figure 5

Construction of random survival forest model with overall survival as an outcome indicator. (A) Evaluation of overall model performance based on Brier scores; (B) Quantification of the importance of model variables based on VIPM algorithm; (C) Comparison of the overall survival with different expressions of sIL-2R, IL-6, and CEA; (D) Time-dependent ROC curves regarding sIL-2R, IL-6, and CEA based on the overall survival

Full size image

Decision tree model construction for clinical practice

The present study has confirmed the predictive value of interluekin2 receptor, interluekin6, and CEA in NSCLC patients. However, the lack of quantitative description will significantly limit the value of their practical clinical application. Hence, the above three variables were further incorporated into the decision tree model to build a clinical decision model. The best five-level decision tree model eventually identified two cutoffs for CEA, three for interluekin6, and two for interluekin2 receptor, as shown in Fig. 6. The decision tree model provided a friendly approach for rapid bedside evaluation. For example, NSCLC patients with IL-6 ≥ 4.6 pg/mL, sIL-2R ≥ 416 U/mL, and CEA ≥ 2.8 ng/mL are predicted to have a 91% chance of death.

Fig. 6
figure 6

Decision tree model construction based on three critical variables for NSCLC patients

Full size image

Discussion

To date, the search for effective biomarkers to objectively assess the prognosis of lung cancer patients remains a prominent issue in clinical research. Clinicians aspire to achieve real-time bedside assessments and make decisions based on low-cost, readily accessible metrics. Several recent research advances have been achieved in predictive models related to non-small cell lung cancer (NSCLC), especially in the traditional TNM staging system and other emerging biomarker-based models. The TNM staging system is the most widely used classic tumor staging method. It has been applied as a standard tool for assessing the prognosis of patients with various cancer types, but its accuracy in individual survival prediction still needs to be improved. For example, a deep learning survival prediction model established for stage III NSCLC showed better prognostic predictive ability than the TNM staging system, with a C-index of 0.725 [19].

Some emerging prognostic biomarkers in peripheral blood, such as circulating tumor DNA (ctDNA) and the C-reactive protein-albumin-lymphocyte index (CALLY index), are increasingly used in the risk assessment of NSCLC. ctDNA detection can identify high-risk patients at the diagnosis phase, in the postoperative period, or during the neoadjuvant period, thus providing a basis for treatment adjustment [20]. The CALLY index has also been proposed as an essential indicator of prognosis in NSCLC patients, significantly correlated with survival with a C-index of 0.697. Predictive models based on the CALLY index have displayed better resolution and accuracy in predicting 3-year and 5-year survival than traditional TNM staging systems [21]. As a critical form of artificial intelligence, machine learning algorithms have been increasingly employed in clinical model-building missions due to their high performance, particularly in high-throughput omics data [22]. Despite the highly predictive performance of omics-related prognostic models constructed based on machine learning algorithms, the poor clinical accessibility of omics data and the lack of validation cohort are still essential reasons limiting the wide application of these models in clinical practice. Therefore, to balance the universal accessibility of modeling data and the high predictive performance of the model, our study adopted ten machine-learning algorithms to build a predictive model based on blood routine examination, intending to guide clinical practice.

In the present study, we first compared the levels of blood indices between patients in the survival and death groups. The results showed that most of the indices related to inflammation, lipid metabolism, and cellular immunity exhibited substantial differences between the two groups of patients, suggesting the potential predictive value. The candidate indices were then incorporated into ten machine-learning methods for training models and screening for variables. Evaluation results based on survival/death dichotomous modeling demonstrated that, compared to the traditional linear method (GLM), the voting ensemble learning methods that integrate multiple weak classifiers displayed higher prediction accuracy, which aligns with previous literature reports [23, 24]. In particular, the neural network model achieved excellent accuracy and AUC value. Furthermore, we identified the variables’ importance scores to improve the transparency of the neural network model by applying the SHAP16 algorithm. The results indicated that the top five ranked by importance were IL-6, sIL-2R, cholesterol, CEA, and Cy211, respectively. Furthermore, we also employed the random survival forest algorithm to construct models that can predict the survival of lung cancer patients. In addition to the classical tumor marker CEA, the importance ranking of variables obtained by related algorithms also confirmed the critical role of interleukin family members in the prognosis of lung cancer. Hence, the underlying association between interleukin and lung cancer warrants further investigation.

Since over 20 years ago, small sample clinical studies have found that serum levels of soluble interleukin-2 receptors (sIL-2R) were closely associated with the clinical stage and pathological type in lung cancer patients. Specifically, elevated levels of sIL-2R were concentrated in patients with non-small cell lung cancer, and the mean levels of sIL-2R in the advanced stage (III-IV) patients were significantly higher than in patients with early-stage lung cancer (I-II) [25]. It has been shown that preoperative sIL-2R concentrations displayed a positive correlation with the rate of intrapulmonary metastasis, and detecting the level of sIL-2R has potential values in monitoring intrapulmonary metastasis [26]. Moreover, another study found that for patients with advanced lung cancer, elevated sIL-2R was more often detected in patients with no response to chemotherapy, which also revealed its potential role in assessing chemotherapy efficacy [27]. Recent studies have demonstrated that sIL-2R levels can be recognized as a biomarker for monitoring the efficacy of chemotherapy combined with immunotherapy in patients with NSCLC, and higher levels of sIL-2R are strongly associated with poorer treatment response [28]. The studies described above unveiled the potential value of sIL-2R in the clinical application of lung cancer. However, the sample size is relatively small, most of the studies are early, and the clearly defined cut-off values have not been determined, so the practical clinical application value is limited. In our research, sIL-2R was screened from a wide range of laboratory indices by machine learning based on a large clinical sample, and its predictive value was confirmed by survival analysis and time-dependent ROC curves, further corroborating previous literature. Additionally, we also identified two cut-off points of sIL-2R, 416 U/mL, and 752 U/mL, by decision tree model, which may be more suitable for personalized prediction of lung cancer prognosis compared to the upper limit of clinical reference value 710 U/mL, which to some extent makes up for the deficiency of previous relevant studies.

Interleukin6 (IL-6) is recognized as a pleiotropic inflammatory factor. Studies have shown that IL-6 regulates several aspects of tumor cell fate, including proliferation, apoptosis, differentiation, and death. For instance, a previous study reported that IL-6 can up-regulate T-cell immunoglobulin domain and mucin domain 4 (TIM4) via the NF-κB signaling axis, which in turn promotes epithelial-to-mesenchymal transition and metastasis in NSCLC cell lines [29]. IL-6 can also up-regulate the expression of anti-apoptosis and DNA repair-related molecules by activating the downstream signaling pathways, including AKT, MAPK, STAT3, and ERK, thereby inducing the resistance of lung cancer cells to cisplatin chemotherapy [30]. Relevant clinical studies revealed that although IL-6 levels are not significantly correlated with the risk of lung cancer [31], elevated IL-6 levels are positively correlated with the deterioration of physical scores [32], which can serve as a potential marker for predicting disease progression and immunotherapy resistance in lung cancer patients [33, 34]. Moreover, another study demonstrated that IL-6-174G/C polymorphism was closely associated with the prognosis of lung cancer patients, but the in-depth mechanisms need further investigation [35]. Our findings also confirmed the significance of IL-6 in the prognosis of lung cancer patients, along with giving three cutoff values, 4.6 pg/mL, 5.7 pg/mL, and 13.0 pg/mL, which may aid physicians in better disease assessment and decision-making at the bedside.

Our research aims to construct a practical tool to assist the prognosis judgment of lung cancer patients based on clinical routine laboratory indices through artificial intelligence methods. This predictive model has potential applications for both rapid prognosis determination and clinical decision-making in lung cancer patients’ treatment. For example, patients with early lung cancer are in a dilemma about whether postoperative adjuvant therapy is necessary after surgery. If the predictive model evaluation results indicate a poor prognosis, more active intervention means, including postoperative adjuvant chemotherapy, are adopted to prevent the occurrence of postoperative recurrence and metastasis and shorten the follow-up interval. For advanced lung cancer patients, multiple chemotherapeutic applications or a combination of immunotherapy can be used to control tumor progression and improve the long-term prognosis according to the model prediction when the physical score of the patients permits. Regarding AUC values, our constructed model’s predictive performance is better than similar predictive models built based on clinical routine laboratory indices. However, whether the interventions based on predictive models can benefit lung cancer patients still needs to be confirmed by subsequent clinical studies.

Several limitations of the current study should be acknowledged and further improved. First, the study was retrospective in design and limited to a single center. Although it reflects the clinical treatment situation, the poor representation of cases and some inevitable confounding factors may introduce potential bias to the research results. Second, even though interleukins are readily accessible in clinical practice, levels of interleukin are susceptible to interference by diseases other than tumors, such as infectious and autoimmune diseases. We excluded patients with combined contagious and autoimmune diseases from this study, so the model’s predictive value for these patients may be limited. Third, it may be insufficient to objectively reflect the prognosis of patients only by using the laboratory tests at the initial visit of the patients, and it is worthwhile to utilize reasonable algorithms to analyze and explore the changes of dynamic indicators in the treatment course and the intrinsic association with the prognosis. Finally, although the predictive model built based on the neural network algorithm displayed relatively desirable predictive capabilities in both the training and validation sets, further evaluation of the model by an external cohort is still required.

Conclusion

In this study, we built clinical prognosis models for lung cancer patients based on ten machine-learning algorithms. The results indicated that the neural network model better predicted lung cancer patients’ survival prognosis. By integrating the variable importance rankings of neural network and random survival forest models, we screened and obtained three indices, CEA, sIL-2R, and IL-6, to construct a decision tree predictive model, which is helpful for clinicians to make quick prognostic assessment and clinical decision-making at the bedside.

Data availability

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding authors.

References

  1. Siegel RL, Miller KD, Fuchs HE, Jemal A. Cancer statistics, 2021. CA Cancer J Clin. 2021;71(1):7–33.

    Article  PubMed  Google Scholar 

  2. Chu T, Zhong R, Zhong H, Zhang B, Zhang W, Shi C, Qian J, Zhang Y, Chang Q, Zhang X, et al. Phase 1b study of Sintilimab Plus Anlotinib as First-line therapy in patients with Advanced NSCLC. J Thorac Oncol. 2021;16(4):643–52.

    Article  CAS  PubMed  Google Scholar 

  3. McLellan R, Marshall H, Dent A, Bowman RV, Yang IA, Fong KM. Diagnosis and treatment of early lung cancer. Aust J Gen Pract. 2020;49(8):508–12.

    Article  PubMed  Google Scholar 

  4. Frydrychowicz M, Kuszel L, Dworacki G, Budna-Tukan J. MicroRNA in lung cancer-a novel potential way for early diagnosis and therapy. J Appl Genet. 2023;64(3):459–77.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Wu J, Shen Z. Exosomal miRNAs as biomarkers for diagnostic and prognostic in lung cancer. Cancer Med. 2020;9(19):6909–22.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Chen K, Kang G, Zhang Z, Lizaso A, Beck S, Lyskjaer I, Chervova O, Li B, Shen H, Wang C, et al. Individualized dynamic methylation-based analysis of cell-free DNA in postoperative monitoring of lung cancer. BMC Med. 2023;21(1):255.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Yousefi M, Ghaffari P, Nosrati R, Dehghani S, Salmaninejad A, Abarghan YJ, Ghaffari SH. Prognostic and therapeutic significance of circulating tumor cells in patients with lung cancer. Cell Oncol (Dordr). 2020;43(1):31–49.

    Article  PubMed  Google Scholar 

  8. Yuan Y, Zhong H, Ye L, Li Q, Fang S, Gu W, Qian Y. Prognostic value of pretreatment platelet counts in lung cancer: a systematic review and meta-analysis. BMC Pulm Med. 2020;20(1):96.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Jin J, Yang L, Liu D, Li WM. Prognostic value of pretreatment lymphocyte-to-monocyte ratio in Lung Cancer: a systematic review and Meta-analysis. Technol Cancer Res Treat. 2021;20:1533033820983085.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Sun S, Qu Y, Wen F, Yu H. Initial neutrophil-to-lymphocyte ratio and platelet-to-lymphocyte ratio as prognostic markers in patients with inoperable locally advanced non-small-cell lung cancer. Biomark Med. 2020;14(14):1341–52.

    Article  CAS  PubMed  Google Scholar 

  11. Liu T, Zhou T, Luo F, Yang Y, Zhao S, Huang Y, Zhao H, Zhang L, Zhao Y. Clinical significance of kinetics of Low-Density Lipoprotein Cholesterol and its Prognostic Value in Limited Stage Small Cell Lung Cancer patients. Cancer Control. 2021;28:10732748211028257.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Zhou T, Zhan J, Fang W, Zhao Y, Yang Y, Hou X, Zhang Z, He X, Zhang Y, Huang Y, et al. Serum low-density lipoprotein and low-density lipoprotein expression level at diagnosis are favorable prognostic factors in patients with small-cell lung cancer (SCLC). BMC Cancer. 2017;17(1):269.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Garon EB, Chih-Hsin Yang J, Dubinett SM. The role of Interleukin 1beta in the pathogenesis of Lung Cancer. JTO Clin Res Rep. 2020;1(1):100001.

    PubMed  PubMed Central  Google Scholar 

  14. Leung JH, Ng B, Lim WW. Interleukin-11: a potential biomarker and molecular therapeutic target in Non-small Cell Lung Cancer. Cells 2022, 11(14).

  15. Wang XF, Zhu YT, Wang JJ, Zeng DX, Mu CY, Chen YB, Lei W, Zhu YH, Huang JA. The prognostic value of interleukin-17 in lung cancer: a systematic review with meta-analysis based on Chinese patients. PLoS ONE. 2017;12(9):e0185168.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Eberst G, Vernerey D, Laheurte C, Meurisse A, Kaulek V, Cuche L, Jacoulet P, Almotlak H, Lahourcade J, Gainet-Brun M, et al. Prognostic value of CD4 + T lymphopenia in non-small cell lung Cancer. BMC Cancer. 2022;22(1):529.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Zhao S, Jiang T, Zhang L, Yang H, Liu X, Jia Y, Zhou C. Clinicopathological and prognostic significance of regulatory T cells in patients with non-small cell lung cancer: a systematic review with meta-analysis. Oncotarget. 2016;7(24):36065–73.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Yuan Y, Giger ML, Li H, Sennett C. Correlative feature analysis on FFDM. Med Phys. 2008;35(12):5490–500.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Yang L, Fan X, Qin W, Xu Y, Zou B, Fan B, Wang S, Dong T, Wang L. A novel deep learning prognostic system improves survival predictions for stage III non-small cell lung cancer. Cancer Med. 2022;11(22):4246–55.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Abbosh C, Hodgson D, Doherty GJ, Gale D, Black JRM, Horn L, Reis-Filho JS, Swanton C. Implementing circulating tumor DNA as a prognostic biomarker in resectable non-small cell lung cancer. Trends Cancer. 2024;10(7):643–54.

    Article  CAS  PubMed  Google Scholar 

  21. Liu XY, Zhang X, Zhang Q, Ruan GT, Liu T, Xie HL, Ge YZ, Song MM, Deng L, Shi HP. The value of CRP-albumin-lymphocyte index (CALLY index) as a prognostic biomarker in patients with non-small cell lung cancer. Support Care Cancer. 2023;31(9):533.

    Article  PubMed  Google Scholar 

  22. Gao Y, Zhou R, Lyu Q. Multiomics and machine learning in lung cancer prognosis. J Thorac Dis. 2020;12(8):4531–5.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Benzekry S, Grangeon M, Karlsen M, Alexa M, Bicalho-Frazeto I, Chaleat S, Tomasini P, Barbolosi D, Barlesi F, Greillier L. Machine Learning for Prediction of Immunotherapy Efficacy in Non-Small Cell Lung Cancer from Simple Clinical and Biological Data. Cancers (Basel) 2021, 13(24).

  24. Liu W, Wang S, Ye Z, Xu P, Xia X, Guo M. Prediction of lung metastases in thyroid cancer using machine learning based on SEER database. Cancer Med. 2022;11(12):2503–15.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Yano T, Yoshino I, Yokoyama H, Fukuyama Y, Takai E, Asoh H, Ichinose Y. The clinical significance of serum soluble interleukin-2 receptors in lung cancer. Lung Cancer. 1996;15(1):79–84.

    Article  CAS  PubMed  Google Scholar 

  26. Kawashima O, Kamiyoshihara M, Sakata S, Endo K, Saito R, Morishita Y. The clinicopathological significance of preoperative serum-soluble interleukin-2 receptor concentrations in operable non-small-cell lung cancer patients. Ann Surg Oncol. 2000;7(3):239–45.

    Article  CAS  PubMed  Google Scholar 

  27. Brunetti G, Bossi A, Baiardi P, Jedrychowska I, Pozzi U, Bacchella L, Bernardo G. Soluble interleukin 2 receptor (sIL2R) in monitoring advanced lung cancer during chemotherapy. Lung Cancer. 1999;23(1):1–9.

    Article  CAS  PubMed  Google Scholar 

  28. Tozuka T, Yanagitani N, Yoshida H, Manabe R, Ogusu S, Tsugitomi R, Sakamoto H, Amino Y, Ariyasu R, Uchibori K, et al. Soluble interleukin-2 receptor as a predictive biomarker for poor efficacy of combination treatment with anti-PD-1/PD-L1 antibodies and chemotherapy in non-small cell lung cancer patients. Invest New Drugs. 2023;41(3):411–20.

    Article  CAS  PubMed  Google Scholar 

  29. Liu W, Wang H, Bai F, Ding L, Huang Y, Lu C, Chen S, Li C, Yue X, Liang X, et al. IL-6 promotes metastasis of non-small-cell lung cancer by up-regulating TIM-4 via NF-kappaB. Cell Prolif. 2020;53(3):e12776.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Duan S, Tsai Y, Keng P, Chen Y, Lee SO, Chen Y. IL-6 signaling contributes to cisplatin resistance in non-small cell lung cancer via the up-regulation of anti-apoptotic and DNA repair associated molecules. Oncotarget. 2015;6(29):27651–60.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Zhou B, Liu J, Wang ZM, Xi T. C-reactive protein, interleukin 6 and lung cancer risk: a meta-analysis. PLoS ONE. 2012;7(8):e43075.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. An J, Gu Q, Cao L, Yang H, Deng P, Hu C, Li M. Serum IL-6 as a vital predictor of severe lung cancer. Ann Palliat Med. 2021;10(1):202–9.

    Article  PubMed  Google Scholar 

  33. Liu C, Yang L, Xu H, Zheng S, Wang Z, Wang S, Yang Y, Zhang S, Feng X, Sun N, et al. Systematic analysis of IL-6 as a predictive biomarker and desensitizer of immunotherapy responses in patients with non-small cell lung cancer. BMC Med. 2022;20(1):187.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Naqash AR, McCallen JD, Mi E, Iivanainen S, Marie MA, Gramenitskaya D, Clark J, Koivunen JP, Macherla S, Jonnalagadda S et al. Increased interleukin-6/C-reactive protein levels are associated with the upregulation of the adenosine pathway and serve as potential markers of therapeutic resistance to immune checkpoint inhibitor-based therapies in non-small cell lung cancer. J Immunother Cancer 2023, 11(10).

  35. Gomes M, Coelho A, Araujo A, Azevedo A, Teixeira AL, Catarino R, Medeiros R. IL-6 polymorphism in non-small cell lung cancer: a prognostic value? Tumour Biol. 2015;36(5):3679–84.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all participants who assisted us in all steps of the study.

Funding

The study was supported by the National Natural Science Foundation of China (No. 81973795, 82204842), the YangFan project from the Science and Technology Commission of Shanghai Municipality (No. 22YF1445400), and the Traditional Chinese Medicine Science and Technology Development Project of Shanghai Medical Innovation & Development Foundation (No.WL-LXBB-2021001 K, WL-QNRC-2022003 K). The funders had no role in the design and conduct of the study; collection, management analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Author information

Author notes

  1. Yuli Wang, Na Mei, Ziyi Zhou these authors contributed equally to the manuscript.

Authors and Affiliations

  1. Clinical Medical Center of Oncology, Shanghai Municipal Hospital of Traditional Chinese Medicine, Shanghai University of Traditional Chinese Medicine, Shanghai, 200071, China

    Yuli Wang, Na Mei, Ziyi Zhou, Yuan Fang, Fanchen Zhao, Zhihong Fang & Yan Li

  2. Central Laboratory, Shuguang Hospital Affiliated to Shanghai University of Traditional Chinese Medicine, Shanghai, 201203, China

    Jiacheng Lin

Authors
  1. Yuli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Na Mei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ziyi Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuan Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiacheng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Fanchen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhihong Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YL: Design of the work; ZhF: Design of the work; YlW: Write and edit the manuscript; NM: Analysis; ZyZ: Interpretation of data; YF: Interpretation of data; JcL: Visualization; FcZ: Analysis.

Corresponding authors

Correspondence to Zhihong Fang or Yan Li.

Ethics declarations

Ethics in publishing

The Ethics Committee of Shanghai Municipal Hospital of Traditional Chinese Medicine approved the human studies. The studies were conducted under local legislation and institutional requirements. The participants provided written informed consent to participate in this study (approval number: 2023SHL-KY-20-01).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Author agreement

All authors had final approval of the version to be published.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Mei, N., Zhou, Z. et al. A novel prediction model for the prognosis of non-small cell lung cancer with clinical routine laboratory indicators: a machine learning approach. BMC Med Inform Decis Mak 24, 344 (2024). https://doi.org/10.1186/s12911-024-02753-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12911-024-02753-3

Keywords

BMC Medical Informatics and Decision Making

ISSN: 1472-6947

Contact us