A Stacking Ensemble Framework for improving Lung Cancer Prediction

Abstract

Lung cancer is the leading cause of cancer-related deaths worldwide and underscores the need to develop effective early detection tools, particularly in health environments with limited resources. This work offers a machine learning model for predicting the risk of lung cancer using non-imaging, formal clinical, and behavioral records from the Kaggle Sanjoli02 dataset. Its methodology uses a systematic, multi-step pipeline consisting of data cleaning, categorical encoding, feature engineering based on correlation, standardization, class-imbalance management with the Adaptive Synthetic (ADASYN), and dimensionality reduction with Principal Component Analysis (PCA) to keep the most discriminative data. A variety of classifiers are then tested, and the most effective heterogeneous learners are stacked in a two-layer ensemble to take advantage of complementary error behavior and increase the final predictive ability. The experimental outcomes prove the usefulness of the suggested framework, where a final classification accuracy of 99.58 is achieved, outperforming the state-of-the-art benchmarks and providing an inexpensive clinical decision-support system that does not require imaging.