Student Dropout Prediction using XGBoost with Boruta Feature Selection, Hyperparameter Optimization, and SHAP Interpretation

Silvia Maulidia Mirosa, Muhammad Iqbal Akbar

Abstract


Predicting students at risk of dropping out can help higher education institutions identify patterns associated with students' academic status. This study develops a student status classification model (Graduate and Dropout) using XGBoost, Boruta feature selection, hyperparameter optimization through GridSearchCV, and SHAP-based interpretation. The study uses the Predict Students’ Dropout and Academic Success dataset from the UCI Machine Learning Repository, comprising 3,630 records and 36 predictor variables after excluding the Enrolled class. Three model configurations were compared: baseline XGBoost, Boruta + XGBoost, and Boruta + GridSearchCV + XGBoost. Boruta reduced the number of features from 36 to 11, representing a reduction of 69.44%. The Boruta + GridSearchCV + XGBoost model achieved an accuracy of 91.74% and an F1-score of 89.21%, comparable to the baseline model (accuracy: 91.74%; F1-score: 89.36%). The McNemar test revealed no statistically significant differences between the models (p > 0.05). SHAP identified Curricular units 2nd sem (approved), Course, and Tuition fees up to date as the features with the greatest contributions to the model's predictions. Accuracy decreased to 90.22% when the second-semester feature was removed and to 74.93% when all academic Curricular units features were excluded.

Keywords


Boruta feature selection; GridSearchCV; SHAP; student dropout prediction; XGBoost

Full Text:

PDF

References


A. J. Fernández-García, J. C. Preciado, F. J. Melchor-González, R. Rodríguez-Echeverría, J. M. Conejero, and F. Sánchez-Figueroa, “A Real-Life Machine Learning Experience for Predicting University Dropout at Different Stages using Academic Data,” IEEE Access, Vol. 9, pp. 133076–133090, 2021, DOI: 10.1109/ACCESS.2021.3115851.

L. G. R. Putra, D. D. Prasetya, and Mayadi, “Student Dropout Prediction using Random Forest and XGBoost Method,” INTENSIF, Vol. 9, No. 1, pp. 147–157, 2025, DOI: 10.29407/intensif.v9i1.21191.

V. Realinho, J. Machado, L. Baptista, and M. V. Martins, “Predicting Student Dropout and Academic Success,” Data, Vol. 7, No. 11, p. 146, 2022, DOI: 10.3390/data7110146.

M. L. Damar Sakti et al., “The Effect of Sample Size on the Stability of XGBoost Model Performance in Predicting Student Study Period,” BAREKENG J. Ilmu Mat. dan Terap., Vol. 19, No. 4, pp. 2679–2692, 2025, DOI: 10.30598/barekengvol19iss4pp2679-2692.

B. Carballo-Mendívil, A. Arellano-González, N. J. Ríos-Vázquez, and M. del P. Lizardi-Duarte, “Predicting Student Dropout from Day One: XGBoost-based Early Warning System using Pre-Enrollment Data,” Appl. SCI., Vol. 15, No. 16, p. 9202, 2025, DOI: 10.3390/app15169202.

S. Praptomo, A. Risman, and R. M. Suri, “Explainable Machine Learning for Predicting Student Dropout and Academic Success using XGBoost and SHAP,” J. Electr. Eng. Comput., Vol. 8, No. 1, pp. 222–231, 2026, DOI: 10.33650/jeecom.v8i1.16978.

H. Ahmadpour, O. Bazrafshan, E. Rafiei-Sardooi, H. Zamani, and T. Panagopoulos, “Gully Erosion Susceptibility Assessment in the Kondoran Watershed using Machine Learning Algorithms and the Boruta Feature Selection,” Sustainability, Vol. 13, No. 18, p. 10110, 2021, DOI: 10.3390/su131810110.

A. Mincevičius, “Interpretable Machine Learning Approaches for Student Dropout Prediction in Higher Education,” New Trends Comput. SCI., Vol. 4, No. 1, pp. 146–160, 2026, DOI: 10.3846/ntcs.2026.27184.

E. Elgeldawi, A. Sayed, A. R. Galal, and A. M. Zaki, “Hyperparameter Tuning for Machine Learning Algorithms used for Arabic Sentiment Analysis,” Informatics, Vol. 8, No. 4, p. 79, 2021, DOI: 10.3390/informatics8040079.

M. Li, H. Sun, Y. Huang, and H. Chen, “Shapley Value: from Cooperative Game to Explainable Artificial Intelligence,” Auton. Intell. Syst., Vol. 4, No. 1, p. 2, 2024, DOI: 10.1007/s43684-023-00060-8.

R. Mitchell, E. Frank, and G. Holmes, “GPUTreeShap: Massively Parallel Exact Calculation of SHAP Scores for Tree Ensembles,” PeerJ Comput. SCI., Vol. 8, p. e880, 2022, DOI: 10.7717/peerj-cs.880.

I. K. Nti and S. Ramanayake, “Explainable Machine Learning for Student Dropout Prediction and Tailored Interventions in Online Personalized Education,” Discov. Artif. Intell., Vol. 6, No. 1, p. 288, 2026, DOI: 10.1007/s44163-026-01016-6.

M. Quimiz-Moreira, R. Delgadillo, J. Parraga-Alava, N. Maculan, and D. Mauricio, “Factors, Prediction, Explainability, and Simulating University Dropout through Machine Learning: A Systematic Review, 2012–2024,” Computation, Vol. 13, No. 8, p. 198, 2025, DOI: 10.3390/computation13080198.

R. Goran et al., “Identifying and Understanding Student Dropouts using Metaheuristic Optimized Classifiers and Explainable Artificial Intelligence Techniques,” IEEE Access, Vol. 12, pp. 122377–122400, 2024, DOI: 10.1109/ACCESS.2024.3446653.

A. M. Priyatno, Y. Ningsih, R. J. Farhas, F. I. Firmananda, R. K. Sari, and Aryadi, “Multi-Stage Feature Selection for Optimizing Student Dropout Prediction,” J. Eng. Technol. Ind. Appl., Vol. 11, No. 55, pp. 86–95, 2025, DOI: 10.5935/jetia.v11i55.1910.

E. Alzabidi, Ö. Yakut, S. Saad, and O. Fındık, “SOD-FE: A Supervised Outlier Detection and Feature Engineering Approach for Student Dropout Prediction,” SCI. Rep., Vol. 16, No. 1, p. 25906, 2026, DOI: 10.1038/s41598-026-56591-6.

F. Y. Dewi, B. R. Pratama, and S. Sunaryono, “Perbandingan Grid Search dan Random Search untuk Optimasi Hyperparameter Random Forest pada Klasifikasi Kanker Payudara,” Bull. Comput. SCI. Res., Vol. 6, No. 4, pp. 1489–1497, 2026, DOI: 10.47065/bulletincsr.v6i4.1087.

S. Kapoor and A. Narayanan, “Leakage and the Reproducibility Crisis in Machine-Learning-based Science,” Patterns, Vol. 4, No. 9, p. 100804, 2023, DOI: 10.1016/j.patter.2023.100804.

S. Bag, K. Gupta, and S. Deb, “A Review and Recommendations on Variable Selection Methods in Regression Models for Binary Data,” arXiv Prepr. arXiv2201.06063, 2022, DOI: 10.48550/arXiv.2201.06063.

G. Canbek, T. Taskaya Temizel, and S. Sagiroglu, “PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics,” SN Comput. SCI., Vol. 4, p. 13, 2023, DOI: 10.1007/s42979-022-01409-1.




DOI: https://doi.org/10.32520/stmsi.v15i10.6993

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.