Analyzing the Impact of SMOTE on Predictive Performance and Random Forest Feature Importance Stability in Financial Distress Prediction

Abstract

Financial distress prediction is often challenged by class imbalance, which can cause classification models to become biased toward the majority class. The Synthetic Minority Over-sampling Technique (SMOTE) is widely used to improve minority-class detection; however, its impact on the consistency of feature importance as a basis for model interpretation remains relatively underexplored. This study examines the impact of SMOTE on predictive performance and feature importance stability in Random Forest models using the Taiwan corporate bankruptcy dataset, comprising 6,819 companies, of which 220 were bankrupt, and 39 financial ratios retained after feature selection. Three scenarios were compared: Random Forest without class imbalance treatment (RF), Random Forest with class weighting (RF + CW), and Random Forest with SMOTE applied within the cross-validation pipeline (RF + SMOTE). All three scenarios were optimized using the same hyperparameter search space through five-fold GridSearchCV, with the F2-score as the optimization metric. Differences in test-set performance were evaluated using paired bootstrap and McNemar's test, while feature importance stability was assessed using Spearman's rank correlation, Kendall's tau, and Jaccard similarity across 10 random seeds. Compared with RF, SMOTE significantly increased recall from 0.2045 to 0.6818 (95% CI for the difference: 0.341 to 0.636), but reduced precision from 0.8182 to 0.3659. Changes in ROC-AUC (from 0.9439 to 0.9551) and PR-AUC were not statistically significant. The predictive performance of RF + SMOTE also did not differ significantly from that of RF + CW. Feature importance rankings changed moderately after SMOTE was applied (Spearman's correlation: 0.644 ± 0.017), indicating greater variation than that observed across random seeds (0.970). Nevertheless, eight financial ratios, including Total Debt/Total Net Worth and Persistent EPS in the Last Four Seasons, consistently appeared among the top 10 features across all repetitions. These findings indicate that SMOTE primarily shifts the model's decision boundary toward greater minority-class detection rather than improving its discriminatory ability. Therefore, the interpretation of feature importance should prioritize financial ratios that remain stable across different modeling scenarios.