Comparing the Accuracy and Interpretability of CNN, Transformer, and ConvNeXt for Skin Lesion Classification

Abstract

Early detection of skin cancer plays an important role in improving treatment outcomes, particularly in cases of melanoma. Numerous studies have employed deep learning models for skin lesion classification; however, most have focused on a single architectural paradigm and placed greater emphasis on improving classification accuracy than on model interpretability. This study presents a comparative analysis of three deep learning architectural paradigms: EfficientNet-B4 as a representative Convolutional Neural Network (CNN), Swin Transformer as a representative Vision Transformer (ViT), and ConvNeXt as a representative modernized CNN for the classification of seven types of skin lesions using the HAM10000 dataset. All models were trained using consistent training configurations, with analytical class weighting applied to address class imbalance. Model performance was evaluated using accuracy, precision, recall, Macro F1-score, and confusion matrices, while Explainable AI (XAI) techniques, including Grad-CAM and attention maps, were employed to assess the transparency of the models' decision-making processes. The experimental results show that ConvNeXt achieved the best overall performance, with an accuracy of 91.62% and a Macro F1-score of 0.8897. It also achieved higher recall for the melanoma class than EfficientNet-B4 and Swin Transformer. The XAI visualizations further demonstrate that ConvNeXt was able to focus its attention on lesion regions that were more clinically relevant compared with the other two models.
These findings indicate that ConvNeXt provides a better balance between classification performance and model interpretability, suggesting its potential as a reliable approach for supporting artificial intelligence-based skin lesion diagnosis systems.