Beyond Accuracy: Protocol-Dependent Robustness and Reliability in Patch-Based Breast Histopathology Classification

Abstract

Breast cancer is one of the diseases that cause a dead worldwide among women. Early diagnosis can save many lives and this will lead to a proper treatment. Histopathological image analysis is an important diagnostic method for detecting breast cancer.  Computer-aided diagnosis of breast images helps radiologists do the task more efficiently and appropriately. The reliable evaluation of deep learning models in computational pathology depends strongly on how the data is split during validation. In patch-based histopathology classification, random image-level splitting can obscure inter-patient heterogeneity and produce overly stable performance estimates.This study investigates the effect of the evaluation protocol on Invasive Ductal Carcinoma (IDC) detection using the Breast Histopathology Images dataset. A lightweight Convolutional Neural Network (CNN) was trained using balanced sampling and evaluated under two 5-fold cross-validation schemes: strict patient-level separation and stratified random image-level splitting. Under patient-level evaluation, the model achieved a mean accuracy of 0.7826 ± 0.0295 and a mean AUC of 0.8691 ± 0.0257, with a Brier score of 0.1582 ± 0.0159. Random splitting produced similar mean accuracy (0.7867 ± 0.0096) and AUC (0.8652 ± 0.0053), with slightly improved Brier score (0.1509 ± 0.0047). While paired statistical testing showed no significant differences in mean performance (p > 0.05), patient-level evaluation exhibited substantially higher fold-wise variability. Calibration analysis indicated moderate reliability with mild overconfidence, and error inspection revealed consistent morphological failure patterns. Overall, the evaluation protocol mainly affects perceived robustness rather than average performance, highlighting the importance of patient-level validation and variability reporting.