Integration of U-Net, GAN-Based Image Augmentation, and a Custom CNN for Breast Cancer Segmentation and Classification

Abstract

Background: Breast cancer remains a leading cause of mortality worldwide, and early detection is critical for improving patient outcomes. Artificial intelligence (AI) has demonstrated significant promise in this field, yet challenges such as inaccurate lesion localization, limited augmentation strategies, and class imbalance persist. Aim: This study proposes a hybrid deep learning framework that integrates U-Net for lesion segmentation, a Deep Convolutional GAN (DCGAN) for synthetic image augmentation, and a custom Convolutional Neural Network (CNN) for classification of breast ultrasound images. The dataset, obtained from the Kaggle Breast Ultrasound Images (BUSI) repository, consists of 780 grayscale scans from 600 female patients aged 25–75, categorized into normal, benign, and malignant classes. Method: The proposed U-Net–GAN–CNN pipeline was trained for 20 epochs, with 15% of the dataset reserved for independent testing. Performance was evaluated using accuracy, precision, recall, and F1-score. Result: The model achieved 97% accuracy, 96% precision, and 95% recall and F1-score on the training set. On the test set, performance remained robust, with 95.5% accuracy, 94% precision, 93% recall, and 93.5% F1-score. Comparative evaluation against three baselines, a CNN trained on raw images, a CNN with traditional augmentation, and a fine-tuned VGG16, demonstrated the superiority of the proposed approach. These findings confirm that integrating segmentation, GAN-based augmentation, and a tailored CNN enhances diagnostic reliability, mitigates class imbalance, and provides a reproducible framework with strong potential to support radiologists in clinical decision-making.