Adversarial Data Augmentation With Vision Transformer for Image Classification Tasks

This work introduces an analysis to a new end-to-end hybrid model that adopts adversarial data augmentation using C-GANs in conjunction with Vision Transformers (ViT) to enhance image classification. ViT incorporates the multi-head self-attention to address the local and global features of the images to improve the accuracy of digit classification. Through the application of self-attention processes, the ViT can identify the local and global contexts in the images. In this work, the original images of MNIST are used along with the images that are created by the help of C-GAN model for improving image quality and dataset expansion. The ViT model is trained by tuning with specific hyperparameters such as number of epochs, weight decay, learning rate, batch size, to improve the classification outcomes. To investigate the influence of synthetic data and data augmentation, the model is assessed based on its performance. The use of both original and synthesized data into the ViT framework contributes to a more diverse model with better generalization with the accuracy of 0.98.