Generative AI-Based Image Captioning System Using Vision Transformers and Large Language Models

DOI

https://www.doi.org/10.59256/ijire.20260702008

Published In

Volume 7, Issue 2 (2026)

Published Online

March-April 2026

Pages

67-73

Article Type

Original Article

Check for updates

Recent developments of Generative Artificial Intelligence (AI) and Large Language Models (LLMs) have helped tremendously in improving the capabilities of machines to comprehend and write human-like language. Nonetheless, linking the visual information with natural language comprehension is a difficult issue to be solved in artificial intelligence. This project is aimed at the creation of the intelligent image captioning system that will provide the link between the computer vision and natural language processing. The suggested system processes the visual information in images and automatically produces valuable and contextual text description of the visual images, which will allow the machines to understand and describe the visual scenes in a better way. The system combines computer vision models, which are based on deep learning, and language generation techniques, which are based on transformers. One of the pretrained vision encoders in the form of a Convolutional Neural Network (CNN) or a Vision Transformer (ViT) is then applied to images and the visual features are then processed by a language decoder to create coherent captions. To train the model and evaluate it, publicly available datasets (MS COCO and Flickr8k/Flickr30k) are used. It is implemented with the help of Python and deep learning systems such as PyTorch or TensorFlow. Standard measures such as BLEU, METEOR and ROUGE are used to measure performance. The suggested system shows how generative AI can be used to enhance image understanding, accessibility, and automated content description with the help of computer vision.