- Home
- Current Issue
- Article
Sentence Boundary Detection in Colloquial Tamil Using Monolingual Adaptation and Fine-Tuned mBART50
Authors
1 Department of Information Technology, Sri Venkateswara College of Engineering, Sriperumbudur, Tamil Nadu, India. 2 Department of Computer Science and Engineering, Sri Venkateswara College of Engineering, Sriperumbudur, Tamil Nadu, India.
Article Information
- Published In
- Volume 7, Issue 6 (2026)
- Published Online
- November-December 2026
- Pages
- 100-105
- Article Type
- Original Article
Abstract
View PDFSentence boundary detection is an essential preprocessing task that converts continuous text into sentence-level units for downstream natural language processing applications. It is particularly challenging in colloquial text, where punctuation is inconsistent and informal expressions differ from standard written forms. This study investigates a sequence-to-sequence approach to sentence boundary detection in colloquial Tamil using mBART50. In the first stage, the pretrained model is adapted using approximately 5,700 monolingual colloquial Tamil sentences, reducing perplexity from 2.7859 to 2.2857. In the second stage, a separate segmentation dataset is prepared by recombining sentences into synthetic paragraph examples containing two to five sentences. These examples are shuffled and used for mixed-difficulty supervised fine-tuning. The proposed configuration achieves a boundary-level precision of 0.8169, recall of 0.8506, and F1-score of 0.8334. The findings indicate the potential of monolingual adaptation for this task under the prepared evaluation setting. However, evaluation on larger, independently sourced conversational Tamil data is needed to assess performance on naturally occurring text.
Related Articles
Share Article
*Instagram doesn't support direct link sharing from web. Copy the link and share it in your Instagram story or post.