- Home
- Archives
- 2026
- Vol. 7 Issue 6
- Article
Domain-Adapted Distilbert for Semantic Representation of Bibliographic Records on Library and Scopus Datasets
Authors
1 Research Scholar, Department of Computer Science, Hindusthan College of Arts and Science, Coimbatore, Tamilnadu, India. 2 Associate Professor, Department of Computer Science with Cognitive Systems, Sri Ramakrishna College of Arts and Science, Coimbatore Tamilnadu, India.
Article Information
- Published In
- Volume 7, Issue 6 (2026)
- Published Online
- November-December 2026
- Pages
- 75-82
- Article Type
- Original Article
- Check for updates
-
References
1. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proc. 2019 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, Minneapolis, MN, USA, 2019, pp. 4171–4186, doi: 10.18653/v1/N19-1423.
2. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” arXiv:1910.01108, 2019.
3. A. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” in Proc. 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 8342–8360, doi: 10.18653/v1/2020.acl-main.740.
4. N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proc. 2019 Conf. Empirical Methods in Natural Language Processing and 9th Int. Joint Conf. Natural Language Processing, 2019, pp. 3982–3992, doi: 10.18653/v1/D19-1410.
5. R. K. Kaliyar, A. Goswami, and P. Narang, “FakeBERT: Fake news detection in social media with a BERT-based deep learning approach,” Multimedia Tools and Applications, vol. 80, no. 8, pp. 11765–11788, 2021, doi: 10.1007/s11042-020-10183-2.
6. J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Proc. 56th Annual Meeting of the Association for Computational Linguistics, vol. 1, 2018, pp. 328–339, doi: 10.18653/v1/P18-1031.
7. Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019.
8. K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “ELECTRA: Pre-training text encoders as discriminators rather than generators,” in Proc. 8th Int. Conf. Learning Representations, 2020.
9. Z. Lan et al., “ALBERT: A lite BERT for self-supervised learning of language representations,” in Proc. 8th Int. Conf. Learning Representations, 2020.
10. X. Jiao et al., “TinyBERT: Distilling BERT for natural language understanding,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 4163–4174, doi: 10.18653/v1/2020.findings-emnlp.372.
11. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv:1301.3781, 2013.
12. J. Pennington, R. Socher, and C. D. Manning, “GloVe: Global vectors for word representation,” in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, 2014, pp. 1532–1543, doi: 10.3115/v1/D14-1162.
13. M. E. Peters et al., “Deep contextualized word representations,” in Proc. 2018 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, 2018, pp. 2227–2237, doi: 10.18653/v1/N18-1202.
14. T. Wolf et al., “Transformers: State-of-the-art natural language processing,” in Proc. 2020 Conf. Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45, doi: 10.18653/v1/2020.emnlp-demos.6.
15. C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge University Press, 2008.
2. V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” arXiv:1910.01108, 2019.
3. A. Gururangan, A. Marasović, S. Swayamdipta, K. Lo, I. Beltagy, D. Downey, and N. A. Smith, “Don’t stop pretraining: Adapt language models to domains and tasks,” in Proc. 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 8342–8360, doi: 10.18653/v1/2020.acl-main.740.
4. N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” in Proc. 2019 Conf. Empirical Methods in Natural Language Processing and 9th Int. Joint Conf. Natural Language Processing, 2019, pp. 3982–3992, doi: 10.18653/v1/D19-1410.
5. R. K. Kaliyar, A. Goswami, and P. Narang, “FakeBERT: Fake news detection in social media with a BERT-based deep learning approach,” Multimedia Tools and Applications, vol. 80, no. 8, pp. 11765–11788, 2021, doi: 10.1007/s11042-020-10183-2.
6. J. Howard and S. Ruder, “Universal language model fine-tuning for text classification,” in Proc. 56th Annual Meeting of the Association for Computational Linguistics, vol. 1, 2018, pp. 328–339, doi: 10.18653/v1/P18-1031.
7. Y. Liu et al., “RoBERTa: A robustly optimized BERT pretraining approach,” arXiv:1907.11692, 2019.
8. K. Clark, M.-T. Luong, Q. V. Le, and C. D. Manning, “ELECTRA: Pre-training text encoders as discriminators rather than generators,” in Proc. 8th Int. Conf. Learning Representations, 2020.
9. Z. Lan et al., “ALBERT: A lite BERT for self-supervised learning of language representations,” in Proc. 8th Int. Conf. Learning Representations, 2020.
10. X. Jiao et al., “TinyBERT: Distilling BERT for natural language understanding,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 4163–4174, doi: 10.18653/v1/2020.findings-emnlp.372.
11. T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” arXiv:1301.3781, 2013.
12. J. Pennington, R. Socher, and C. D. Manning, “GloVe: Global vectors for word representation,” in Proc. 2014 Conf. Empirical Methods in Natural Language Processing, 2014, pp. 1532–1543, doi: 10.3115/v1/D14-1162.
13. M. E. Peters et al., “Deep contextualized word representations,” in Proc. 2018 Conf. North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, 2018, pp. 2227–2237, doi: 10.18653/v1/N18-1202.
14. T. Wolf et al., “Transformers: State-of-the-art natural language processing,” in Proc. 2020 Conf. Empirical Methods in Natural Language Processing: System Demonstrations, 2020, pp. 38–45, doi: 10.18653/v1/2020.emnlp-demos.6.
15. C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge, U.K.: Cambridge University Press, 2008.
Related Articles
Share Article
*Instagram doesn't support direct link sharing from web. Copy the link and share it in your Instagram story or post.