Domain-Adapted Distilbert for Semantic Representation of Bibliographic Records on Library and Scopus Datasets

DOI

https://www.doi.org/10.59256/ijire.20260706008

Published In

Volume 7, Issue 6 (2026)

Published Online

November-December 2026

Pages

75-82

Article Type

Original Article

Check for updates

Bibliographic records incorporate conventional language (text) and structured data, with the domains of all the text fields involving vocabulary and norm conventions. General-purpose pretrained language models might capture some more general regularities of language while missing out on terminology occurring in library catalogues and scholarly-indexing records. In this paper, we introduce Domain-Adapted DistilBERT (DA-DistilBERT), a domain-adaptive text-representation method which further continues masked-language-model pretraining based on two different sources of corpus fields, a self-collected Library Dataset and a Scopus Dataset. In the Library corpus, titles, author strings and call numbers were utilized and in the Scopus corpus, titles and publisher names were utilized. The contextual representations are obtained from the candidate Transformer layers and scores are taken by using Cosine similarity and Spearman's rank correlation. According to Synopsis-reported results, for Library Dataset, DA-DistilBERT scores a cosine similarity of 0.934 and a Spearmen-ρ score of 0.912 while for Scopus Dataset, the scores of DA-DistilBERT is a cosine similarity of 0.921 and a Spearmen-ρ score of 0.896. These values exceed the corresponding reported scores for Vanilla BERT (0.891/0.856 and 0.874/0.831) and FakeBERT (0.913/0.883 and 0.899/0.862). The absolute gain in cosine similarity with Vanilla BERT are 0.043 and 0.047 and the absolute gain in Spearman ρ are 0.056 and 0.065 for the Library and Scopus data sets respectively. The results show the potential of using domain adaptation for representation of bibliographic works in the given evaluation protocol. However, the provided synopsis does not give out the corpus size, training hyperparameters, manner in building the evaluation-pair and specification on confidence intervals and repeated-runs statistics are not provided, which are required for independent replications, and statistical interpretations. It gives the mathematical formation of the same and a broadcasting technique that can be copied without the addition of any missing trial information.