University of Limerick
Fazil_2023_Transformer.pdf (647.24 kB)

Transformer architecture-based transfer learning forpoliteness prediction in conversation

Download (647.24 kB)
journal contribution
posted on 2023-09-25, 07:25 authored by Shakir Khan, Mohd Fazil, Agbotiname Lucky Imoize, Bayan Ibrahimm Alabduallah, Bader M. Albahlal, Saad Abdullah Alajlan, Abrar Almjally, Tamanna Siddiqui

Politeness is an essential part of a conversation. Like verbal communication, politeness in textual conversation and social media posts is also stimulating. Therefore, the automatic detection of politeness is a significant and relevant problem. The existing literature generally employs classical machine learning-based models like naive Bayes and Support Vector-based trained models for politeness prediction. This paper exploits the state-of-the-art (SOTA) transformer architecture and transfer learning for respectability prediction. The proposed model employs the strengths of context-incorporating large language models, a feed-forward neural network, and an attention mechanism for representation learning of natural language requests. The trained representation is further classified using a softmax function into polite, impolite, and neutral classes. We evaluate the presented model employing two SOTA pre-trained large language models on two benchmark datasets. Our model outperformed the two SOTA and six baseline models, including two domain-specific transformer-based models using both the BERT and RoBERTa language models. The ablation investigation shows that the exclusion of the feed-forward layer displays the highest impact on the presented model. The analysis reveals the batch size and optimization algorithms as effective parameters affecting the model performance.



Sustainability 15(14), 10828



Other Funding information

Deanship of Scientific Research at Mohammad Ibn Saud Islamic University (IMSIU) through grant number RP-21-07-06 and Princess Nourah Bint Abdulrahman University Researchers Supporting Project number (PNURSP2023R440).

Department or School

  • Computer Science & Information Systems

Usage metrics

    University of Limerick