LGI: Label-Graph Inference Multi-Benchmark Emotion Detection Transformer Framework for Intelligent Chatbots

Karam Muayad Abdullah

Abstract


The fundamental ability of intelligent chatbots is to detecting emotions evident in intelligent recommendation systems, learning agent systems and mental health systems. Emotions are a key element in facilitating interaction and communication between humans. Researchers face challenges in studying the transmission of emotions through text. Despite significant advances in language transformation models, emotion recognition remains a major challenge due to the high semantic overlap between emotion categories and unbalanced distributions in datasets. This paper suggests an improved framework for enhanced transformer model based on a Label-Graph Interface (LGI) for ratings to robustly detect emotions across texts using various criteria specific to social media and conversations between humans and human-machine. It combines a pretrained transformer encoder with lightweight label relation inference layer that uses a row normalized graph prior and sample adaptive gate to propagate evidence among related emotion labels. In single label case, cross entropy loss is used to optimize the model, but in multi label detection a binary cross entropy is used with logits, class aware positive weighting and threshold selection based on validation sets.
Six datasets were used in order to assess the framework: DAIR Emotion, achieved an macro-F1 to 89.10%. TweetEval Emotion is the second dataset reached an macro-F1 is 81.56%. DailyDialog, the macro-F1 arrived to 58.62%. GoEmotions, got micro-F1 about 60.03%. Empathetic Context dataset reached to 85.80% Top-3 accuracy and macro-F1 is 85.69% on the 32-class task. Lastly EmoWOZ is the most important dataset because it contains real conversation between human-machine has an accuracy about 92.86%, macro-F1 to 59.95% and AUC to 94.97%.
The results demonstrate that the significant value and usefulness of relational inference at the classification level for emotion-based chatbot systems.

Keywords


intelligent chatbot; emotion detection; label-graph inference; transformer encoder; natural language processing

Full Text:

PDF

References


S. Kusal, S. Patil, J. Choudrie, K. Kotecha, D. Vora, and I. Pappas, "A Systematic Review of Applications of Natural Language Processing and Future Challenges with Special Emphasis in Text-based Emotion Detection", Artificial Intelligence Review, Vol. 56, No. 12, pp. 15129-15215, 2023, DOI: 10.1007/s10462-023-10509-0.

F. A. Acheampong, H. Nunoo-Mensah, and W. Chen, "Transformer Models for Text-based Emotion Detection: A Review of BERT-based Approaches". Artificial Intelligence Review, Vol. 54, No. 8, pp. 5789-5829, 2021, DOI: 10.1007/s10462-021-09958-2.

P. Thakur, D.R. Shrivastava, and A. DR, "A Review on Text based Emotion Recognition System", International Journal of Advanced Trends in Computer Science and Engineering, Vol. 7, No. 5, 2018.[Online]. Available: http://www.warse.org/IJATCSE/static/pdf/file/ijatcse01752018.pdf.

X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, "Pre-Trained Models for Natural Language Processing: A Survey", Science China Technological Sciences, Vol. 63, No. 10, pp. 1872-1897, 2020, DOI: 10.1007/s11431-020-1647-3.

N. M. Gardazi, A. Daud, M. K. Malik, A. Bukhari, T. Alsahfi, and B. Alshemaimri, "BERT Applications in Natural Language Processing: A Review", Artificial Intelligence Review, Vol. 58 No. 6, Art. No. 166, 2025, DOI: 10.1007/s10462-025-11162-5.

M. A. K. Raiaan, M. S. H. Mukta, K. Fatema, N. M. Fahad, S. Sakib, M. M. J. Mim, and S. Azam, "A Review on Large Language Models: Architectures, Applications, Taxonomies, Open Issues and Challenges", IEEE Access, Vol. 12, pp. 26839-26874, 2024, DOI: 10.1109/ACCESS.2024.3365742.

A. Al Maruf, F. Khanam, M. M. Haque, Z. M. Jiyad, M. F. Mridha, and Z. Aung, "Challenges and Opportunities of Text-based Emotion Detection: A Survey", IEEE Access, Vol. 12, pp. 18416-18450, 2024, DOI: 10.1109/ACCESS.2024.3356357.

U. Frevert, M. Scheer, and A. Schmidt, "Emotional Lexicons: Continuity and Change in the Vocabulary of Feeling 1700-2000", Oxford Unversity Prees, 2014. [Online]. Avalabile: https://reviews.history.ac.uk/review/1682.

M. V. Mäntylä, D. Graziotin, and M. Kuutila,"The Evolution of Sentiment Analysis—A Review of Research Topics, Venues, and Top Cited Papers", Computer Science Review, Vol. 27, pp 16-32, 2018, DOI: 10.1016/j.cosrev.2017.10.002.

S. Jaiswal, and G. C. Nandi, "Robust Real-Time Emotion Detection System using CNN Architecture", Neural Computing and Applications, Vol. 32, No. 15, pp. 11253-11262, 2020, DOI: 10.1007/s00521-019-04564-4.

A. Akula, , G. Budha, G. Bingi, U. Chanda, A. R. Borra, D. B. Yadav, and M. Saravanan, "Emotion Recognition from Facial Expressions using CNNs", International Journal of Engineering & Extended Technologies Research (IJEETR), Vol. 8, No. 1, pp. 120-125, 2026, DOI: doi.org/10.15662/IJEETR.2026.0801013.

C. Yerukonda, "Multimodal Context-Aware Emotion Recognition using Transformer-based Fusion of Facial and Speech Cues", In 7th International Conference on Intelligent Communication Technologies and Virtual Mobile Networks (ICICV), pp. 1782-179, IEEE, 2026, DOI: 10.1109/ICICV68925.2026.11554829.

Y. Ghafoor, S. Jinping, F. H. Calderon, Y. H. Huang, K. T. Chen, and Y. S. Chen, "TERMS: Textual Emotion Recognition in Multidimensional Space", Applied Intelligence, Vol. 53, No. 3, pp. 2673-2693, 2023, DOI: 10.1007/s10489-022-03567-4.

Z. Ye, T. Zuo, W. Chen, Y. Li, and Z. Lu, "Textual Emotion Recognition Method based on ALBERT-BiLSTM Model and SVM-NB Classification", Soft Computing, Vol. 27,No. 8, pp. 5063-5075, 2023, DOI: 10.1016/j.specom.2022.03.002.

J. K. Rout, K. K. R. Choo, A. K. Dash, S. Bakshi, S. K. Jena, and K. L. Williams, "A Model for Sentiment and Emotion Analysis of Unstructured Social Media Text", Electronic Commerce Research, Vol. 18, pp. 181-199, 2018, DOI: 10.1007/s10660-017-9257-8.

J. Song, M. Wang, and Y. Li, "An Exploratory Study of Deep Learning-based Sentiment Analysis among Weibo Users in China", Current Psychology, pp. 1-14, 2023, DOI: 10.1007/s12144-023-05493-1.

C. Li, and F. Li, "Emotion Recognition of Social Media Users based on Deep Learning", PeerJ Computer Science, Vol. 9, Art. No. e1414, 2023, DOI: 10.7717/peerj-cs.1414.

J. Cahn, "CHATBOT: Architecture, Design, & Development", Ph.D. Thesis, University of Pennsylvania School of Engineering and Applied Science Department of Computer and Information Science, 2017.

E. Adamopoulou, and L. Moussiades, "An Overview of Chatbot Technology", In IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 373-383, Springer, Cham, 2020, DOI: 10.1007/978-3-030-49186-4_31.

M. Nißen, D. Selimi, A. Janssen, D. R. Cardona, M. H. Breitner, T. Kowatsch, and F. von Wangenheim, "See You Soon Again, Chatbot? A Design Taxonomy to Characterize User-Chatbot Relationships with Different Time Horizons", Computers in Human Behavior, Vol. 127, Art. No. 107043, 2022, DOI: 10.1016/j.chb.2021.107043.

G. Bilquise, S. Ibrahim, and K. Shaalan, "Emotionally Intelligent Chatbots: A Systematic Literature Review", Human Behavior and Emerging Technologies, Vol. 2022, No. 1, 2022, pp. 9601630, DOI: 10.1155/2022/9601630.

M. Ehtesham-Ul-Haque, J. D’Rozario, R. Adnin, F. T. Utshaw, F. Tasneem, I. J. Shefa, and A. A. Al Islam, "EmoBot: Artificial Emotion Generation through an Emotional Chatbot during General-Purpose Conversations", Cognitive Systems Research, Vol. 83, Art. No. 101168, 2024, DOI: 10.1016/j.cogsys.2023.101168.

M. Kozłowski, K. Gabor-Siatkowska, I. Stefaniak, M. Sowański, and A. Janicki, "Enhanced Emotion and Sentiment Recognition for Empathetic Dialogue System using Big Data and Deep Learning Methods", In International Conference on Computational Science, Cham: Springer Nature Switzerland, pp. 465-480, 2023, DOI: 10.1007/978-3-031-35995-8_33.

S. Lee, Y. Park, and G. Park, "Using AI Chatbots in Climate Change Mitigation: A Moderated Serial Mediation Model", Behaviour & Information Technology, Vol. 43, pp.1-17, 2024, DOI: 10.1080/0144929X.2023.2298305.

E. Saravia, H. C. T. Liu, Y. H. Huang, J. Wu, and Y. S. Chen, "CARER: Contextualized Affect Representations for Emotion Recognition", In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3687-3697, 2018, DOI: 10.18653/v1/D18-1404.

F. Barbieri, J. Camacho-Collados, L. E. Anke, and L. Neves, "TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification", In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1644-1650, 2020, DOI: 10.18653/v1/2020.findings-emnlp.148.

S. Mohammad, F. Bravo-Marquez, M. Salameh, and S. Kiritchenko, "Semeval-2018 Task 1: Affect in Tweets", In Proceedings of the 12th International Workshop on Semantic Evaluation, pp. 1-17, 2018, DOI: 10.18653/v1/S18-1001.

Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu, "Dailydialog: A Manually Labelled Multi-Turn Dialogue Dataset", In Proceedings of the Eighth International Joint Conference on Natural Language Processing, Vol. 1, pp. 986-995, 2017, [Online]. Available: https://aclanthology.org/I17-1099/

D. Demszky, D. Movshovitz-Attias, J. Ko, A. Cowen, G. Nemade, and S. Ravi, "GoEmotions: A Dataset of Fine-Grained Emotions", In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4040-4054, 2020, DOI: 10.18653/v1/2020.acl-main.372.

H. Rashkin, E. M. Smith, M. Li, and Y. L. Boureau, "Towards Empathetic Open-Domain Conversation Models: A New Benchmark and Dataset", In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 5370-5381, 2019, DOI: 10.18653/v1/P19-1534.

S. Feng, N. Lubis, C. Geishauser, H. C. Lin, M. Heck, C. van Niekerk, and M. Gasic, "Emowoz: A Large-Scale Corpus and Labelling Scheme for Emotion Recognition in Task-Oriented Dialogue Systems", In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pp. 4096-4113, 2022, [Online]. Available: https://aclanthology.org/2022.lrec-1.436/

D. Ghosal, N. Majumder, A. Gelbukh, R. Mihalcea, and S. Poria, "Cosmic: Commonsense Knowledge for Emotion Identification in Conversations", In Findings of the association for computational linguistics: EMNLP 2020, pp. 2470-2481, 2020, DOI: 10.18653/v1/2020.findings-emnlp.224.

B. Vu, N. Keshri, S. Chandna, M. Jalali, S. Mehraeen, "A Systematic Approach to Fine-Tuning Transformers for Emotion Detection on the Empathetic Dialogues Benchmark", International Journal of Information Technology, Vol. 17, No. 7, pp.3895-3912, 2025 DOI: https://doi.org/10.1007/s41870-025-02645-3.

J. Yan, P. Pu, and L. Jiang, “Emotion-RGC Net: A Novel Approach for Emotion Recognition in Social Media using RoBERTa and Graph Neural Networks”, Plos one, Vol. 20, No. 3, Art. No. e0318524, 2025, DOI: 10.1371/journal.pone.0318524.




DOI: https://doi.org/10.32520/stmsi.v15i8.6669

Article Metrics

Abstract view : 1 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.