‏Saedah Mohammed Omar Albeerish

Abstract:

This study seeks to explore the complex relationship between psychological mechanisms for measuring user satisfaction and technical challenges related to the accuracy of the response of NLP-based interactive chatbots (NLP) and large language modeling (LLMs) in modern electronic platforms. This research is driven by a clear gap between the accelerated investment and institutional expansion of these technologies, and the actual levels of consumer satisfaction, as statistics indicate that 58% of companies rely on chatbots.  Users are only 37% satisfied with the accuracy of their answers.  The study is based on a qualitative-analytical design framework to deconstruct these phenomena. Psychologically, the study employs “expectation violation theory” (EVT) to explain how input interfaces (free text vs. selected options) affect the formulation of users’ expectations and default behavior: click-based interfaces create a high expectation ceiling that makes any technical error a “severe negative violation,” while free text provides an opportunity to achieve a “positive violation” that enhances satisfaction when an accurate answer is provided.  Technically, the study highlights the structural deterioration faced by models in multi-turn conversations, where robots lose about 39% of their basic accuracy and their reliability collapses by up to 112% due to complex phenomena such as “lost-in-middle” influence, premature answer attempts, answer inflation and error accumulation. The study also examines the phenomenon of “conversational compliance”) that lead bots to sacrifice their correct conclusions and get caught up in the wrong user assumptions.  The study found the inadequacy of traditional operational metrics, and the need to adopt advanced semantic evaluation frameworks based on linguistic models as a judge (LLM-as-a-judge) such as the RAG triad  (retrieval accuracy, reliability, and answer relevance) to ensure the quality of the response and bridge the gap between technical performance and the user’s psychological awareness.

 Keywords:

( Chatbots, Natural Language Processing (NLP), User Satisfaction, Expectation Violation Theory (EVT), Multi-Turn Conversations, Response Accuracy, RAG Triad, Dialogical Compliance (Sycophancy) )