AI-powered chatbot boosts engagement and efficiency in higher education

By incorporating a knowledge base of commonly asked questions and program-specific queries, the chatbot responds to both predefined and dynamically generated questions. Fixed responses serve prospective students seeking basic program information, while current students receive personalized answers, such as assignment deadlines or thesis requirements, through automated actions that query the Moodle database in real time.

Researchers at the Democritus University of Thrace have developed a new artificial intelligence-powered chatbot or conversational agent that assists postgraduate students in a fully online educational environment, delivering personalized assistance around the clock.

Published in Information, the study "AIMT Agent: An Artificial Intelligence-Based Academic Support System," introduces the AIMT Agent, a personalized conversational agent integrated into the Moodle learning management system (LMS). With 24 students rating it highly for usefulness and ease of use, the findings signal a leap forward in educational technology, particularly for distance learning.

Developed by researchers Chris Lytridis and Avgoustos Tsinakos, the AIMT Agent leverages the open-source Rasa framework, which provides natural language understanding and response capabilities. Unlike large language model-based systems, the Rasa-based approach was chosen for its flexibility, cost-effectiveness, and seamless integration with existing university infrastructure. The agent was installed on a dedicated local server and is accessible at all times to students enrolled in the university’s MSc program in Immersive Technologies - Innovation in Education, Training and Game Design.

By incorporating a knowledge base of commonly asked questions and program-specific queries, the chatbot responds to both predefined and dynamically generated questions. Fixed responses serve prospective students seeking basic program information, while current students receive personalized answers, such as assignment deadlines or thesis requirements, through automated actions that query the Moodle database in real time.

The conversational agent’s interface, implemented as a Moodle plugin, enables students to interact via text or voice, offering real-time responses within the Moodle platform. The system not only answers academic queries but also logs unanswered questions, helping administrators update the knowledge base for continuous improvement. In addition to academic support, the AIMT Agent also monitors student activity. If a student has not accessed the platform for a specified duration, the system automatically notifies both the student and their instructors, supporting proactive engagement strategies.

To assess the AIMT Agent’s effectiveness, the research team employed the Technology Acceptance Model (TAM), surveying 24 postgraduate students on perceived usefulness and ease of use. Results indicated a strong positive response to the system’s ease of use and moderate approval of its usefulness in supporting academic activities. While students appreciated time-saving features and centralized information access, they were less reliant on the agent as a primary academic tool.

The study noted that while most users successfully interacted with the chatbot, some queries failed to yield correct responses. Out of 196 recorded interactions during the evaluation period, 66.8% resulted in accurate answers, 16.8% returned either no answer or an incorrect one due to query formulation, and 11.2% were unrecognized due to gaps in the knowledge base. The authors attributed some inaccuracies to language issues, as most students were non-native English speakers, affecting phrasing and grammar.

System response times were found to be efficient, averaging 0.07 seconds for predefined responses and 0.92 seconds for personalized queries involving database access. These response times were well within user expectations, contributing to overall satisfaction with the system’s performance.

While the initial rollout of the AIMT Agent has been deemed a success, the research highlights opportunities for enhancement. Increasing the volume and variety of query examples in the knowledge base could improve comprehension and response accuracy. Future development goals include integrating the chatbot with actual course materials, such as lecture slides, quizzes, and wikis, to transform the system from an administrative assistant to a comprehensive academic companion.

The team further plans to beef up the knowledge base with more query variants and expand its reach into course materials. With a 66.8% success rate already, tweaks could push it higher, especially for non-English speakers.