AI Course Advisers Trade Accuracy for Answers, Landmark Study Finds

Every student who has ever stared at an online course catalogue, paralyzed by thousands of options, has met the recommender system. These algorithms quietly shape what millions of learners study next, yet most of them operate as inscrutable black boxes, spitting out suggestions without a word of justification. Now a team of researchers at the University of Cagliari has delivered one of the most rigorous audits to date of how well different families of recommendation algorithms perform when they are forced to explain themselves, and the results expose a fundamental tension at the heart of educational artificial intelligence: the models that recommend courses most accurately are often the ones least able to say why.

The study, published in the journal Data Mining and Knowledge Discovery, was led by Neda Afreen, together with Ludovico Boratto, Gianni Fenu, Francesca Maridina Malloci, Mirko Marras, and Alessandro Soccol. Their starting point was a growing enthusiasm for knowledge graph reasoning as a route to explainable recommendation. A knowledge graph represents a domain as a network of entities and relationships: learners, courses, concepts, prerequisites, and enrollment histories all become nodes and edges. Reasoning methods can then trace multi-hop paths through this network, connecting a student to a suggested course through a chain of intermediate facts. Such a path can be converted into a human-readable sentence, for example that a course is recommended because it builds on a concept the learner already covered in a previous class.

That sounds ideal for education, where trust and transparency matter enormously, but the Cagliari team found the evidence base surprisingly shaky. Most knowledge graph reasoning methods are evaluated in entertainment and e-commerce, where the stakes of a bad suggestion are low and the data is dense. Studies in education have been scattered and inconsistent, using different datasets, techniques, and evaluation metrics, making it nearly impossible to know which approaches genuinely work. The researchers therefore set out to run a controlled comparison: they transformed four public educational datasets into knowledge graph structures and evaluated a broad spectrum of state-of-the-art methods under a single, unified evaluation protocol.

The lineup of contenders reads like a taxonomy of modern recommender systems. At one end sat traditional collaborative filtering models, including efficient neural matrix factorization and disentangled variational approaches, which learn patterns from user behavior alone. Next came knowledge-aware methods that weave graph information into the recommendation process without explicit path reasoning. Then the explainability specialists: PGPR, a reinforcement learning agent that learns to walk paths through the knowledge graph; CAFE, a neuro-symbolic reasoner that combines coarse neural search with fine symbolic refinement; and two generative models, PLM and PEARLM, which use path language modeling to generate recommendation trajectories as if they were sentences. Each was judged on utility metrics such as ranking accuracy, on beyond-utility qualities like novelty, diversity, and serendipity, and on explainability measures including the diversity of explanation types.

The headline finding is a stark trade-off. Traditional models such as ENMF and MacridVAE consistently achieved the highest utility across all four datasets, meaning they placed the courses students actually took at the top of their ranked lists more reliably than any other paradigm. Knowledge-aware models like CFKG proved competitive in mid-density settings such as the MOOCube dataset, where enough interaction data exists for graph signals to help. But the knowledge graph reasoning methods, the very approaches designed to produce explanations, generally exhibited lower ranking accuracy. In other words, the price of a legible justification is often a worse recommendation, at least when accuracy is the yardstick.

Yet the story is not simply that explainable methods lose. When the researchers looked beyond raw accuracy into sparse datasets, where user interaction data is thin, the reasoning-based and generative approaches revealed distinct strengths. PGPR and PLM delivered strong performance on serendipity-related metrics, reaching a score of 0.95 on the MOOPer dataset, suggesting they can surface pleasantly unexpected course suggestions that a pure accuracy optimizer would never propose. PEARLM led in explanation type diversity across all data settings, meaning it could justify its choices in more varied ways rather than recycling the same template. CAFE, meanwhile, generated explanations grounded in popular entities, which the authors note may enhance familiarity and interpretability for learners, since people tend to trust reasons that reference things they recognize.

These nuances matter because different educational contexts demand different virtues. A platform recommending electives to thousands of university students might prioritize ranking accuracy and accept opacity. A tutoring system guiding an individual learner through a curriculum, where the student must understand and consent to each step, might willingly sacrifice a few percentage points of accuracy for explanations that build trust. The Cagliari results give system designers, for the first time, a like-for-like map of where each paradigm sits on that spectrum, replacing a literature of incomparable case studies with a unified audit.

The technical machinery behind the comparison is itself a contribution. Converting raw educational datasets into knowledge graphs required decisions about what counts as an entity and a relation: enrollments become edges between learners and courses, courses link to the concepts they cover, and prerequisite relationships connect concepts to one another. The evaluation then had to balance three metric families that do not always pull in the same direction. Utility metrics reward putting the right course first. Beyond-utility metrics reward surfacing items that are novel, diverse, or serendipitous, which can actively conflict with accuracy. Explainability metrics assess whether the system can produce meaningful, varied justifications. Running all methods through this triple lens, on identical data splits, is what allows the trade-offs to be quantified rather than merely asserted.

The work also carries a broader message for the explainable AI movement. There is a temptation to assume that transparency is a free upgrade, a matter of bolting explanations onto existing systems. This study suggests otherwise: explanation capability is baked into a model’s architecture, and the paradigms that reason explicitly over knowledge structures behave differently from those that optimize embeddings for ranking. Reinforcement learning path walkers, neuro-symbolic reasoners, and generative path language models each occupy their own niche, excelling at serendipity, popular-entity grounding, or explanation diversity respectively. Choosing among them is not a technical afterthought but a design decision about what a recommender is for.

For the fast-growing world of online education, where massive open online courses and digital campuses increasingly rely on algorithmic guidance, the implications are immediate. Developers of learning platforms now have open evidence, and the researchers have released their source code publicly, on which to base architecture choices that match their pedagogical values. The study also flags open questions: whether the trade-offs hold as datasets grow denser, how learners actually perceive and use the generated explanations, and whether hybrid systems can capture both the accuracy of collaborative filtering and the legibility of path reasoning. What is clear is that the era of the silent course adviser may be ending, and the systems that replace it will have to choose, deliberately, between being right and being understood.

Subject of Research: Explainable course recommendation using knowledge graph reasoning methods

Article Title: Explainable course recommendation with knowledge graphs: a comparative audit of diverse modeling paradigms

Article References: Afreen, N., Boratto, L., Fenu, G., Malloci, F. M., Marras, M., & Soccol, A. (2026). Explainable course recommendation with knowledge graphs: a comparative audit of diverse modeling paradigms. Data Mining and Knowledge Discovery, 40(6), Article 94. https://doi.org/10.1007/s10618-026-01261-4

Image Credits: AI Generated

DOI: 10.1007/s10618-026-01261-4

Keywords: knowledge graphs, recommender systems, explainable AI, course recommendation, education technology, machine learning, reinforcement learning, neuro-symbolic reasoning, generative models, collaborative filtering, serendipity, transparency