The research, published in the Journal of New Approaches in Educational Research by Francielle Marques, Davinia Hernández-Leo, and Carlos Castillo, analyzed official course satisfaction surveys from two academic years, 2021-2022 and 2022-2023, at the Spanish university. The dataset originally covered 3,066 bachelor-level course groups across 25 degree programs. To ensure statistical reliability, the team applied a filtering method based on the bound of the error of estimation in survey sampling, known as the Reliability Assessment Score, keeping only groups whose responses were numerous enough to yield dependable averages. That left 586 course groups distributed across nine schools, spanning communication, economics and business, engineering, health and life sciences, humanities, law, political and social sciences, and translation and language sciences.

What makes the study distinctive is its decision to deconstruct satisfaction itself. Rather than treating the survey as a single number, the researchers separated the items that probe satisfaction with specific facets of the learning design: the teaching methodology, including the design of class sessions, student activities, and materials; the perceived workload, measured against the credits assigned to the subject; and compliance with the teaching plan and obligations. Students rated each dimension on a numerical scale from zero, complete dissatisfaction, to ten, complete satisfaction. Crucially, the surveys were administered toward the end of each quarter or semester but before final exams, meaning that professors received summaries only weeks after grades were submitted, ruling out any feedback loop in which evaluations could influence grading.

The first major finding concerns what actually drives overall course satisfaction. Satisfaction with teaching methodology correlated strongly with the holistic judgment of the subject, and satisfaction with the teaching received tracked it closely. By contrast, satisfaction with the workload and with compliance with the teaching plan showed comparatively weaker correlations with overall satisfaction. All correlations were statistically significant, with p-values below 0.001. In practical terms, the way a course is taught, its sessions, activities, and materials, appears to dominate how students remember and judge the entire experience, while the sheer volume of work plays a smaller, though still measurable, role.

Then comes the question at the heart of the grade-bias debate. Using aggregate performance statistics for each course group, the proportions of students failing, passing, earning a notable, or achieving excellence, the researchers estimated average performance and correlated it with every satisfaction measure. The result: correlations were weak, ranging from 0.16 to 0.25, though statistically significant with p-values below 0.0001. Satisfaction scores were spread across a broad range at every level of performance. Notably, the highest satisfaction ratings were almost never found in courses where performance was abnormally low, where fewer than one in five students passed. Instead, the densest cluster of satisfied students sat in courses where large majorities passed, often with high grades, a pattern that is more consistent with good teaching producing both learning and satisfaction than with grades purchasing goodwill.

The study also examined contextual factors. Class size emerged as a consistent negative influence: larger classes, particularly those with 100 or more students, tended to be penalized in satisfaction scores for methodology, workload, and overall subject satisfaction. Course level showed only a weak relationship with overall satisfaction. These contextual effects matter because they represent factors largely outside an individual instructor’s control, and the authors argue that institutional analytics should account for them before survey results are used to judge teaching quality or make promotion decisions.

Perhaps the most intriguing analysis concerns workload outliers. Focusing on courses where at least some students rated the workload poorly, the team split the data into cases where satisfaction with teaching was higher than expected given the workload ratings, and cases where it was lower. In the first group, satisfaction with the teacher’s fulfillment of obligations and adherence to the teaching plan was high and showed low variance, suggesting that students are more forgiving of a demanding, or even excessive, workload when the professor honors the plan and meets their obligations. In the second group, satisfaction dropped across multiple dimensions, with the largest gap appearing in ratings of methodology, the very dimension most strongly tied to overall course satisfaction. Workload, in other words, does not operate in isolation; its effect on evaluations depends heavily on the surrounding quality of the teaching.

The authors frame their analytical approach through the concept of generative uncertainty, an interpretive stance in which institutional data is used not to deliver final verdicts but to provoke productive questions. Under this view, a dashboard showing a course with low satisfaction and low grades should trigger inquiry rather than punishment: is the problem the design of the course, the size of the class, the composition of the cohort, or something else entirely? The researchers argue that universities should build this uncertainty-aware mindset into their institutional analytics, using patterns in the data to generate new hypotheses and continuous reflection rather than treating survey averages as self-evident truths about teaching quality.

The implications cut against a belief that is widespread among faculty and echoed in some earlier studies, which reported that grading leniency boosts evaluations. In this Spanish context, at least, the hypothesis that professors who give good grades secure better student satisfaction found only minimal support. Courses with strong methodological design achieved high satisfaction ratings that could not be explained by grades or by easy workloads. The authors are careful about scope: the study covers a single university and two academic years, and only aggregate grade distributions were available rather than individual scores, which limits how precisely average performance could be estimated. They call for future work incorporating more years of data, finer-grained performance measures, learning management system interactions, and qualitative analysis of the written comments students leave in evaluations.

Still, the core message is one that universities, instructors, and students can all use. Satisfaction surveys are not pure noise, and they are not simply mirrors of the gradebook. They respond most powerfully to the visible craft of teaching, the design of sessions, activities, and materials, and they punish impersonal, oversized classes. Grades leave only a faint statistical fingerprint on how students rate their courses, and even that fingerprint is inconsistent across contexts. For institutions that rely on these surveys to steer quality assurance and career advancement, the lesson is to read them holistically, alongside performance data and contextual factors, and to remain alert both to bias and to the possibility that, sometimes, the survey is telling the truth about teaching after all.

Subject of Research: Bias and learning design factors in higher education student satisfaction surveys

Article Title: Beyond bias in student satisfaction surveys: exploring the role of grades and satisfaction with the learning design

Article References: Marques, F., Hernández-Leo, D., & Castillo, C. (2025). Beyond bias in student satisfaction surveys: exploring the role of grades and satisfaction with the learning design. Journal of New Approaches in Educational Research, 14(1), Article 9. https://doi.org/10.1007/s44322-025-00030-3

Image Credits: AI Generated

DOI: 10.1007/s44322-025-00030-3

Keywords: student satisfaction surveys, higher education, learning design, grade bias, teaching evaluations, institutional analytics, class size, workload, teaching methodology, course quality, educational research, Pompeu Fabra University