Graduate students rank content and interface above accessibility in platform survey

Graduate students participating in a university-based survey rated the educational substance of an online learning platform as more important than its accessibility or interactive features, according to an Analytic Hierarchy Process study by QHQinyan Huang and Guanlei Yin at Guangxi Minzu Normal University in China. Fifty graduate students with prior experience of online learning assessed four dimensions: content quality, user interface, accessibility and interactivity. After their pairwise judgments were aggregated and normalized, content quality received the largest priority weight, 0.416, or 41.6%, followed by user interface at 0.297, or 29.7%. Interactivity ranked third with 0.147, or 14.7%, and accessibility ranked fourth with 0.140, or 14.0%. The findings describe what this group considered important when evaluating platforms; they do not show that the criteria caused better engagement or academic results.

The survey was administered through a structured questionnaire to graduate students at Guangxi Minzu Normal University who had used online learning platforms in coursework. The researchers used a discipline-stratified convenience-sampling approach, with participants drawn from engineering, management, education and the social sciences. Invitations were circulated through departmental mailing lists and graduate-program coordinators: 68 students were invited, 50 completed the questionnaire, and the response rate was 73.5%. Most participants reported at least one year of experience with online learning, while commonly used platforms included Moodle, Coursera and Google Classroom. All participants gave written informed consent, and the questionnaire was distributed digitally to preserve anonymity. The participants were at different stages of graduate study and were required to have previous experience with online learning platforms so that they could make informed comparative judgments about the four dimensions.

Each respondent compared the four criteria in pairs using Saaty’s nine-point AHP scale, on which 1 represents equal importance and 9 represents extreme importance. With four criteria, each participant made six pairwise comparisons. These judgments were converted into an individual pairwise-comparison matrix. To combine the judgments into a group result, the researchers calculated the geometric mean of corresponding entries across the 50 matrices. They then divided each matrix element by its column sum, averaged the normalized values in each row and rescaled the resulting weights so that they summed to 1. The final weights describe relative priority within this four-criterion model, not an estimate of an objective effect on learning. The AHP hierarchy had two levels: the overall goal of evaluating online learning platforms and the four criteria beneath it. Moodle, Coursera and Google Classroom were discussed as representative examples but were not entered as competing alternatives in the calculation.

Consistency testing gave the aggregated judgments a Consistency Ratio of 0.038, below the commonly used 0.10 threshold for acceptable consistency in AHP analysis. The principal eigenvalue of the four-by-four matrix was 4.1031, and the random index used for a four-criterion matrix was 0.90. The researchers also reported that a geometric-mean verification produced nearly identical weights and that changing key pairwise entries by one scale step did not alter the rank order. The questionnaire was researcher-developed from the four criteria identified through a literature review, and its content was based on established AHP pairwise-comparison procedures. Because the instrument collected comparative judgments rather than responses on a psychometric Likert scale, the researchers did not calculate a measure such as Cronbach’s alpha. They instead assessed the coherence of the comparisons through the AHP consistency procedure. The authors caution that consistency supports the internal coherence of the aggregated preferences but does not establish their validity for all students or institutions.

The criteria represented distinct dimensions of platform use. Content quality covered the accuracy, relevance, structure, currency, completeness and alignment of learning materials, including the suitability of formats such as text, video, images, simulations and assessment materials. User interface concerned navigation, page organization, dashboards, resource access and the ease of completing academic tasks. Accessibility included the ability to use resources across devices, locations and internet conditions, as well as features such as screen-reader compatibility, alternative text, adjustable text size and keyboard navigation. Interactivity encompassed opportunities for communication, participation, responses and collaboration through tools such as quizzes, polls, discussion forums, peer assessment and collaborative activities. Other concepts discussed in the literature, including flexibility, scalability, feedback, self-regulation and learning analytics, were not modeled as separate criteria in the AHP hierarchy.

The prominence given to content and interface design is consistent with the way these features were defined in the study. Accurate, current and logically structured materials may help students connect new information with prior knowledge and identify course objectives, while an orderly interface may reduce the effort needed to locate resources and move among lectures, assignments and assessments. Clear layouts and consistent navigation can support academic tasks, whereas complex navigation and poorly organized information can add unnecessary demands as students work with course materials. The study therefore presents a perceptual hierarchy rather than a direct test of those mechanisms. Although online learning platforms can collect navigation and participation records, the questionnaire did not use behavioral records, examination grades, grade-point averages or course-completion rates to evaluate engagement or performance. References to academic performance concern respondents’ perceived relevance of the platform characteristics to learning, not objectively measured learning gains.

Interactivity received a comparatively low weight in the group model, despite the role assigned to quizzes, feedback and peer participation in the surrounding literature. Interactive quizzes, polls and discussion boards can give students opportunities to participate, test their understanding and receive responses, while feedback can help them identify misunderstandings and adjust their learning strategies. The AHP result does not show that interactive features are unimportant to engagement. Instead, it indicates that this sample allocated the largest shares of relative importance to content and interface when comparing the four selected criteria. Accessibility, which also ranked below content and interface, represents a condition for meaningful participation: a platform cannot deliver an effective learning experience if students cannot consistently reach or use its resources. The nearly adjacent weights for interactivity and accessibility should therefore not be interpreted as evidence that either dimension can be neglected.

The hierarchy did not rank Moodle, Coursera and Google Classroom against one another. Those systems appeared in the study as representative examples of learning-management, massive-open-online-course and collaborative platforms, but no platform-level alternatives entered the AHP calculation and no alternative priority vectors were produced. The analysis consequently cannot identify one of the three services as superior under the model. Moodle was described as a customizable learning-management system supporting forums, quizzes, assignments, peer assessment and analytics; Coursera as a scalable MOOC environment offering university-linked courses, quizzes, peer-graded work and discussion forums; and Google Classroom as a comparatively simple collaborative environment integrated with productivity tools. The platform descriptions provide context for the type of environment in which students had learned, but they were not tested against one another as ranked alternatives.

The results offer a context-specific prioritization for platform design and evaluation. Within this sample, institutions that want to align platform selection with graduate students’ stated priorities would have the strongest numerical basis in the model for prioritizing strong instructional materials and a clear, usable interface. Accessibility and interactivity remain legitimate design dimensions, particularly for students with differing access needs and for learning activities built around feedback or collaboration, but their lower weights do not translate into measurable benefits or harms. The authors call for replication with larger and more diverse samples, probability-based recruitment, domain experts, specific platform alternatives and longer-term study of whether priorities change as students gain more online-learning experience. Future work could also compare student and expert judgments and examine uncertainty in the pairwise assessments. With the present sample drawn from one Chinese university, the ranking should be treated as a bounded finding rather than a universal formula for evaluating online education.

Subject of Research: Graduate students’ perceived priorities for evaluating online learning platforms

Article Title: A multi-criteria evaluation of online learning platforms for student engagement and academic performance: an Analytic Hierarchy Process

Article References: Huang, Q., & Yin, G. (2026). A multi-criteria evaluation of online learning platforms for student engagement and academic performance: an Analytic Hierarchy Process. Frontiers in Psychology, 17, Article 1906004. https://doi.org/10.3389/fpsyg.2026.1906004

Image Credits: AI Generated

DOI: 10.3389/fpsyg.2026.1906004

Keywords: online learning platforms, Analytic Hierarchy Process, content quality, user interface, accessibility, interactivity, student engagement, graduate students, higher education, pairwise comparison, China, multi-criteria