The researchers analyzed anonymized records from 5,326 students enrolled in 360 courses across 48 majors and 18 fall and spring semesters from autumn 2014 through autumn 2023. The university’s information technology division supplied registration and grade data, which were combined with records of students’ majors, and the university institutional review board approved the use of the data. Students who had completed fewer than 10 courses or more than 60 were excluded, removing cases with insufficient history for prediction or histories considered atypically long. Courses taken fewer than 100 times were encoded as an “OTHER” token, as were majors with fewer than 500 students.

The data were divided by student, with 90% of students used for training and 10% held out for testing. For each student, the researchers created multiple input-output examples from the academic record. An input could contain a declared major and the courses and grades from the first semester, with the second semester’s courses and grades as the target. Later examples included two or more completed semesters as input and the next semester as output. Grade values were converted to standardized GPA values ranging from 0 for an F to 4.3 for an A+, and training used batches of 32 for 20 epochs on one NVIDIA RTX 3090 Ti.

TRACE was designed around a scheduling feature that many learning-analytics models flatten: courses taken in the same semester are concurrent rather than separate events in a meaningful order. Assigning every course within a semester the same positional encoding makes the representation invariant to their order while preserving the sequence of semesters. This differs from a token-level prediction scheme in which the order of courses could falsely appear important. It also gives the model a way to represent course load, because the number of simultaneously taken courses remains part of each term’s set. In the authors’ view, this is more faithful to academic histories than assigning an arbitrary rank to every course in a term.

The architecture uses separate learned embedding layers for majors, courses and grades. Course names were assigned random integer identifiers without using subject or course-number information, allowing the model to learn latent course representations from enrollment patterns rather than relying on hand-engineered features. Grades received their own embedding because numerical marks need not vary linearly and grading distributions may differ across courses. A Transformer encoder then applies multi-head self-attention and position-wise feed-forward networks to the sequence. Attention can connect a current course with relevant courses taken many semesters earlier, while a causal mask prevents information from future semesters entering the prediction. Padding masks block artificial sequence positions from the attention and loss calculations.

Course selection and grade estimation are combined in a multi-task objective. For each semester, TRACE produces a probability distribution over the course catalogue and pairs course identities with predicted grades. Because a semester’s courses have no natural order, the authors rejected ordinary token-by-token cross-entropy for this component: a correct prediction could be penalized simply because its courses were listed in a different order. Instead, the courses actually taken were represented as a binary vector normalized so its entries summed to one, and the predicted distribution was compared with that target through Kullback–Leibler divergence. Grade predictions were trained with mean squared error, and the total objective combined course loss with a weighted grade loss. Cross-validation was used to select the balance, with a weight near one used in the reported experiments.

During training, the Transformer operated through one-step-ahead examples. For a sequence containing courses and grades, each truncation divided the available history into an input and a future target, and the model iterated through the possible split points. The decoder read the encoder’s representation of prior semesters and used masked self-attention plus cross-attention to produce course and grade outputs. Masking also excluded grades for courses a student did not take, because no observed grade exists for those courses. The study therefore evaluated grade errors over courses actually enrolled in the target semester rather than allowing the accuracy of course selection to directly determine the reported grade metric.

To test whether the gains arose simply from attention or a newer neural architecture, the researchers compared TRACE with an encoder-decoder long short-term memory with attention, a unidirectional LSTM, a bipartite graph neural network and XGBoost. TRACE’s mean squared error was approximately 30% lower and its mean absolute error approximately 15% lower than those of the two LSTM models. Its mean squared and absolute errors were 15% to 20% lower than those of the graph neural network, which represented students and courses as linked nodes and propagated information through enrollment neighborhoods. The graph model required roughly 50% more training time on the same hardware. The authors associated that additional time with its per-prediction subgraph construction and message-passing operations, although they did not establish a definitive cause.

The largest difference came from an ablation that removed course prediction while retaining TRACE’s course embeddings and other architectural features. This grade-only Transformer had a mean absolute error of 0.2496, compared with 0.1339 for TRACE, while its mean squared error was 3.5 times greater. The result indicates that the auxiliary course-set task helped the model learn more informative representations of courses and their relationships. It does not show that the system predicts enrollment accurately enough to recommend courses. On the held-out students, TRACE achieved a course-prediction precision of 0.5076, recall of 0.4090 and F1 score of 0.4432 across the 360-course catalogue. The study noted that random selection of five courses would have expected precision of about 0.014, or 5 divided by 360, but enrollment decisions may also depend on preferences, timetables, advice and equivalent courses outside the model’s inputs.

Another ablation tested whether declared major supplied information beyond the transcript. Removing major slightly improved the reported results: mean squared error was 0.0360 rather than 0.0392, and mean absolute error was 0.1327 rather than 0.1339. The authors interpret this as suggesting that major-like patterns may already be recoverable from early course selections, but the experiment did not directly establish that mechanism. Keeping a major embedding nevertheless demonstrated how other attributes could be incorporated. The available data included only major alongside course and grade histories, however, so the model did not directly test demographic, financial-aid or learning-management-system variables. The study also identified future work involving set-oriented models such as Set Transformers and DeepSets, which might align more closely with the unordered nature of semester course sets.

Several limitations constrain generalization. The data came from a single, medium-sized private university, whose curriculum, course offerings and student population may not represent large public universities, community colleges or institutions in other countries. The model lacked information on socioeconomic conditions, engagement, classroom environment and non-cognitive skills, and historical grades may reflect disparities associated with instructors, demographics or socioeconomic circumstances. It also faced a cold-start problem for students without university records, while newly introduced courses absent from the training vocabulary were mapped to “OTHER,” limiting predictions for novel curricula. The authors propose that TRACE might eventually support early-alert programs by identifying course combinations or trajectories that warrant attention, but no advising workflow was deployed and the study did not measure interventions, retention or subsequent grades. Cross-institutional validation and subgroup-specific fairness audits remain necessary before the approach can be generalized or used operationally.

Subject of Research: Transformer-based joint prediction of concurrent university course sets and grades using decade-long institutional records.

Article Title: Jointly predicting courses and grades using a transformer-based model

Article References: Savala, P. (2026). Jointly predicting courses and grades using a transformer-based model. Discover Education, 5(1), Article 1201. https://doi.org/10.1007/s44217-026-02333-9

Image Credits: AI Generated

DOI: 10.1007/s44217-026-02333-9

Keywords: learning analytics, grade prediction, Transformers, course enrollment, semester concurrency, multi-task learning, student records, educational data mining, university students, machine learning, Jointly, predicting