What Student Clicks Reveal and How AI Can Keep Those Records Local?

The researchers examined the Open University Learning Analytics Dataset, known as OULAD, and EdNet to determine which could support a meaningful picture of exploratory behavior.

A student returns to a reading, opens several resources and checks a course forum across different days, leaving behind a trail that grades alone cannot describe. Those records could help researchers understand how students explore online learning spaces, raising a practical question about how much personal activity needs to be gathered in one place.

The study 'Privacy-aware learning analytics for exploratory behaviour through federated and explainable AI,' published in Frontiers in Artificial Intelligence, examines whether useful patterns can be recovered from these digital traces through a system that keeps raw records local. Rommel Gutierrez and colleagues combined behavioral analysis, distributed AI training and methods for explaining model decisions, finding that decentralization preserved both performance and the main signals behind the results in their simulation.

Student exploration is more than a click count

The researchers examined the Open University Learning Analytics Dataset, known as OULAD, and EdNet to determine which could support a meaningful picture of exploratory behavior. EdNet's inspected collections were largely organized around answering questions, making OULAD better suited to studying resource browsing, content access and activity beyond assessments. OULAD supplied 10,900,970 interaction records, which the team organized into 29,228 units representing a student's activity within a particular course module and course offering. These units describe learning trajectories rather than 29,228 separate people.

Forum activity accounted for roughly 22.6% of recorded interactions, with substantial activity involving educational content and navigation pages. Students differed in how they used these spaces: a large number of clicks could be concentrated on a small collection of resources, and a less active student could visit a broader range of materials.

The researchers built an exploratory score from five equally weighted components: activity unrelated to assessment, resource diversity, revisits, the spread of activity over time and irregularity in navigation. The highest-scoring quarter, comprising 7,307 trajectories, received a high-exploration label.

What the AI Results Actually Tell Us

The team tested how much the labels changed when they adjusted the score cutoff or altered its ingredients. Moving the cutoff from the 75th percentile to the 70th or 80th percentile preserved about 95% agreement across all labels, with substantial overlap among the trajectories classified as highly exploratory.

Adding forum participation or activity outside the expected course week reshaped the selected group, and the expanded seven-component version produced an overlap score below 0.50 against the original high-exploration group. The scores formed a continuous distribution without a clear natural divide, making the cutoff a practical research choice whose meaning depends on the definition of exploration.

Two centralized models, logistic regression and random forest, were tested on their ability to recover those labels. Logistic regression achieved an F1 score of 0.977 and an AUC-ROC of 1.000; random forest reached 0.920 and 0.993. F1 measures the balance between finding relevant cases and avoiding incorrect labels, and AUC-ROC measures how well a model distinguishes the two groups.

The near-perfect result needs careful interpretation because the models were recovering a label built from related behavioral variables. It demonstrates that the scoring rule could be reconstructed, rather than proving that AI can independently recognize real-world informal learning.

Removing the variables directly used to construct the score provided a tougher check. Logistic regression retained an F1 of 0.905 and AUC-ROC of 0.991, with random forest reaching 0.873 and 0.983, suggesting that related information existed across the broader activity patterns.

Keeping Records Local Without Losing Performance

Federated learning lets separate participants train a shared model by exchanging model updates and retaining their raw records locally. The study used FedAvg, a method that combines those local updates into a shared model.

The researchers simulated seven participants divided by academic module and a second arrangement containing ten balanced random groups. A compact neural network was trained centrally and through these distributed arrangements, providing a comparison using the same model architecture. The central network and both main federated versions recorded an F1 of 0.671 and AUC-ROC of 0.886. A version that added a small amount of Gaussian noise to the updates produced almost identical results, at 0.673 and 0.887.

Federated training preserved this network's performance; it did not match the stronger logistic regression results. Estimated communication for exchanging updates remained modest, at approximately 1.616 MB for the module arrangement and 2.308 MB for the random groups over 35 training rounds. Individual module results ranged from below 0.50 to above 0.75 on F1, showing that a stable shared result can coexist with substantially weaker performance for particular groups.

No raw interaction records were exchanged during the simulated federated training. The system did not include secure aggregation, cryptographic protection or formal differential privacy, so local storage and the noise experiment should not be read as guarantees against information leakage.

Consistent Explanations and Questions for Future Research

To examine what the models relied on, the researchers shuffled individual input variables and measured the effect on performance, a technique called permutation importance. The central neural network and federated versions shared all ten of their highest-ranked features. Their ranking correlations reached 0.985 for module-based training and 0.979 with added noise, indicating very similar explanations.

The spread of activity over time remained the strongest signal, with movement between resources, content interaction and collaborative activity among the other leading features. The model family appeared to influence explanations more than decentralization did.

These findings support the possibility of studying exploratory activity with compact AI models that retain useful, interpretable patterns without centralizing raw records during training. The evidence comes from simulated divisions within one educational environment, leaving questions about performance across independent institutions and the educational meaning of the labels.

The authors identify testing across institutions, stronger privacy mechanisms and richer representations of activity over time as future directions. Establishing whether the measured patterns reflect meaningful learning would give educators a firmer basis for using them to support students.