Blogs · Essay
A Trojan Horse in The Alma Mater
a degree has the useful property of being understood by people who were absent for almost everything that produced it. an employer sees the institution’s name attached to a result in a particular subject, with little sense of what the student went through to reach it. the university lends its name to the claim that something happened to this person during their education, and that certain capabilities can reasonably be expected of them afterward. granted, the assignment supplies some of the evidence for that claim. an assessor can inspect a submitted essay in considerable detail, although establishing how much its author understands requires rather more inference. ai complicates that inference because its contribution to a submission can be difficult to distinguish from the student’s own understanding. while the credential remains legible throughout, what becomes uncertain is how much assurance it can offer.
quaint equestrianism
imagine a student who gets stuck halfway through a proof because they cannot see why a particular step follows. reread the textbook as they might, it has not helped, so they ask an ai assistant to explain the step and show how the same reasoning works in a simpler example. after a few follow-up questions, they work through the proof in their own words before checking their understanding on a similar problem.
in a slightly different exchange, the assistant might assemble the proof around the missing step, leaving the student with an explanation they recognize without being able to reconstruct. recognition can feel reassuringly close to understanding while the solution remains in view. this discrepancy becomes apparent when a later problem changes an assumption and the student has to decide whether the same method still applies. so in this case, both encounters might produce an excellent submission, but the first has helped the student develop a capability; the second leaves more uncertainty about what they could do with a related problem. an assessor looking only at the finished page has to distinguish between outcomes whose differences may no longer be visible there.
thus, the horse enters through an ordinary request for help, carrying a complication for the institution that admitted it. the assignment still arrives through the same portal and receives a grade in the same format, while its relationship to the student’s understanding may have changed considerably. no elaborate conspiracy is required, and the student could have followed every rule about permitted assistance. the university can be clear about permitted use and still lack evidence of what was learned.
the university can be clear about permitted use and still lack evidence of what was learned.
office nanoseconds
a misunderstanding can occur several days before the next opportunity to discuss it with a teacher. help available during the original attempt can let the student work out why their approach failed before they carry the mistake into later work.
in a randomized crossover study published in Scientific Reports in 2025, Greg Kestin and colleagues examined 194 students in an undergraduate physics course at Harvard. across two lessons, students performed better on immediate post-lesson tests after using a purpose-built ai tutor than after the comparison active-learning classes. the study concerned those lessons and their short-term outcomes, leaving longer-term retention and broader educational effects to further investigation.
instructors had prepared the materials and tested the interactions, while the system guided students through the work at their own pace and provided feedback along the way. teaching effort had gone into deciding what the student would encounter and how the exchange should proceed. the result gives universities a reason to investigate that instructional design rather than assume that any available chatbot will provide the same educational experience.
credential–formation duality
suppose two students submit equally convincing analyses of the same dataset. both have used ai within the course rules and acknowledged their use. one can explain why the method fits and adjust the analysis when changed data exposes an assumption that no longer holds. the other can recount the submitted explanation but struggles to adapt it. discovering that both used ai would do little to resolve the difference, since their use has produced submissions of comparable quality while leaving them differently prepared for the next question.
the credential–formation duality concerns this relationship between a qualification that others recognize and the capabilities its holder has developed. the certificate makes those capabilities portable as a social claim: somebody who never watched the student work can nevertheless take the qualification into account. the institution must decide what evidence justifies that confidence, including when sophisticated assistance participates in producing the work it assesses. a 2025 preprint by Eason Chen and colleagues at Carnegie Mellon gives the distinction a useful empirical reference. the researchers evaluated a system combining feedback on mathematical proofs with a chatbot for questions. during the period when the experimental group had access and the comparison group did not, homework scores were significantly higher for the students with access, while the corresponding exam comparison showed no statistically significant advantage. the study did not establish an absence of learning; it demonstrated that the improvement observed in assisted homework could not simply be assumed to appear in exam performance.
the tutoring systems differed between the two studies, as did the subjects and assessment designs. universities need to investigate how particular interactions affect learning, then assess whether students can use what they have learned when the task changes. a better submitted answer is worth having, but an institution awarding a qualification also needs to know what its author can subsequently do. assessment was never a transparent window into understanding. Australia’s Tertiary Education Quality and Standards Agency describes generative ai as exacerbating existing assessment challenges while creating opportunities for teaching and learning. the familiar methods deserve examination on their own merits, rather than protection on the assumption that they perfectly revealed competence before the chatbot arrived. what has the university established, and is the assurance it gives proportionate to the evidence? beyond campus, an employer may rely on that assurance when deciding whether someone is ready for responsibility.
a certificate can carry the same crest through considerable changes in how it is earned, and the person relying on it should not have to discover the implications of those changes after assigning the work.
neo-assessmenets
the university could begin by asking what each assessment is meant to establish about the student. some capabilities may need to be demonstrated independently, while others include the competent use of tools. where the purpose is to evaluate an assisted analysis, students should explain why they accepted its recommendations, including how they checked for errors and whether the conclusions survive changed conditions.
the University of Sydney has adopted one approach to this distinction. ai is permitted in open, unsupervised assessments, with acknowledgment and guidance, while it is generally prohibited in secure, supervised assessments unless expressly permitted. these settings can supply different kinds of evidence about the student, allowing assisted work to remain part of the education without carrying the entire burden of assessment. for the two students with equally polished analyses, a follow-up might change an assumption and ask which conclusions survive contact with reality. their explanations could be considered alongside practical work, with later attempts showing how they respond to feedback, provided the assessment includes appropriate accommodations and does not mistake confidence or speed for understanding.
return to the student and the troublesome proof a month later, when the problem has changed and the earlier solution will no longer suffice. the assistant may still be open beside the work, but this time the student can explain which assumption supports their argument and why a suggested alternative would lead it astray.
they then attempt the revised problem without the earlier solution in view, giving the university evidence beyond the quality of the submitted page. the earlier help has left the student better equipped for the next problem. because tools of this kind are now a standing part of how work gets done, the practical question is how to design assessment around them so that the credential continues to mean what it says.