Methodology
How the AI Resilience Score Is Built
A plain-English explanation of how we read a transcript, match it to course syllabi, and measure how much of a GPA was earned in formats where AI tools could not help.
Scoring pipeline of March 2026 · Written August 2026
What the report measures
A GPA tells you how well a student performed. It does not tell you how that performance was assessed, and since late 2022, that distinction matters. A grade earned on a proctored, closed-book exam could not have been produced by ChatGPT. A grade earned on take-home essays and online quizzes could have been.
The REALdegrees report separates the two. For every course on a student's transcript, we look at the actual syllabus from that course and term, work out what share of the final grade came from assessment formats an AI tool could realistically complete, and roll that up across the whole degree. The headline number, the AI Resilience Score, answers one question: of the academic performance behind this GPA, how much came from work AI could not have done?
Five steps from transcript to score
Read the transcript
The student's transcript is parsed into a course list: every course, the semester and year it was taken, the grade earned, and the credit hours. Nothing is scored yet. This simply establishes what the student took and when.
Find each course's syllabus
Each course is matched to the syllabus for that same course in that same term, drawn from our library of syllabi collected from Texas public universities. The term matters: a course's assessment mix in 2021 is often very different from the same course in 2025.
If no syllabus can be located for a course, that course is excluded from scoring. We never guess. The report discloses coverage on its face: how many courses were on the transcript, and how many we could actually analyze.
Score every graded assignment
An AI reader (Anthropic's Claude) reads the syllabus and extracts every graded component (exams, papers, quizzes, presentations, participation), the share of the final grade each one carries, and whether it happens in class or at home. Each component is then assigned a single number: the grade an AI tool alone could realistically have earned on it.
That number does not come from the AI reader's judgment. It comes from a fixed rubric the reader must apply exactly, with two dimensions:
Supervision. Work completed under supervision scores at or near zero no matter the year. A closed-book in-class exam, a supervised lab, a live presentation: AI cannot sit in the room.
Era. Unsupervised work is scored against what AI tools could actually do when the course was taught. Anything before ChatGPT's release (December 2022) scores zero: the tools didn't exist, so no student can be penalized for that period. From Spring 2023 forward the rubric steps upward as the tools improved, so the same take-home essay is scored more leniently in 2023 than in 2025.
The rubric's specific values are proprietary. What matters for reading a report is its shape: supervised work counts as AI-resistant regardless of era, unsupervised work counts as more exposed the later it was assigned, and assignments that mix both fall in between. Anything the syllabus leaves genuinely ambiguous is flagged for human review rather than guessed at.
Combine into a course score
Each assignment's rubric value is weighted by its share of the final grade, and the pieces are summed into a single AI vulnerability figure for the course: in effect, the final grade an AI tool alone could have earned in that course.
Example: a government course, Fall 2025. Two midterms and a final, all in-class and closed-book, carry 60% of the grade. Take-home essays carry 25%, and discussion posts the remaining 15%.
The supervised 60% counts as AI-resistant. The take-home 40% is scored against what AI could do in late 2025, which is most of that work. The course lands at roughly one-third AI-exposed: about two-thirds of this grade had to be earned in the room.
Roll up to the whole transcript
Course scores are combined across the degree, with each course counting in proportion to its credit hours. Letter grades are converted to a numeric scale that mirrors the standard 4.0 GPA scale; pass/fail, withdrawn, and incomplete courses are left out. Three headline figures come from this roll-up:
AI Resilience Score (0–100)
The centerpiece of the report. Every course's contribution to overall performance is split into the portion earned through AI-resistant formats and the portion earned through AI-exposed formats. The score is the AI-resistant portion as a percentage of the whole. A score of 80 means 80% of the grade-weighted performance behind the GPA came from work AI could not have done. Because it is grade-weighted, a student who earned their strongest grades in proctored, supervised courses scores higher than one whose best grades came from take-home work.
AI Resistance (0–100)
The same idea applied to course selection rather than performance. It reflects how AI-resistant the student's chosen coursework was across the credit hours taken, independent of the grades earned in it.
Department Comparison
The student's average vulnerability within their major's department, set against the department-wide average computed from every syllabus we hold for that department. This shows whether the student's course choices were more or less AI-resistant than what a typical student in the same major would encounter.
Safeguards built into the scoring
- The pre-ChatGPT rule. Every course completed before December 2022 is scored at 0% vulnerability, without exception. Students are never penalized for an era in which the tools did not exist.
- Fixed rubric, not model opinion. The AI reader extracts what the syllabus says; the vulnerability values are set by the rubric. The same syllabus produces the same score.
- No syllabus, no score. Courses we cannot match to a syllabus are excluded and counted, not estimated. The report states its coverage, e.g. “38 of 42 courses analyzed.”
- Ambiguity is flagged, not guessed. When a syllabus doesn't make an assignment's format clear, the item is marked for human review.
- Incomplete coverage lowers the portfolio score. The course-portfolio resistance figure is scaled down in proportion to the share of courses we could not analyze, so missing data can only make that number more conservative, never better.
What the score is not
The AI Resilience Score does not claim that any student did or did not use AI. It measures the assessment formats a student's grades were earned under: the opportunity for AI involvement, not conduct. A high score means the GPA was earned mostly in settings where AI could not have contributed. It is a statement about the structure of the coursework, not an accusation or an endorsement of anyone's behavior.
Who makes the judgments
Two different kinds of work go into a score: judgment about what AI tools can do, and the repetitive work of applying that judgment to thousands of syllabus lines. We split them deliberately.
- The faculty team sets the standard. The rubric (which formats AI could complete, how well, and in which era) was written and revised by the professors behind REALdegrees; the pipeline runs on its fourth generation. No value in the scoring reflects a model's opinion.
- Software applies the standard, identically, every time. The AI reader extracts what each syllabus says at deterministic settings, and the rubric supplies the values. The same syllabus always produces the same score. No student's result depends on which reviewer got their file, or on a reviewer's mood that day. Hand-scoring cannot make that promise; a fixed rubric applied by software can.
- People handle the edge cases. When a syllabus leaves an assignment's format ambiguous (an “exam” that doesn't say whether it is proctored, a “project” that could be supervised or take-home), the item is flagged and resolved by human review, or excluded. It is never guessed. Reports are checked and released by the team, not auto-emailed by the pipeline.
- We check the pipeline against ourselves. The team hand-scores samples of syllabi and transcripts and compares them with the pipeline's output. When the two disagree, the fix goes into the rubric so every future course benefits; that is what the four rubric generations are. Individual scores are never hand-edited to taste.
Reading the score
Scores map to four bands, used consistently across the report:
The report's “Bottom Line” paragraph translates the number into hiring-manager language across five tiers, from exceptional AI resilience (85 and above) down to a candid statement, below 40, that the assessment mix limits how much of the GPA can be attributed to independent work. Mid-range tiers also state the share of grade-weighted coursework that was AI-exposed, so the reader sees both sides of the ledger.
Questions about your own report
If you want to know how a specific course on your transcript was scored, or you think a course was matched to the wrong syllabus, write to admin@realdegrees.ai and we will walk you through it.
Scoring methodology write-up, August 2026. Describes the client-report pipeline current as of March 2026 (rubric generation 4). Syllabus extraction is performed with Anthropic's Claude at deterministic settings against the fixed rubric described above.