Why every course at the institute ends with an evaluation, not a demo
A note from the Dean on the pedagogy behind the academy — and why we grade validation plans more heavily than model accuracy.
Dr. Ayesha Rehman
Professor of Machine Learning · Dean of the Academy
Visitors are sometimes surprised that our grading rubrics weight validation at thirty-five percent and model quality at twenty. The reason is simple. A model that performs well on an honest evaluation is valuable. A model that performs well on a dishonest one is dangerous, and it is far more common.
So we teach validation first, and we keep coming back to it. Learners in Foundations of Machine Learning implement cross-validation before they implement a second model. In LLM Engineering & Evaluation they build an evaluation set before they build the assistant. Capstones are defended in front of an examiner whose first question is always the same: how do you know?
This is slower than teaching tools. It is also the only approach we have found that produces engineers we would trust with a system that affects people.
- pedagogy
- academy
Get the next note by email
One considered letter a month on research, admissions and what we are building.
More in Perspectives
What a model card should tell a bank
We reviewed the documentation for twelve models in use at financial institutions in the region. Most would not pass a basic procurement check.