LLM Engineering & Evaluation
Build language systems you can measure
Associate Professor · Language & Reasoning
About this course
The course for engineers who have shipped a prototype on a language model and discovered that the hard part had not started. We cover context engineering, retrieval, structured outputs, tool use and agents — and, running through all of it, evaluation: how to know whether a change made things better.
You will build one assistant three ways and measure each on an evaluation set you construct. Every lab runs in the browser; the Claude-powered tutor can see your code and the brief.
What you will be able to do
- Design prompts, context windows and caching strategies with measured cost and quality.
- Build retrieval-augmented generation with hybrid search and defensible chunking.
- Construct evaluation sets and judge pipelines; detect regressions before users do.
- Implement tool-using agents with permission boundaries, observability and budgets.
Curriculum
11 lessons · 3h 35m
- 01
Language models as components
Treat the model as an unreliable but powerful function, and engineer around it.
- The model is a function with a distribution over outputsPreview16 min
- Context engineering18 min
- Lab · Token budget planner30 min
- 02
Retrieval-augmented generation
Give the model the right facts at the right time, and prove that it helped.
- Chunking, embeddings and hybrid search22 min
- Constructing an evaluation set20 min
- Module quiz · RAG and evaluation8 min
- 03
Tools, agents and safety
Let the model act — inside boundaries you can defend.
- Tool use and the agent loop21 min
- Guardrails, permissions and red-teaming17 min
- Lab · Safe tool dispatcher35 min
- 04
Operating LLM features
- Cost, latency and caching16 min
- Final assessment12 min