NLP with Transformers
How language models actually work
Associate Professor · Language & Reasoning
About this course
Tokenisation, attention, positional encodings, pre-training objectives and fine-tuning — taught so that when you read a model card you understand every line. Includes a focus on Urdu and other low-resource languages, where the tricks that work for English often fail.
What you will be able to do
- Explain self-attention and the transformer block from memory.
- Choose and adapt tokenisers for multilingual text.
- Fine-tune encoder and decoder models for classification and generation.
Curriculum
7 lessons · 2h 15m
- 01
Representing text
- Tokenisation is a modelling decisionPreview16 min
- Embeddings and similarity18 min
- 02
Attention and the transformer
- Self-attention from scratch24 min
- Lab · Attention weights35 min
- Module quiz · Transformers8 min
- 03
Fine-tuning
- Encoders for classification15 min
- Decoders and generation19 min