Instructor’s guide¶

Why we teach this lesson¶

Intended learning outcomes¶

Timing¶

Preparing exercises¶

e.g. what to do the day before to set up common repositories.

Other practical aspects¶

Before teaching the lesson, instructors should decide how far to go into the attention calculation. The important distinction is between:

attention, which exchanges information between token positions;
feed-forward layers, which transform each token representation;
the training objective, which adjusts both components through examples.

The lesson should avoid presenting attention heads as hand-designed linguistic rules. Examples involving nouns, adjectives, or pronouns are useful illustrations, but the actual features learned by a model are distributed and often difficult to interpret directly.