Transformer

Introduced in 2017, the transformer replaced sequential processing with attention, letting the model look at every token in the input at once and weigh their relevance to each other. That parallelism is what made training on internet-scale text practical. The ‘T’ in GPT stands for transformer.

In practice: In ’the trophy did not fit in the case because it was too big’, attention is how the model connects ‘it’ to ’trophy’.