Why were Transformers such a big deal?

They removed the bottleneck of processing sequences step by step, which let models scale to far more data and parameters — the precondition for today's large language models.

Do I need to understand attention to use these models?

Not to use them. It helps to know that attention is what gives them their grasp of context, but day-to-day work is mostly prompting and fine-tuning, not rebuilding the architecture.

What is a Transformer (in AI)? — Getting Digital glossary

Transformers

Also: transformer architecture, attention model

The Transformer is a neural-network architecture that processes a whole sequence at once and uses an 'attention' mechanism to weigh how much each part should influence every other part.

Attention instead of step-by-step

Older sequence models read text one token at a time, which made them slow and forgetful over long passages. The Transformer reads the whole sequence together and lets every token *attend* to every other — deciding, for each word, which other words matter most to its meaning. That single idea made models both faster to train on modern hardware and far better at long-range context.

It is the architecture underneath nearly all large language models and most current generative AI, and it has spread well beyond text into computer vision. When people say the field changed around 2017, the Transformer is what they mean.

The "T" you keep seeing

The T in GPT stands for Transformer. The architecture is the common ancestor of most models you interact with today.

Key takeaways

→Attention lets a Transformer weigh relationships across an entire sequence at once.
→It is the shared foundation of modern LLMs and much of generative AI, and now reaches into vision too.

Related concepts

Broader topicDeep Learning
The Transformer is a deep-learning architecture.
Leads on toLarge Language Models
LLMs are built on the Transformer architecture.
RelatedComputer Vision
Transformers have spread from language into vision.

FAQ

Why were Transformers such a big deal?: They removed the bottleneck of processing sequences step by step, which let models scale to far more data and parameters — the precondition for today's large language models.
Do I need to understand attention to use these models?: Not to use them. It helps to know that attention is what gives them their grasp of context, but day-to-day work is mostly prompting and fine-tuning, not rebuilding the architecture.

Transformers

Attention instead of step-by-step

Key takeaways

Related concepts

Courses that teach this

FAQ