How does a GPT-style transformer actually work?
Follow one sentence through every part of a tiny, real language model. Nothing here is faked: a 4-layer transformer with weights trained on a toy corpus runs live in your browser, and every number you see is computed by it. Edit the sentence at the top and watch the whole page recompute.
d_model = 82 heads4 layers
vocab …… parameters
GPT-3 has 175,000,000,000