Overview
From 2018 to 2025, the GPT series went from an obscure paper to a world-changing product in seven years. Understanding its evolution means understanding the core trajectory of LLM technology.
GPT-1 (2018): The Paradigm
The paper “Improving Language Understanding by Generative Pre-Training” introduced a new approach: pre-train on massive unlabeled text, then fine-tune on specific tasks. 117M parameters, trained on BooksCorpus (7,000 books).
GPT-2 (2019): Zero-Shot Emerges
Scaled to 1.5B parameters, the model unexpectedly gained zero-shot capabilities — translating, summarizing, and answering questions without any fine-tuning. OpenAI initially refused to release the full model, sparking a public debate about AI safety.
GPT-3 (2020): Few-Shot Learning
At 175B parameters with 45TB of training data, GPT-3’s key discovery was in-context learning — a few examples in the prompt, and the model learns new tasks without gradient updates. This catalyzed the rise of Prompt Engineering Methodology.
InstructGPT (2022): Learning to Follow Instructions
Introduced RLHF (Reinforcement Learning from Human Feedback): SFT with human demonstrations, training a reward model from preference comparisons, and PPO optimization. This made ChatGPT possible.
GPT-4 (2023): Multimodal + Reasoning Leap
Widely regarded as the first model approaching Turing-test-level conversation. Multimodal (images + text), 128K context window, outperforming 90% of human test-takers on bar exams and GRE.
GPT-4o (2024) added real-time multimodal interaction with 200-300ms voice latency. Every generation is built on Transformer Attention Mechanism Explained, with inference optimization constantly evolving — see LLM Inference Optimization.