AI Engineering Tutorial · LLM Basics · Lesson 7 of 25
AI Engineering: Origins & Evolution
How building with language models became its own engineering discipline: from the Transformer paper to APIs, retrieval, tool calling, agents and the Model Context Protocol.
How it started
Language models are old; building products on top of general-purpose ones is new. The Transformer architecture (2017) made it practical to train very large models on very large text collections. GPT-3 (2020) showed that one large model could do many tasks from instructions and a few examples in the prompt, and it was offered through an API, so developers could use it without training anything. The same year, the retrieval-augmented generation paper combined a model with a search index so answers could draw on documents the model had never seen.
When ChatGPT launched in November 2022, millions of people met conversational AI, and companies started asking their engineers to build with it. The work that followed (prompts, retrieval, tool calling, evaluation, guardrails, cost and latency) is what is now called AI engineering.
Timeline
- Agents, MCP and evals as standard practiceCurrent
- Most production AI features combine retrieval, tools, evaluation sets and guardrails; agentic coding tools are common in software teams
- MCP gains remote transport and authorizationHistoric
- The MCP specification adds Streamable HTTP and an OAuth-based authorization framework for remote servers
- Agentic coding toolsHistoric
- Coding agents that read code, edit files and run commands in a loop (for example Claude Code) move from previews into everyday use
- Model Context ProtocolHistoric
- An open protocol for connecting AI applications to tools and data, so integrations are written once as servers
- Tool calling in model APIsHistoric
- Model APIs add structured tool (function) calling, the building block of agents
- Claude and GPT-4 releasedHistoric
- Larger context windows and stronger reasoning make document-heavy and multi-step applications practical
- ChatGPT launchesHistoric
- Conversational AI reaches a mass audience; companies start building LLM features
- ReAct and chain-of-thought promptingHistoric
- Papers show that prompting models to reason step by step, and to interleave reasoning with actions, improves results
- Retrieval-augmented generation (RAG) paperHistoric
- Combining a model with a search index over documents
- GPT-3 and the model API eraHistoric
- Few-shot prompting: one model, many tasks, through an API
- The TransformerHistoric
- The "Attention Is All You Need" paper introduces the architecture behind modern language models
How the work changed
| Early LLM apps (2022 to 2023) | Production AI engineering today | |
|---|---|---|
| Inputs | One long prompt | System prompts, templates, retrieved context, tool definitions |
| Knowledge | What the model memorised | Retrieval over current documents, with citations |
| Actions | Text only | Tool calling, agents, MCP servers, human approval |
| Quality | Trying a few prompts by hand | Evaluation sets, graders, CI gates, online monitoring |
| Safety | Prompt instructions | Layered guardrails, least privilege, red-teaming |
| Cost | Rarely measured | Token budgets, caching, routing to smaller models |
Resources
- Attention Is All You Need (2017)
- Language Models are Few-Shot Learners: GPT-3 (2020)
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020)
- Chain-of-thought prompting (2022) and ReAct (2022)
- Introducing the Model Context Protocol (2024)
- Building effective agents
Last reviewed 21 October 2026. Release and support dates come from the official release pages linked above.