Context Engineering with DSPy

DSPy stands for Declarative Self-improving Python. It is an open source framework, released under the MIT license in October 2023 and developed from work by Omar Khattab and colleagues at Stanford. Instead of writing a prompt, you define an AI task by its inputs and outputs in a Signature, combine tasks into modules, and use optimizers to improve the program against a dataset and an evaluation metric.

Context engineering is the discipline of systematically curating what information a language model receives, to maximize performance while minimizing costs and errors. The context is everything the model can see when it responds: system instructions, messages, conversation history, tool definitions and results, retrieved documents, memories and examples.

Prompt engineering is concerned with changes inside the prompt: instructions, examples and fixed chains of prompts. Context engineering also covers the wider system around the prompt: retrieving documents and memories, formatting content, designing the harness, and pruning irrelevant information on long-running tasks. The book treats prompt engineering as a subset of context engineering.

Context rot is the way model performance degrades as the input gets longer, a term from the vector database company Chroma. Chapter 1 describes 11 ways long contexts fail, including context distraction, lost in the middle, context confusion, context clash, context drift and context poisoning, and 10 techniques for fixing them.

GEPA (Genetic-Pareto) is a DSPy optimizer that rewrites your prompt instructions by reflecting on where the program failed. It is the author’s default optimizer once a dataset has 10 or more examples. In the book’s benchmark of optimization approaches on one AI-detection task, GEPA took accuracy from 53.75% to 80% for about $0.58.

Harness engineering is OpenAI’s term for the runtime scaffolding around a model: the environment, tools, feedback loops and control systems that deliver context to an agent and act on its outputs. The book’s view is that this is still context engineering. The harness is what delivers the context, and getting the context right is what delivers performance.

Prompt optimization means improving a prompt against a dataset and an evaluation metric instead of by trial and error. In DSPy, a prompt optimizer selects or generates few-shot examples, proposes and tests better instructions, or both. The book’s Chapter 1 example cuts a 1,784-token prompt to 43 tokens while accuracy moves from 64% to 67%.

LLM-as-a-judge uses one language model to evaluate another model’s output, which makes it possible to score subjective tasks like writing. The book’s method is to label 20 to 50 outputs yourself, train the judge on those labels until it agrees with you 80 to 90 percent of the time, and then use the judge as the metric an optimizer works against.

MIPROv2 is a DSPy optimizer that searches over both instructions and few-shot examples. The book recommends trying it when you have a large dataset of 200 or more examples and a substantial optimization budget. In the Chapter 6 benchmark it reached 80% on validation but 66.25% on the locked test, a reminder to hold out data.

Chapter 8 builds the same agent in DSPy, LangChain, Pydantic AI, the OpenAI Agents SDK and CrewAI. The book’s argument for DSPy is that every agent is a DSPy program, so it can be evaluated and optimized. You don’t have to choose one: many teams optimize a prompt with DSPy, export it, and run it in the framework they already use, which Chapter 10 covers.

No. Chapter 1 covers context engineering without any DSPy, Chapter 2 introduces the framework’s five core abstractions, and Chapter 3 builds a first program end to end. The examples are in Python, and the concepts transfer to DSPy ports such as Ax for TypeScript, dspy.rb for Ruby and dspy-go.

Fewer than most people expect. The book’s guidance is that 10 to 50 well-chosen examples combined with an optimizer will often outperform a handcrafted prompt, and that you can start with 10 to 20 and grow the dataset from the errors you see in production.

Yes. The book cites Shopify using DSPy with GEPA to extract structured metadata across every shop, Dropbox optimizing the relevance judge behind Dash, and Databricks building DSPy-powered judges and RAG pipelines, along with JetBlue, Replit and Moody’s.

A recursive language model puts a large input into a code environment and calls the model on smaller parts of it, rather than loading everything into one context window. DSPy provides the pattern as dspy.RLM. The book uses it for documents too large to fit in context, such as financial filings.

Yes, and Chapter 11 is about exactly that. A skill is a Markdown instruction file (SKILL.md) that a coding agent such as Claude Code, Codex or Cursor loads for a task, so it can be tested against failed sessions and rewritten by GEPA. The same approach works for the always-loaded AGENTS.md or CLAUDE.md file and for agent persona files.

The companion repository pins DSPy 3.3.0 and supports Python 3.12 through 3.14. Each chapter has notebooks you can run locally, and many run in Google Colab.