DSPy

Creating a useful application with a large language model is not simply a matter of writing an instruction and connecting an API. Developers often need to experiment with different prompts, provide suitable examples, control response formats, evaluate the generated results, and make repeated changes to improve consistency.

The work can become particularly difficult when an application contains several language-model operations. A customer support assistant, for example, might classify a message, identify its subject, retrieve relevant information, and generate a final response. Each step may require different instructions and output requirements.

DSPy is an open-source Python framework designed to make this type of language-model development more structured and measurable. It allows programmers to define tasks through signatures, organize operations into reusable modules, and use optimization algorithms to improve program behavior against evaluation criteria.

Instead of managing every instruction as a manually written prompt, developers can describe what an application should accomplish and let DSPy handle important parts of the program construction and optimization process.

This approach is particularly relevant to AI researchers, Python developers, machine learning engineers, and companies creating custom LLM applications. The framework supports classification, information extraction, retrieval-augmented generation, tool-using agents, and other language-model workflows.

Its main distinction is that it treats an LLM application as a program that can be tested and improved rather than a collection of unrelated prompts.

Why DSPy Takes a Different Approach to Language-Model Development

Traditional prompt engineering often depends on manually creating instructions and repeatedly modifying them until the results become acceptable.

A developer might begin with a basic prompt, add examples, change the wording, test several model responses, and repeat the process whenever the application produces an unexpected result.

This method is understandable and can work well for smaller projects. However, maintaining several prompts across a larger application can become complicated.

Consider a customer service application that receives thousands of messages. It may need to identify the customer’s issue, determine the appropriate department, retrieve information from a knowledge base, and generate a response.

The application could require separate prompts for classification, retrieval, and final response generation.

When developers manually control all these instructions, one change may affect another part of the workflow. This can introduce additional testing and maintenance work.

DSPy approaches the problem through structured programming.

Its programming model allows developers to describe tasks using signatures and organize those tasks through modules. Rather than embedding every requirement into a manually maintained prompt, programmers specify what an operation should receive and what it should return.

DSPy then uses these definitions when preparing language-model requests.

An important idea is the separation between task requirements and prompt implementation.

The application describes the expected behavior, while the framework provides mechanisms for constructing and optimizing the instructions used to produce that behavior.

For instance, a developer creating an email classification application might define an operation that accepts an email message and returns its category.

The developer does not need to manually maintain every wording variation of the underlying prompt. They can focus on defining the classification task, preparing useful examples, and evaluating the result.

DSPy stands for Declarative Self-improving Python. The project is associated with the Stanford NLP research community and is available as open-source software.

It does not remove the need for prompts, language-model instructions, or software development. Instead, it organizes these elements into a framework that can be evaluated and optimized systematically.

This is particularly useful when application behavior needs to improve over time and changes must be measured rather than judged only through occasional manual testing.

Understanding Signatures, Modules, and Reusable AI Programs

Signatures and modules form an important part of DSPy’s programming model.

A signature describes the structure of an operation. It tells the framework which information should be provided as input and what type of result should be returned.

Think of a signature as a definition of a task’s interface.

For example, a product review analysis program might accept a review as input and return a sentiment label and a short explanation.

Another signature could receive a technical document and return its main topic, important entities, and key dates.

This structured approach makes the application’s requirements easier to understand. Developers can examine what each operation is expected to accomplish without searching through a large collection of manually written prompts.

DSPy supports simple signature declarations as well as more detailed class-based definitions using typed fields.

These options are useful when an application has complex input and output requirements.

Modules determine how individual operations are performed.

DSPy provides several modules for different types of language-model behavior.

Predict

Predict offers a straightforward interface for making a language-model prediction based on a defined signature.

It is useful when an application needs a relatively direct mapping between its input information and the expected output.

For example, a developer might use Predict for basic text classification, information extraction, or response generation.

ChainOfThought

ChainOfThought provides a reasoning-oriented approach to performing tasks.

It introduces an additional reasoning process before the model produces the requested result. This can be useful for tasks that involve several logical steps.

ProgramOfThought

ProgramOfThought takes a different route by allowing code generation and execution to assist with particular problems.

It can be useful when a task involves calculations or operations that are more appropriately handled through executable code.

ReAct

ReAct supports agent-style execution. It allows a language model to interact with tools as part of completing an assigned task.

A tool might be a Python function, an external service, or another operation made available by the developer.

MultiChainComparison

MultiChainComparison can compare multiple generated responses and help select a suitable final answer.

This can be useful when the application benefits from examining several possible outputs rather than relying on one prediction.

RLM

RLM provides a recursive language-model approach for working with large contexts through a Python REPL environment and additional model interactions.

It can be useful for specialized applications where information needs to be explored through smaller operations instead of placing the entire context into one request.

These modules can be combined into larger applications.

For example, a business document assistant might use one module to identify the user’s question, another to retrieve the appropriate information, and a final module to generate a structured response.

The developer can then evaluate the overall program and improve the components that are producing weak results.

This makes DSPy useful for teams that want a more organized development structure instead of maintaining every language-model operation independently.

DSPy Optimizers: Turning Evaluation Results Into Better Instructions

The optimizer system is one of DSPy’s most distinctive capabilities.

An optimizer is a component that attempts to improve an existing DSPy program using examples, evaluation criteria, and different optimization strategies.

The process begins with a program that performs a defined task.

Developers then prepare training examples and select a metric that describes the quality they want to improve.

The framework uses these elements to explore different program configurations.

For example, a classification application might use accuracy as its main metric. The goal would be to increase the number of correctly classified examples.

A retrieval-augmented generation application may require a more detailed metric involving answer correctness, relevance, and the use of retrieved evidence.

The optimizer evaluates possible improvements according to the selected objective.

BootstrapFewShot

BootstrapFewShot generates few-shot demonstrations using training examples and model-produced outputs.

These demonstrations can provide useful examples for language-model operations without requiring developers to manually write every demonstration.

The process relies on suitable examples and evaluation conditions.

BootstrapFewShotWithRandomSearch

This optimizer explores different combinations of demonstrations and evaluates the resulting program configurations.

It is useful when developers want to compare multiple candidate combinations rather than relying on one manually selected arrangement.

MIPROv2

MIPROv2 is designed to optimize instructions and demonstrations through a more systematic search process.

It can help developers explore different instruction and example combinations according to program performance.

GEPA

GEPA is an optimizer that uses reflection and textual feedback to propose improvements.

Instead of depending exclusively on numerical results, it can examine traces and feedback from program execution to identify why a particular operation produced an unsuccessful result.

For example, an information extraction application might repeatedly confuse a document publication date with a transaction date.

Feedback describing these mistakes can help GEPA propose instructions that better distinguish the two fields.

BootstrapFinetune

BootstrapFinetune supports a fine-tuning-oriented workflow using demonstrations generated from an existing DSPy program, subject to the supported model provider and training configuration.

This can provide another development route when a team wants to move beyond prompt-level optimization.

Different optimizers have different computational requirements and may be more suitable for different datasets and applications.

A small development task may not require an extensive optimization process. A larger application with measurable quality requirements may benefit from more sophisticated methods.

The important consideration is that the optimizer is only as useful as the objective and evaluation data provided to it.

If the metric rewards the wrong behavior, optimization can improve the score while making the actual application less useful.

Developers should therefore maintain separate training and validation examples, inspect the resulting outputs, and verify that improvements carry over to realistic user requests.

Building RAG Applications and Information Retrieval Systems With DSPy

Retrieval-augmented generation is another important application area for DSPy.

RAG combines external information retrieval with language-model generation. Instead of asking a model to answer entirely from its existing knowledge, an application retrieves relevant information and supplies it as context.

DSPy helps developers organize the language-model side of this workflow through signatures, modules, and evaluation.

A typical RAG application contains a retrieval component and a response-generation component.

The retriever searches an external information source, while the language model prepares an answer using the retrieved material.

For example, an organization may have thousands of technical documents covering product specifications, service procedures, and internal policies.

A developer can create an assistant that retrieves the relevant passages before generating a response.

DSPy makes it possible to define how the retrieval and answer-generation operations should work and evaluate the quality of the completed pipeline.

The approach can also be extended to multi-hop retrieval.

In a multi-hop RAG application, a question may require several related searches. The system can retrieve information from one source, use the result to identify another relevant query, and continue the process before generating the final answer.

This can be valuable for research applications and complex knowledge systems.

DSPy also supports tool-using agents through its ReAct module.

An agent can be given callable functions that perform selected operations. These might include a search function, a database query, a calculation, or an external API request.

The model decides whether a tool is necessary and uses the available functionality according to its instructions.

DSPy’s tool-related utilities also include support for working with Model Context Protocol tools.

This makes it possible to build applications that connect model-driven reasoning with external services.

However, a useful agent requires more than tool availability. Developers need to define permissions, validate inputs, handle exceptions, and examine how the agent responds when a tool fails.

The framework provides the development components, while the application team remains responsible for the reliability and security of the complete system.

Connecting Models and Evaluating Application Performance

DSPy uses a common interface for configuring supported language models.

The dspy.LM interface allows developers to specify a model provider and model identifier together with the necessary authentication settings.

This helps organize model connections inside the application.

Different projects may use different language models according to their requirements. Some applications prioritize response quality, while others need lower latency, structured outputs, or more economical API usage.

The ability to work with supported providers through a shared programming interface makes model experimentation easier.

However, a common interface does not mean every model has identical capabilities. Developers must check provider-specific support for tool calling, structured responses, context limits, streaming, and other features.

DSPy’s evaluation functionality is particularly important in this context.

Before optimizing an application, developers need to understand how well it performs in its original configuration.

A baseline evaluation provides a point of comparison.

For a classification application, the metric might measure how many examples receive the correct label.

For an information extraction system, it might compare extracted fields with the expected values.

For a RAG program, the evaluation process may examine whether relevant information was retrieved and whether the final response accurately reflects that information.

Developers can create custom metrics to reflect the requirements of their applications.

This provides more flexibility than relying on one universal score for every type of LLM task.

The framework also documents functionality for caching, asynchronous execution, streaming, saving and loading programs, and debugging.

These features can help when applications contain several connected model calls and require more careful development management.

The package information supplied in the research draft identifies DSPy 3.4.1, released October 12, 2026, with supported Python versions from 3.10 to below 3.16.

Developers should review the current package release information and test provider compatibility before upgrading a project.

For production applications, repeatable evaluation is especially important. The goal should not be to produce one impressive result but to maintain suitable performance across a representative collection of requests.

Choosing DSPy Alongside Other AI Development Frameworks

DSPy occupies a different position from several popular LLM development frameworks.

LangChain provides a broad set of components for connecting models, tools, and application services.

LangGraph concentrates on controlling agent execution and stateful workflows.

Pydantic AI provides a Python-focused development approach built around typed structures and validated outputs.

Haystack is particularly relevant to search, information retrieval, and modular RAG applications.

OpenAI Agents SDK provides development components for building agent-based applications around the OpenAI ecosystem.

DSPy focuses more directly on the programming and optimization of language-model behavior.

This difference matters when selecting the right development technology.

A developer who wants to create an application with external tools and predictable agent execution may prefer an agent-oriented framework.

A team that already has an LLM pipeline and wants to improve its instructions through evaluation may find DSPy’s optimizer system more relevant.

DSPy is especially useful when model behavior needs to be improved systematically and the development team can define meaningful evaluation criteria.

It is not intended to replace every other AI framework, nor is it automatically the best choice for every chatbot or automation project.

Some applications may benefit from combining technologies. For example, a team could use an orchestration framework for application control while applying DSPy to selected language-model modules that require optimization.

The decision should be based on the application’s architecture rather than the number of available framework features.

What Developers Should Consider Before Adopting DSPy

DSPy provides a structured development environment, but it does not eliminate the work required to build a dependable AI application.

The first consideration is the quality of the examples used during development and optimization.

Training data should represent the tasks the application is expected to perform. When examples are too narrow, an optimized program may perform well on familiar inputs but struggle with less common user requests.

The evaluation metric deserves equal attention.

A metric should measure the behavior that matters to the application. A classification system, for example, may need more than overall accuracy when certain types of mistakes are more serious than others.

Optimization expenses are another practical issue.

The framework does not require a mandatory subscription, but optimizers can make additional model requests while evaluating candidate instructions and demonstrations.

Large experiments may therefore consume significant API credits.

Developers should estimate these costs before running extensive optimization jobs.

Model compatibility also needs attention. Different providers expose different capabilities, and moving a program between models may require configuration changes or renewed evaluation.

Security remains the application’s responsibility. If an agent can call external functions or retrieve private information, developers should implement appropriate permissions and input validation.

Finally, the technical learning curve should not be underestimated.

DSPy introduces concepts such as signatures, modules, metrics, optimizers, training datasets, and program evaluation. These ideas can take time to understand, especially for developers who have only worked with basic chatbot APIs.

However, for people creating complex language-model applications, the structured approach offers a practical way to manage improvements.

Its strongest use case is not simply generating text. It is developing language-model software whose behavior can be evaluated, adjusted, and improved through a repeatable programming process.