DSPy is useful once you can score an answer
A prompt change that fixes an extraction example can break another. DSPy gives that familiar problem a repeatable workflow: describe a task, compose a Python program, and let an optimizer compare instructions or demonstrations against a metric. The framework's README names classifiers, retrieval pipelines, and agent loops as applications. The common requirement is a way to tell whether a change helped.
That makes DSPy a good candidate for an engineer maintaining a document extractor with reviewable answers. It is a harder sell for an open-ended assistant whose success means something different in every conversation. You can still write that assistant with DSPy, but the optimization machinery needs a target. Buying into the framework before defining one reverses the useful order of work.
What happened when we ran it
Our sandbox installed DSPy in 55 seconds, built it in 5 seconds, and completed pytest successfully in 805 seconds at commit 099982b on 2026-10-13. Installation recorded 173 dependencies and occupied 680 MB. Pytest reported 2,410 passed, 0 failed, and 584 skipped. The supplied summary also labels 2,410 as its total, so we report the fields separately rather than manufacture a reconciled denominator.
Pip-audit reported 2 known vulnerabilities. The supplied evidence does not identify their packages, severity, or exposure in a particular application. A passing suite does not answer those questions. Dependency review remains part of adopting this checkout, even though the repository commands all succeeded.
The sandbox configuration recorded 3 CPUs, 8 GB RAM, and the image ghcr.io/astral-sh/uv:python3.12-bookworm. That label alone does not establish the interpreter or operating-system version. These results cover repository setup and checks, not inference speed, answer accuracy, optimization cost, or a comparison with a manually written prompt. None of those outcomes was measured here.
The optimization guide asks you to separate development from evaluation
DSPy's optimization guide begins with an existing system and a way to evaluate it. It then calls for training and held-out data, and warns that prompt optimizers can overfit small training sets. That warning deserves more attention than the promise of automatic improvement. Searching many candidates gives you more opportunities to exploit a weak scorer.
For an extractor, a sensible trial would keep reviewed documents aside and compare the original program with the optimized version on those untouched examples. This is an adoption recommendation, not a result from our sandbox. Check the errors as well as the aggregate score: a metric that accepts the wrong invoice amount is a poor guide even if it rewards the correct output format.
The extra work is defensible when you expect to revise a task repeatedly. It is harder to justify when a short prompt already meets a clear requirement and rarely changes. DSPy moves engineering effort into the program, examples, and evaluation rule. Those assets need owners just as application code does.
A configured model still needs an endpoint and a request budget
The model guide documents hosted credentials and local endpoints. Running locally therefore means supplying a model service as well as the Python framework. A successful package install says nothing about whether your chosen model is reachable or follows the output contract well enough for your task.
Optimization evaluates candidates through model requests. Set a budget and record the provider and model configuration before comparing results. DSPy's documentation describes request history and usage metadata, which can help inspection, but our run did not validate provider billing or enforce a monetary ceiling. Treat those as separate operational checks.
GEPA can finish without improving the seed program
The 3.4.1 release notes say a reflection model must return parseable JSON. When a proposal fails, GEPA can keep the current candidate and finish with the seed unchanged. That is a specific reason to inspect optimization logs and compare the resulting program with the starting one. Completion alone does not establish improvement.
The same release notes describe compatibility changes in reflection requests. Keep the measured commit distinct from whichever release you adopt. Our green run does not certify every later configuration, and a release note describes intended behavior rather than an independent result for your workload.
Choose the framework around the work you need to repeat
LangChain centers agents, model interfaces, and integrations. Prefer it when connecting components is the main problem. Haystack describes explicit control over retrieval, routing, memory, and generation through modular pipelines. Prefer that structure when tracing document flow matters more than searching instructions. These are architectural choices, not a performance ranking.
Our setup score of 4/5 means the measured repository commands passed while live-model setup remains unverified. The documentation score of 4/5 reflects concrete connection and optimization guidance with application-specific checks still required. Community support and production maturity remain unscored because this investigation did not establish either.
Before adopting DSPy, write down an error you need to reduce and collect examples that expose it. If you cannot agree on how to score those examples, an optimizer will make that disagreement more expensive.