mrkeyoor.com_
Thu 17 Sept 15:59 UTC
LLM Toolsevaluationupdated 17 Sept 2026

harness-sdk review

Strands Agents is an in-process SDK for building AI agents in Python or TypeScript. It supplies the loop around a model: tools, limits, sessions, memory, streaming, MCP, provider adapters, traces, and multi-agent patterns. Teams use it when a hand-built tool-calling loop has started to need production controls.

Verdict

Our harness-sdk run passed 4,733 of 4,733 Vitest tests in 88 seconds, but the install left 11 known npm vulnerabilities. Strands is worth adopting when you will use its limits, hooks, sessions, provider adapters, and observability instead of rebuilding them. For approval-sensitive or non-idempotent tools, add application-level safeguards and test the two open failure paths before production.

We ran it

Lab card: what happened when we ran harness-sdkScreenshot of harness-sdk (strandsagents.com)
Install✓ · 22s443 packages · 398 MB
Build✓ · 24s
Tests✓ · 88s4733 passed · 0 failed of 4733 (vitest)
Known vulns110 critical · 3 high · 8 moderate · 0 low (npm audit)
Repo2547 files~395,625 lines of source · 26.5 MB · 38 CI workflows

Answers from our run

Does harness-sdk build from source?

Dependencies installed in 22 seconds (443 packages), and the build succeeded in 24 seconds. We cloned commit e7fd22b into a clean Debian container with 3 CPUs and no project-specific setup.

Do harness-sdk's tests pass?

Yes: 4733 of 4733 passed when we ran the project's own test command (vitest). Some failures need services or credentials a bare container does not have.

Does harness-sdk have known vulnerabilities in its dependencies?

npm audit flagged 11 known advisories in the dependency tree at the time of our run.

Who should not use harness-sdk?

Small applications that only need one model call and a few tools: our checkout installed 443 packages and occupied 398 MB before any provider workload.

What are the alternatives to harness-sdk?

Pydantic AI, OpenAI Agents SDK, Mastra. Our harness-sdk run passed 4,733 of 4,733 Vitest tests in 88 seconds, but the install left 11 known npm vulnerabilities.

Setup3/5Clean build, but 443 packages plus cloud credentials are substantial
Docs5/5Separate SDK guides, API references, examples, and production notes
Community5/57,313 stars with same-day pushes and active issue handling
Maturity4/54,733 tests passed, with open tool-control bugs still under review

Who it’s for

Python or TypeScript teams that want the agent loop inside their own application process.
AWS Bedrock users who want the shortest documented path from credentials to a working agent.
Platform teams that need provider choice, MCP tools, sessions, tracing, cancellation, and token budgets in one SDK.
Organizations prepared to pin versions and test their exact provider, hook, and tool combinations.

Who it’s NOT for

Small applications that only need one model call and a few tools: our checkout installed 443 packages and occupied 398 MB before any provider workload.
Teams expecting a credential-free quick start: both SDKs default to Amazon Bedrock and require AWS credentials plus model access unless another provider is configured.
Approval-critical systems using raw handler interrupts with non-default error policies: open issue 4371 reproduces a path where on_error='proceed' runs the tool without approval.
Workflows with non-idempotent tools and casual retry logic: open issue 4338 shows a completed tool running again after an AfterToolsEvent exception.
Buyers who assume one passing npm suite covers both SDKs: our lab ran the root Node workspace, while the Python SDK has a separate Hatch test path.

Setup reality

Our sandbox installed 443 npm packages in 22 seconds and used 398 MB. The build passed in 24 seconds. Vitest then passed all 4,733 tests in 88 seconds, with 0 failures. Npm audit reported 11 known vulnerabilities: 3 high and 8 moderate.

The TypeScript SDK requires Node.js 22 or newer; the Python SDK requires Python 3.10 or newer and uses a separate Hatch workflow. Both default to Amazon Bedrock, so the README expects AWS credentials and Claude Sonnet model access. Anthropic, OpenAI, Gemini, Ollama, and other providers need their own configuration.

This is a 2,547-file monorepo with about 395,625 source lines and a 26.5 MB checkout. It has 38 CI workflows and npm workspaces, but no Dockerfile. Our npm measurements cover the root TypeScript workspace, not the independent Python test matrix.

4,733 passing tests support a large agent surface

Our TypeScript-side run passed 4,733 tests with 0 failures, which is the best argument for taking Strands seriously. The SDK owns far more than a model call. It provides execution limits, structured output, tool hooks, MCP clients, sessions, memory, streaming, tracing, guardrails, evaluations, and several multi-agent patterns. Both Python and TypeScript implementations live in the same repository, with a shared documentation site and governance material beside them.

That breadth solves a real engineering problem. A small agent loop usually gains cancellation, retries, context management, provider accounting, approval pauses, persistence, and traces one incident at a time. Strands already has named places for those concerns. It also runs in your process without requiring a hosted control plane. The trade is commitment: once application behavior depends on its event loop and lifecycle events, changing frameworks becomes an architecture project.

The 398 MB install covers only the TypeScript path

Our sandbox installed 443 npm packages in 22 seconds and used 398 MB on disk. The root build finished in 24 seconds. TypeScript users need Node.js 22 or newer, while the Python package supports Python 3.10 through 3.14 and uses Hatch for development. The monorepo's root npm scripts point at strands-ts, so our clean result should be read as evidence for that workspace, not as a substitute for the Python matrix.

The quick start also assumes Amazon Bedrock. You need working AWS credentials and access to the documented Claude Sonnet model before the default agent can answer. The SDK supports Anthropic, OpenAI, Gemini, Ollama, and other providers, but each changes credentials, optional dependencies, response shapes, and sometimes feature behavior. Provider choice is portable at the interface level. It still deserves an integration suite using the model and region you plan to run.

What happened when we ran it

Our run at commit e7fd22b installed 443 packages in 22 seconds, built in 24 seconds, and completed tests in 88 seconds. Vitest reported 4,733 passed and 0 failed out of 4,733. The fresh unprivileged container had 3 CPUs and 8 GB of RAM. Dependency installation occupied 398 MB, while the checked-out repository itself was 26.5 MB.

The repository contained 2,547 files and about 395,625 lines of source in our scan. We found 38 CI workflow files, npm workspaces, no Dockerfile, and no directory named tests. Npm audit reported 11 known vulnerabilities, split into 3 high and 8 moderate, with none rated critical or low. Those numbers describe the measured Node environment only; they do not report Python dependency advisories or model-service behavior.

Two open bugs affect tool approval and replay

Issue 4371 was opened on September 16, 2026 against Python v1.56.0 and current main. Its reproduction says a handler calling event.interrupt() can be treated as an ordinary handler error. With on_error='proceed', the sensitive tool ran without the requested approval. The default throw policy and the separate Confirm action were not affected in that report, and paired fix pull requests were already open the same day.

Issue 4338 covers a different failure around completed tools. In its reproduction, a tool finishes, an AfterToolsEvent callback raises, and the result is not appended to conversation state. Resuming the agent runs the completed tool again. Payments, messages, orders, and database writes can all be harmed by duplicate execution. Until the fix lands in the version you deploy, give those tools idempotency keys and make recovery check external state before retrying.

Python v1.56.0 and TypeScript v1.18.0 move separately

Python v1.56.0 and TypeScript v1.18.0 were both released on September 15, 2026. Separate tags are useful because the SDKs can ship on their own cadence, yet they also make parity a versioned question. The issue queue includes explicit port requests between languages, and the latest TypeScript release notes include changes copied from Python. Check the feature page for your chosen language rather than assuming a sibling implementation behaves identically.

The repository was pushed on September 17, the day we checked it. GitHub showed 7,313 stars, 499 open issues, and 260 open pull requests. That is an active project with an unusually busy public queue, not 759 confirmed bugs. The written lifecycle policy tells production users to pin minor versions for MCP, A2A, OpenTelemetry conventions, and experimental modules because those areas may change faster than the core compatibility promise.

443 packages make sense only if the controls get used

The 443-package install is hard to justify for a thin chatbot that calls one provider and two functions. Strands earns that cost when the application needs budgets, cancellation, sessions, MCP, traces, provider switching, structured results, and operator hooks together. The Apache-2.0 license and in-process design keep deployment choices open, while 38 CI workflows and 4,733 passing tests show serious maintenance work behind the API.

Adopt it with a pinned SDK version, a real provider test, and explicit rules for tool side effects. Keep approvals on the documented safe path, make irreversible tools idempotent, and review the 11 npm advisories in the dependency paths you ship. The measured suite says the TypeScript foundation is dependable. The two open tool-control reports say your application still owns the final safety boundary.

Alternatives

ProjectWhat it isPick it when
Pydantic AIA Python agent framework centered on typed inputs, outputs, and validation.pick this instead when Python typing and Pydantic models matter more than a matched TypeScript SDK.
OpenAI Agents SDK gh↗A smaller Python framework for agents, handoffs, guardrails, and tracing.pick this instead when a compact Python API and OpenAI-oriented workflow fit the application.
Mastra gh↗A TypeScript framework for agents, workflows, memory, and application integrations.pick this instead when the whole team is on TypeScript and wants an application framework around the agent.
LangChain gh↗A broad Python framework with a large integration ecosystem for LLM applications.pick this instead when connector breadth and existing LangChain components outweigh a smaller agent-focused API.

What people are saying

  1. [github-trending] strands-agents/harness-sdk

Sources

  1. Strands Agents repository and README
  2. TypeScript workspace package definition
  3. Python SDK package definition
  4. Strands feature lifecycle policy
  5. TypeScript v1.18.0 release
  6. Python v1.56.0 release
  7. Approval interrupt bug report
  8. Completed tool replay bug report

More llm tools reviews

12306-mcp · ouroboros · headcount · useagent · claude-skills · RAG_Techniques · the whole board →