mrkeyoor.com_
Sun 01 Nov 15:13 UTC
Tech3 min read

Nolan Lawson tests what JavaScript frameworks cost an AI coding agent

Lawson compares agent-built Pokémon apps across JavaScript frameworks. The results separate API-equivalent cost from subscription spending and leave interactive workloads untested.

Nolan Lawson published a JavaScript-framework benchmark on October 28 that asks how much work a coding agent needs to build the same app. His results put Hono with htmx at a median API-equivalent cost of $2.67 and Angular at $4.97. The comparison gives developers a testable question about stack choice, with a narrow answer: one model building a small Pokémon app.

Lawson used Opus 5.5 at high effort, reporting medians from five builds per framework except Angular, where two failed and three remained. The dollar figures use API prices reported by Claude Code. They aren't his actual subscription bill: he says the runs were covered by his $100 monthly Max plan.

What the harness keeps separate

The benchmark repository describes fresh Podman containers for agent runs and separate containers for measuring the resulting apps. Each agent gets the same specification and tests. After generation, only the app directory moves into a clean container for the final tests, so changes an agent makes to the test suite cannot determine that result.

The repository also records a boundary around its browser measurements: it measures page loads, excluding later interactions such as search and client-side navigation. Its content-heavy workload favors server-rendered pages on first-load metrics. MrKeyoor inspected the published account and repository documentation; we haven't rerun the builds or verified the rendered apps.

A result to adapt before choosing a stack

For a developer considering a similar test, the repository's requirements include Linux, Podman, Node and Claude credentials. Frameworks and their AI tooling are deliberately unpinned, while installed versions are recorded. A future run therefore samples a changed ecosystem rather than repeating a frozen environment.

The published comparison is a reason to measure your own workload before changing frameworks. Watch for replications with other agents and more interactive apps: those would test whether the cost differences survive a different task, rather than extending this small app's ranking to every project.

Sources

  1. The AI cost of JavaScript frameworks | Nolan Lawson
  2. Agent framework benchmark | GitHub