mrkeyoor.com_
Tue 01 Sept 17:43 UTC
Automationevaluationupdated 30 Aug 2026

skyvern review

Skyvern is an open-source system for automating browser work with language models, computer vision, and Playwright-compatible controls. It aims to replace fragile scripts tied to fixed selectors with workflows that can interpret pages, take actions, extract structured data, and check results.

+12 / 2dstars / 7d
Verdict

Our build finished in 8 seconds, but the test run ended with 150 passed, 1 failed, 3 skipped, and 16 collection or setup errors. Skyvern is compelling when websites vary enough that maintaining fixed selectors has become the real cost, and its SDK plus workflow builder give it more range than a narrow agent library. Use it for supervised or recoverable operations first, and require clean tests and task-level validation before trusting consequential browser work.

We ran it

Install✓ · 23s59 packages · 51 MB
Build✓ · 8s
Tests✗ · 6s150 passed · 1 failed · 3 skipped · 16 errors of 167 (pytest)
Known vulns0(pip-audit)
Repo5426 files~1,226,366 lines of source · 352.9 MB · 13 CI workflows · Dockerfile · tests dir

Answers from our run

Does skyvern build from source?

Dependencies installed in 23 seconds (59 packages), and the build succeeded in 8 seconds. We cloned commit 96618fc into a clean Debian container with 3 CPUs and no project-specific setup.

Do skyvern's tests pass?

Not all of them: 150 of 167 passed and 1 failed when we ran the project's own test command (pytest), with 16 collection errors. Some failures need services or credentials a bare container does not have.

Does skyvern have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use skyvern?

Teams that need deterministic, zero-variance automation for every run

What are the alternatives to skyvern?

browser-use, Stagehand, Playwright. Our build finished in 8 seconds, but the test run ended with 150 passed, 1 failed, 3 skipped, and 16 collection or setup errors.

Setup3/5Fast install and build, but our test run was not clean
Docs4/5Clear quickstarts, deployment choices, SDK commands, and fixes
Community4/522,879 stars, a same-day push, and a release six days ago
Maturity3/5Version 1.0.51 is active, but our tests exposed rough edges

Discussed on

  1. hnShow HN: Skyvern – Browser automation using LLMs and computer vision422 points
  2. hnLaunch HN: Skyvern (YC S23) – open-source AI agent for browser automations327 points
  3. hnShow HN: MCP Server to let agents control the browser14 points
  4. hnRustwright: Playwright rewritten in Rust that uses 70% less memory12 points
  5. hnShow HN: Skyvern 2.0 – open-source AI Browser Agent scoring 85.8% on WebVoyager9 points

Who it’s for

Engineering teams automating multi-step work across changing websites
Operations teams that need a no-code workflow builder backed by an API
Python or TypeScript developers who already understand browser automation
Organizations prepared to self-host an AI service and review its browser actions

Who it’s NOT for

Teams that need deterministic, zero-variance automation for every run
Small scripts where plain Playwright selectors are cheaper and easier to debug
Buyers who need a permissive open-source license, because the repository uses AGPL-3.0
Anyone expecting the full test suite to pass in a minimal fresh container without investigation

Setup reality

In our sandbox, installation succeeded in 23 seconds with 59 packages and 51 MB on disk, and the build succeeded in 8 seconds. Tests did not pass cleanly: pytest reported 150 passed, 1 failed, 3 skipped, and 16 collection or setup errors out of 167 in 6 seconds. The log showed FastAPI import errors and one async test reported as unsupported, so the README's quickstart looks approachable, but a production-minded self-hosted deployment still needs dependency validation, provider configuration, and investigation of the test environment before trust.

Skyvern trades brittle selectors for an AI decision layer

Skyvern tackles a familiar automation problem: a script works until a website moves a button, changes its markup, or presents a slightly different flow. Instead of depending only on DOM paths and XPath selectors, it uses language models and computer vision to interpret a page, decide what to do, and operate the browser through Playwright. That makes it most interesting for workflows spread across many sites or interfaces you do not control, where maintaining one hand-written script per site becomes expensive.

The project is broader than a single Python helper. Its README describes a Playwright-compatible SDK, a no-code workflow builder, a local server and packaged UI, plus a managed cloud option. Version 1.0.51 was released on August 24, 2026, and the repository was pushed again on August 30. That combination matters: this is an actively changing product rather than an old demo with a large star count.

What happened when we ran it

Our run used commit 96618fc in an unprivileged Debian container with 3 CPUs and 8 GB of RAM, without secrets. The checkout was substantial: 5,426 files, about 1,226,366 lines of source, and 352.9 MB. Installation succeeded in 23 seconds, adding 59 packages that occupied 51 MB, while the build completed successfully in 8 seconds. Those are encouraging first steps for a project with an API, UI, database support, and several deployment paths.

The tests were the warning sign. In 6 seconds, pytest reported 150 passed, 1 failed, 3 skipped, and 16 collection or setup errors out of 167. The visible log included multiple errors where FastAPI's Depends could not be imported, plus test_alembic_with_running_loop, which failed because async functions were not natively supported in that run. We cannot infer more than the log shows, but we can say the tested checkout did not produce a clean suite in this fresh environment.

Security scanning was cleaner: pip-audit found 0 known vulnerabilities. The repository also includes 13 CI workflow files, a Dockerfile, a Compose file, and a tests directory. Those signals show that packaging, automation, and repeatable deployment are treated as real engineering concerns. They do not cancel the failed test step, so teams should reproduce the suite with the project's intended development dependencies before promoting an image.

The SDK offers useful levels of control

The strongest design choice is that Skyvern does not force every task into a fully autonomous agent. The page object exposes four focused commands: act for natural-language actions, extract for structured data, validate for checking page state, and prompt for an arbitrary model request with an optional schema. Developers can combine those calls with ordinary Playwright logic, keeping deterministic code around login, navigation, retries, or business rules while using AI only where page interpretation is genuinely hard.

There is also a higher-level agent interface and a workflow builder for less technical operators. For local use, the documented pip route supports Python 3.11, 3.12, or 3.13 and defaults to SQLite, while Docker Compose bundles PostgreSQL, the API, and the UI. The README even records fixes for specific version 1.0.31 SQLite and dependency-resolution problems, which is unusually practical documentation even though it also reveals recent packaging churn.

The managed service adds parallel instances, proxies, anti-bot mechanisms, and CAPTCHA solvers according to the README. Those capabilities explain why the cloud offering may be easier than self-hosting for hostile or high-volume sites. They also clarify a boundary: the open repository gives you the automation system, but operating reliable browser infrastructure still involves networking, credentials, model-provider configuration, storage, observability, and site-specific failure handling.

Flexibility creates operational uncertainty

AI-directed browser steps are less predictable than explicit selectors. A visually plausible action can still be the wrong business action, and a validation prompt is not automatically a transaction guarantee. Skyvern's approach is best suited to recoverable work, human-reviewed queues, data collection, form preparation, and workflows with clear postconditions. For payments, account changes, deletion, or regulated submissions, add approvals and deterministic checks outside the agent.

The repository's AGPL-3.0 license also deserves an early legal review, particularly for organizations modifying and serving the software over a network. At 22,879 stars, Skyvern has strong visibility, but 222 open issues indicate a meaningful support and triage surface. Open issue count alone does not prove poor maintenance; paired with a push on August 30 and the August 24 release, it instead describes a popular, fast-moving project with active demand and ongoing rough edges.

It belongs inside a controlled automation service

In a real stack, put Skyvern behind your job queue, secrets manager, audit logging, and approval rules. Feed it narrow tasks with explicit success criteria, save screenshots and extracted results, and let another service decide whether an outcome is acceptable. Keep plain Playwright for stable pages and reserve visual reasoning for the sections where selectors actually fail. This hybrid layout limits model cost and makes failures easier to diagnose.

Skyvern is more convincing as an automation platform than as a drop-in replacement for every browser script. Its 23-second installation and successful 8-second build show that trying it is reasonable, while our failed test run says production adoption needs more proof. If changing sites are consuming engineering time, a supervised pilot is justified. If your target pages are stable, ordinary Playwright remains the simpler default.

Alternatives

ProjectWhat it isPick it when
browser-use gh↗An open-source library for agents that operate web browsers.Pick this instead when you want a code-first browser agent library rather than Skyvern's broader server and workflow-builder package.
StagehandA browser automation framework combining natural-language actions with code.Pick this instead when a TypeScript-first API and a thinner layer over browser automation fit your stack better.
Playwright gh↗A general-purpose browser automation and testing framework.Pick this instead when stable selectors and deterministic scripts can solve the job without an AI decision layer.

What people are saying

  1. [velocity-scout] Skyvern-AI/skyvern

Sources

  1. Skyvern GitHub repository
  2. Skyvern homepage
  3. Skyvern documentation

More automation reviews

rclone · lego · OpenCLI · web-access · Karabiner-Elements · WiiUDownloader · the whole board →