mrkeyoor.comAsk

ramenvr/autopilottested 18 Nov 2026commit 34ed2db

autopilot review

Use Aura Autopilot for a focused evaluation in a documented game engine, with a specific interaction and an inspectable report.

  • Install 1.5 s
  • README snippet failed
854/854tests passed
  • 4.5 MB on disk
  • 9 dependencies
AI Toolsevaluationupdated 19 Nov 2026
+5 / 2dstars / 7d

Aura Autopilot, maintained at ramenvr/autopilot, is an AI game-playtesting project. Its README describes a coding agent that reads project source and creates gameplay harnesses, with a decision model choosing actions during play.

Verdict

Use Aura Autopilot for a focused evaluation in a documented game engine, with a specific interaction and an inspectable report. Our source installation, build and test steps passed at commit 34ed2db. Skip it if you require an existing Bevy connector, additional upstream decision providers or a spending ceiling before unattended work. Start with a known defect and record whether the report identifies it alongside the run's charges.

autopilot infographic: creates gameplay harnesses and produces a report, plus our test result and who it suits
Tested 19 November 2026: the README snippet failed. Before that: 18 Nov: 854 of 854 source tests passed; uv install took 1.5 s.

We ran it

Lab card: what happened when we ran autopilot
Lab card from our sandbox run
Screenshot of autopilot (tryaura.dev/autopilot)
tryaura.dev/autopilot
Install✓ · 26s73 packages · 475 MB
Build✓ · 2s
Tests✓ · 523s854 passed · 0 failed · 10 skipped of 854 (pytest)
Known vulns0(pip-audit)
Repo647 files~132,412 lines of source · 15 MB · 5 CI workflows · tests dir

Answers from our run

Does autopilot build from source?

Dependencies installed in 26 seconds (73 packages), and the build succeeded in 2 seconds. We cloned commit 34ed2db into a clean Debian container with 3 CPUs and no project-specific setup.

Do autopilot's tests pass?

Yes. The project's own test command finished in 523 seconds and reported 854 passed, 0 failed, 10 skipped (pytest).

Does autopilot have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use autopilot?

Bevy developers seeking an existing integration: issue #11 requests one.

What are the alternatives to autopilot?

Claude Code, Playwright. Use Aura Autopilot for a focused evaluation in a documented game engine, with a specific interaction and an inspectable report.

Setup4/5In our 2026-11-19 run it installed in 1.5 s; the README gave no task we could run as written.Editor’s note: The records contain source setup results but no completed supported editor-onboarding run from which to assess setup ease.
Docs2/5The 9,780-character README has no usage instructions or examples.Editor’s note: The README supplies engine onboarding and issue #11 identifies missing protocol documentation; the evidence does not support a numerical assessment.
Community4/5Last push on 2026-11-19, 876 stars and maintainers answered or closed 6 of the 6 most-discussed issues we sampled.Editor’s note: Sampled issues show maintainer replies, without measurements of support capacity or wider adoption.
Maturity3/5Latest release v1.0.0-1ebf2a5 on 2026-11-16, repository created 2026-10-30 and its test suite passed in our sandbox (854 of 854 passed).Editor’s note: The source checks passed, but the recorded run does not measure repeated production use or known-defect detection.

How we score

Who it’s for

Game developers evaluating the documented Unreal, Unity, Godot or Roblox integration against a specific gameplay task.
Python contributors assessing the source workflow measured at commit 34ed2db.
Teams prepared to inspect gameplay reports and recorded charges before expanding delegated work.

Who it’s NOT for

Bevy developers seeking an existing integration: issue #11 requests one.
Teams requiring a spending ceiling before unattended runs: issue #46 requests that control.
Buyers requiring upstream decision-provider selection beyond Jev: issue #45 describes a proposal and fork experiments.

Setup reality

Our November 18, 2026 source run used commit 34ed2db. Installation succeeded in 26 seconds, the build in 2 seconds and tests in 523 seconds. We recorded 73 dependencies and 475 MB on disk. The pytest summary reports 854 passed, 0 failed and 10 skipped, with a supplied total of 854. pip-audit reported 0 known vulnerabilities; its recorded coverage was 72 scanned packages and one skipped package.

The README directs users to an engine download and onboarding. It lists Windows and macOS integrations for Unreal Engine 5, Unity 6, Godot 4 and Roblox Studio. Coding-agent onboarding supports an Aura account or a TypeSafe API key for the own-key route.

A separate November 19 quickstart-lib recipe recorded commit 34ed2db713ed and installed PyPI autopilot 0.3.0. It stopped at the snippet step because the README had no Python code block to run. Its package measurements appear separately in the body.

Our Autopilot source checks passed at commit 34ed2db: installation took 26 seconds, the build took 2 seconds and tests took 523 seconds. Aura Autopilot suits game developers evaluating a documented engine integration against a specific task. Teams requiring an existing Bevy connector, additional upstream decision providers or a spending ceiling before unattended work should choose another route. The supplied issues describe requests for those capabilities.

Aura Autopilot is the game-playtesting project at ramenvr/autopilot. Its README describes an LLM coding agent that reads project source, crafts gameplay harnesses and requests action choices from a decision model. Autopilot then produces a report. Connecting a coding assistant and selecting the model that makes gameplay decisions are separate configuration decisions, a distinction that matters when choosing an account, provider or integration route. Aura Autopilot README

What people use autopilot for

The README documents four prompt types: exploration, testing, verification and gameplay goals. Its examples include exploring a crafting system, inspecting a level, checking whether a tree responds to an axe and defeating a boss while preserving health. Choose an interaction with a clear expected result for a first evaluation, then compare that expectation with the generated report. Documented gameplay prompts

The supplied issues show people approaching Autopilot from four directions: connecting a Godot project, experimenting with decision providers, examining charges and reproducing a startup failure. These are useful adoption questions because each produces a different kind of result. Account access, reaching a game's credits, a bill and an error reproduction answer different questions about using the project.

Who and task What happened What to take from it
AlexFrigenti attempted autonomous playtesting of a Godot 4.7.2 mobile game on Windows Fresh registration in Edge without ad blockers restored access; the plugin reported a running agent service An onboarding workaround. Godot account report
The author of issue #45 replayed one harness with four providers on a Phaser 4 game All four reportedly reached the credits; Anthropic recorded one timeout A gameplay experiment using a development bridge on a fork. Provider experiment
The author of issue #46 inspected a two-episode run using claude-opus-5-5 in fast mode The report records agent, reading-model and Aura charges separately A billing example. Run-cost report
The author of issue #3 reproduced simulated-game startup failure from Linux source checkouts under /tmp Bubblewrap startup failed and the MCP request stayed pending; a watchdog interrupted each reported attempt after 20 seconds A startup and error-reporting problem. MCP startup report

What happened when we ran it

Our November 18, 2026 sandbox run at commit 34ed2db passed installation, build and tests. The recorded configuration allocated 3 CPUs and 8 GB RAM and used the image label ghcr.io/astral-sh/uv:python3.12-bookworm. We measured the source workflow, including its dependency footprint and test duration. The 523-second test step gives contributors a concrete feedback time for this run.

Recorded check Result from our sandbox
Installation Succeeded in 26 seconds
Dependencies and disk 73 dependencies recorded; 475 MB on disk
Build Succeeded in 2 seconds
Test step Succeeded in 523 seconds
Pytest summary 854 passed, 0 failed, 10 skipped; supplied total field: 854
Dependency audit pip-audit reported 0 known vulnerabilities
Audit coverage 72 packages scanned; 73 reported; one skipped
Repository scan 647 files, approximately 132,412 source lines, 15 MB checked out
Project signals 5 CI workflow files, a tests directory and no Dockerfile detected

The recorded test summary lists passing, failing and skipped results separately, so the table retains those fields and the supplied total. Installation, build and test status each succeeded. The audit record likewise separates its result from its coverage: zero known vulnerabilities, 72 packages scanned, 73 reported and one skipped. Those details belong beside the headline outcomes when assessing the source workflow.

A separate November 19 quickstart-lib recipe recorded commit 34ed2db713ed and installed PyPI autopilot 0.3.0 in a fresh unprivileged container. Its configuration used 2 CPUs, 4 GB RAM, no published ports and no secrets, with image label ghcr.io/astral-sh/uv:python3.12-bookworm. The table keeps this Python-library recipe separate from the source installation and test measurements above.

Recipe step Recorded outcome Interpretation
Clone quickstart-lib recipe recorded 2 seconds Recipe preparation.
Install uv pip install autopilot installed autopilot 0.3.0 in 1.5 seconds PyPI installation result.
Installed size 4.5 MB in the virtual environment's site-packages Recipe package footprint.
Dependency count 9 packages besides autopilot Recipe dependency count.
Snippet Failed because the README had no Python code block to run The recipe stopped at its snippet requirement.
Finding no-snippet: the README has no Python code block to run The recipe recorded the failed step as a finding.
Teardown Reported ok Cleanup completed.

Where people get stuck: login, startup and engine adapters

Issue #8 records account recovery through fresh registration in Edge without ad blockers. The Godot plugin had loaded, but authentication blocked progress. The reporter then received a welcome email, signed in and confirmed that the plugin recognized the account with data sharing disabled. The actionable sequence is fresh registration in the suggested browser configuration, followed by checking the account and service status. Account recovery thread

Report Action supported by the record Resolution shown
Registration crashes or device_already_registered appears Recreate the account in a browser without ad blockers; the reporter used Edge Successful login and a running service. Issue #8
Password recovery reports success but no email arrives The maintainer suggested fresh registration, then contacting support if necessary Fresh registration delivered a welcome email. Issue #8
MCP start_run stays pending after sandbox failure The issue reproduces failure from a Linux checkout under /tmp using a simulated-game test The issue is marked closed. Issue #3
Bevy integration and protocol documentation are missing A contributor proposed a Rust plugin and Python adapter; the maintainer invited a contribution and suggested studying implementation files Open integration proposal and documentation workaround. Issue #11

Issue #3 identifies a failure before gameplay: bubblewrap exits before opening the session, the external-agent startup worker raises SandboxError and the MCP startup request stays pending. Its reproduction uses the existing simulated-game test without an engine, credentials or model calls. Contributors investigating the same symptoms can compare the checkout location and startup state with that report. Startup failure account

Issue #11 gives integration contributors two concrete jobs: an engine plugin and a Python adapter. It also identifies the absent protocol document referenced by autopilot/bridge.py. The maintainer says the document contains private-repository details and suggests deriving documentation from implementation files. For a Bevy contributor, that makes reading the bridge implementation part of the integration work before preparing a pull request. Bevy integration discussion

Unattended play needs a cost decision first

Issue #46 reports $2.66 for the agent, $0.53 for the reading model and a $3.06 Aura bill. The author describes a two-episode run with claude-opus-5-5 in fast mode and identifies Jev on TypeSafe as the decision model. Preserve the amounts in their original categories when using this example to discuss payment routes. Reported run charges

The author requests advance estimates, confirmation and a maximum-dollar setting. Proposed estimate commands would display expected costs without starting gameplay, connecting a plugin or calling a model. The maintainer raises concerns about underestimating long goal-driven runs. For a team deciding whether to delegate repeated activity, the practical question is whether its required spending control is available before it authorizes that work. Cost-control proposal

Security reporting and onboarding choices

The maintainer confirmed enabling GitHub private vulnerability reporting in issue #7. A researcher had reported that SECURITY.md directed people to a disabled channel and requested a private way to send details. The maintainer's reply provides a completed resolution to that reporting-channel problem, giving researchers a place to submit sensitive details privately. Private reporting resolution

The README makes run sharing an onboarding choice and describes reading project source and probing the running game as normal operation. For a proprietary project, inspect those choices and decide which project copy to use before connecting it. Keep the account route, model configuration and sharing choice in the evaluation record alongside the prompt and resulting report. Onboarding and gameplay instructions

Autopilot vs Claude Code and browser test tools

Claude Code supports code investigation and edits; Playwright supports web automation and testing. Autopilot documents gameplay through engine integrations and an MCP connection for coding agents. Choose according to the target you need to control and the result you need to inspect. Claude Code can also act as an Autopilot client, so these tools can form parts of the same workflow. Claude Code README, Playwright README, Autopilot MCP setup

Choice Why consider it Supporting evidence
Autopilot with Claude Code Combine coding assistance with Autopilot's gameplay interface Autopilot documents its MCP connection and one-time setup.
Claude Code alone Investigate, explain or edit project code Its README describes a terminal coding tool that understands a codebase and handles development tasks.
A dedicated decision service inside Autopilot Assess component-provider choice as a procurement requirement Issue #45 proposes additional decision providers and reports fork experiments.
Playwright Write browser checks with expected behavior or give an agent browser control Its README documents web automation and testing across Chromium, Firefox and WebKit.

The README's one-time Claude Code connection is autopilot connect claude, after installation and onboarding. The own-key route requires a TypeSafe API key. Issue #45 concerns replacing the decision component, a separate choice from the coding assistant. Its author reports replaying one harness through four providers on a fork; the maintainer distinguishes dedicated decision interfaces from ordinary language-model use. Connection instructions, Provider discussion

Which Autopilot this review covers

This autopilot review covers Aura Autopilot at ramenvr/autopilot, the game-playtesting project. The supplied search results also include Tesla driving assistance, Microsoft's personal agent, advertising discussions and an investing app. Use the repository name when looking for setup reports or alternatives so that advice about another Autopilot product stays separate from this game's development workflow. Tesla result, Microsoft result, Advertising discussion, Investing discussion

Questions people ask

Godot setup requires the plugin in addons/autopilot and activation in Project Settings > Plugins. The README then directs users through account or own-key onboarding and an agent-service readiness check. The questions below come from the supplied issue synthesis and cover setup, access, integrations, billing and startup. Their answers point to the relevant instructions or discussion. Godot setup instructions

How do I set up and use ramenvr/autopilot for autonomous Godot playtesting?

The README specifies Godot 4.7 or later. Download the Godot build, place the plugin in addons/autopilot, enable it in Project Settings > Plugins and complete onboarding. After the service-ready message, submit a task; the 10-minute exploration run is optional. Godot installation instructions

Why does Aura registration crash or say my device is already registered?

Issue #8's reporter restored access by registering again in Edge without ad-blocking extensions. The successful sequence included the welcome email, sign-in and recognition by the Godot plugin. Registration workaround

What should I do if Aura says it sent a password reset but no email arrives?

The maintainer suggested fresh registration in a browser without ad blockers, then contacting support@tryaura.dev if that failed. Account recovery advice

How can I privately report an Autopilot security vulnerability?

GitHub private vulnerability reporting is the channel the maintainer confirmed enabling in issue #7. Use it for sensitive details. Maintainer confirmation

Can Autopilot playtest Bevy games?

Issue #11 requests Bevy support through a Rust plugin and Python adapter. The maintainer invited a pull request. Bevy integration proposal

Where is the protocol documentation needed to build another engine plugin?

The maintainer says docs/protocol.md was withheld because it contains private-repository details and suggests studying autopilot/bridge.py to derive documentation. Protocol documentation discussion

Can I estimate an Autopilot run's cost before starting it?

Issue #46 proposes estimate commands; its command examples are feature-design proposals. The maintainer raises concerns about underestimating long runs. Estimate proposal

Can I set a spending limit for unattended Autopilot runs?

Issue #46 requests a maximum-dollar setting and reports that no setting stops a run that can cost too much. Spending-control request

Can I choose a decision model other than Jev?

Issue #45 proposes additional providers and reports four experimental runs on a fork. The maintainer welcomes dedicated decision-model interfaces and rejects ordinary Anthropic models for this role. Decision-provider discussion

Why does MCP start_run remain pending when sandbox startup fails?

Issue #3 reports that the external-agent startup worker raises SandboxError while the MCP request stays pending after bubblewrap exits. Pending startup report

What we have not tested yet

Our run covered source installation, build and tests, rather than an upstream editor session finding a known game defect. Native downloads, multiplayer behavior, retained-memory quality and improvement across repeated runs remain outside our measurements. The Godot account report ends at readiness to test. The Phaser experiment supplies one run per provider on a fork, rather than an independent bug-detection benchmark or provider ranking.

The generic recipe's PyPI autopilot 0.3.0 installation does not identify that distribution as the intended Aura engine workflow, and its missing-snippet failure does not demonstrate broken gameplay. The sandbox image labels do not establish actual Python or operating-system versions. Raw toolchain logs and pytest output are needed to resolve runtime versions and the inconsistent test total. The audit is not a comprehensive security assessment, and optional sharing controls do not establish local-only execution.

Issue #3's closed label comes without a demonstrated repair or successful rerun in the supplied extract. The account workaround does not prove a universal authentication cause or repair to the original account's reset-email delivery. The provider proposal does not establish shipped upstream alternatives, and the reference bill neither establishes typical costs nor explains how its categories reconcile. The proposed spending controls are not demonstrated implementations.

Aura's announcement claims 43% greater accuracy, almost twice the speed and an advantage over Claude Code, without supplied comparative methodology. Those vendor claims do not determine this verdict. A useful next evaluation is a supported plugin run against a known defect: record the prompt, inspect whether the report identifies the defect and keep the bill beside the result. Aura announcement

Alternatives

ProjectWhat it isPick it when
Claude Code gh↗A terminal coding assistant that works with project code.Choose it for code investigation and edits. Autopilot's README also documents connecting Claude Code to its gameplay integration through MCP.
Playwright gh↗A framework for web automation and testing.Choose it for scripted browser checks with explicit expected behavior. Its README also documents browser control for AI agents.

What people are saying

  1. [velocity-scout] ramenvr/autopilot

Sources

  1. Aura Autopilot README
  2. MrKeyoor source sandbox run, November 18, 2026
  3. Godot account workaround
  4. Decision-model proposal and fork experiment
  5. Run-cost and spending-control request
  6. MCP startup failure
  7. Bevy and protocol documentation discussion
  8. Private vulnerability reporting resolution
  9. Claude Code README
  10. Playwright README

More ai tools reviews