Our Autopilot source checks passed at commit 34ed2db: installation took 26 seconds, the build took 2 seconds and tests took 523 seconds. Aura Autopilot suits game developers evaluating a documented engine integration against a specific task. Teams requiring an existing Bevy connector, additional upstream decision providers or a spending ceiling before unattended work should choose another route. The supplied issues describe requests for those capabilities.
Aura Autopilot is the game-playtesting project at ramenvr/autopilot. Its README describes an LLM coding agent that reads project source, crafts gameplay harnesses and requests action choices from a decision model. Autopilot then produces a report. Connecting a coding assistant and selecting the model that makes gameplay decisions are separate configuration decisions, a distinction that matters when choosing an account, provider or integration route. Aura Autopilot README
What people use autopilot for
The README documents four prompt types: exploration, testing, verification and gameplay goals. Its examples include exploring a crafting system, inspecting a level, checking whether a tree responds to an axe and defeating a boss while preserving health. Choose an interaction with a clear expected result for a first evaluation, then compare that expectation with the generated report. Documented gameplay prompts
The supplied issues show people approaching Autopilot from four directions: connecting a Godot project, experimenting with decision providers, examining charges and reproducing a startup failure. These are useful adoption questions because each produces a different kind of result. Account access, reaching a game's credits, a bill and an error reproduction answer different questions about using the project.
| Who and task | What happened | What to take from it |
|---|---|---|
| AlexFrigenti attempted autonomous playtesting of a Godot 4.7.2 mobile game on Windows | Fresh registration in Edge without ad blockers restored access; the plugin reported a running agent service | An onboarding workaround. Godot account report |
| The author of issue #45 replayed one harness with four providers on a Phaser 4 game | All four reportedly reached the credits; Anthropic recorded one timeout | A gameplay experiment using a development bridge on a fork. Provider experiment |
| The author of issue #46 inspected a two-episode run using claude-opus-5-5 in fast mode | The report records agent, reading-model and Aura charges separately | A billing example. Run-cost report |
| The author of issue #3 reproduced simulated-game startup failure from Linux source checkouts under /tmp | Bubblewrap startup failed and the MCP request stayed pending; a watchdog interrupted each reported attempt after 20 seconds | A startup and error-reporting problem. MCP startup report |
What happened when we ran it
Our November 18, 2026 sandbox run at commit 34ed2db passed installation, build and tests. The recorded configuration allocated 3 CPUs and 8 GB RAM and used the image label ghcr.io/astral-sh/uv:python3.12-bookworm. We measured the source workflow, including its dependency footprint and test duration. The 523-second test step gives contributors a concrete feedback time for this run.
| Recorded check | Result from our sandbox |
|---|---|
| Installation | Succeeded in 26 seconds |
| Dependencies and disk | 73 dependencies recorded; 475 MB on disk |
| Build | Succeeded in 2 seconds |
| Test step | Succeeded in 523 seconds |
| Pytest summary | 854 passed, 0 failed, 10 skipped; supplied total field: 854 |
| Dependency audit | pip-audit reported 0 known vulnerabilities |
| Audit coverage | 72 packages scanned; 73 reported; one skipped |
| Repository scan | 647 files, approximately 132,412 source lines, 15 MB checked out |
| Project signals | 5 CI workflow files, a tests directory and no Dockerfile detected |
The recorded test summary lists passing, failing and skipped results separately, so the table retains those fields and the supplied total. Installation, build and test status each succeeded. The audit record likewise separates its result from its coverage: zero known vulnerabilities, 72 packages scanned, 73 reported and one skipped. Those details belong beside the headline outcomes when assessing the source workflow.
A separate November 19 quickstart-lib recipe recorded commit 34ed2db713ed and installed PyPI autopilot 0.3.0 in a fresh unprivileged container. Its configuration used 2 CPUs, 4 GB RAM, no published ports and no secrets, with image label ghcr.io/astral-sh/uv:python3.12-bookworm. The table keeps this Python-library recipe separate from the source installation and test measurements above.
| Recipe step | Recorded outcome | Interpretation |
|---|---|---|
| Clone | quickstart-lib recipe recorded 2 seconds | Recipe preparation. |
| Install | uv pip install autopilot installed autopilot 0.3.0 in 1.5 seconds | PyPI installation result. |
| Installed size | 4.5 MB in the virtual environment's site-packages | Recipe package footprint. |
| Dependency count | 9 packages besides autopilot | Recipe dependency count. |
| Snippet | Failed because the README had no Python code block to run | The recipe stopped at its snippet requirement. |
| Finding | no-snippet: the README has no Python code block to run | The recipe recorded the failed step as a finding. |
| Teardown | Reported ok | Cleanup completed. |
Where people get stuck: login, startup and engine adapters
Issue #8 records account recovery through fresh registration in Edge without ad blockers. The Godot plugin had loaded, but authentication blocked progress. The reporter then received a welcome email, signed in and confirmed that the plugin recognized the account with data sharing disabled. The actionable sequence is fresh registration in the suggested browser configuration, followed by checking the account and service status. Account recovery thread
| Report | Action supported by the record | Resolution shown |
|---|---|---|
| Registration crashes or device_already_registered appears | Recreate the account in a browser without ad blockers; the reporter used Edge | Successful login and a running service. Issue #8 |
| Password recovery reports success but no email arrives | The maintainer suggested fresh registration, then contacting support if necessary | Fresh registration delivered a welcome email. Issue #8 |
| MCP start_run stays pending after sandbox failure | The issue reproduces failure from a Linux checkout under /tmp using a simulated-game test | The issue is marked closed. Issue #3 |
| Bevy integration and protocol documentation are missing | A contributor proposed a Rust plugin and Python adapter; the maintainer invited a contribution and suggested studying implementation files | Open integration proposal and documentation workaround. Issue #11 |
Issue #3 identifies a failure before gameplay: bubblewrap exits before opening the session, the external-agent startup worker raises SandboxError and the MCP startup request stays pending. Its reproduction uses the existing simulated-game test without an engine, credentials or model calls. Contributors investigating the same symptoms can compare the checkout location and startup state with that report. Startup failure account
Issue #11 gives integration contributors two concrete jobs: an engine plugin and a Python adapter. It also identifies the absent protocol document referenced by autopilot/bridge.py. The maintainer says the document contains private-repository details and suggests deriving documentation from implementation files. For a Bevy contributor, that makes reading the bridge implementation part of the integration work before preparing a pull request. Bevy integration discussion
Unattended play needs a cost decision first
Issue #46 reports $2.66 for the agent, $0.53 for the reading model and a $3.06 Aura bill. The author describes a two-episode run with claude-opus-5-5 in fast mode and identifies Jev on TypeSafe as the decision model. Preserve the amounts in their original categories when using this example to discuss payment routes. Reported run charges
The author requests advance estimates, confirmation and a maximum-dollar setting. Proposed estimate commands would display expected costs without starting gameplay, connecting a plugin or calling a model. The maintainer raises concerns about underestimating long goal-driven runs. For a team deciding whether to delegate repeated activity, the practical question is whether its required spending control is available before it authorizes that work. Cost-control proposal
Security reporting and onboarding choices
The maintainer confirmed enabling GitHub private vulnerability reporting in issue #7. A researcher had reported that SECURITY.md directed people to a disabled channel and requested a private way to send details. The maintainer's reply provides a completed resolution to that reporting-channel problem, giving researchers a place to submit sensitive details privately. Private reporting resolution
The README makes run sharing an onboarding choice and describes reading project source and probing the running game as normal operation. For a proprietary project, inspect those choices and decide which project copy to use before connecting it. Keep the account route, model configuration and sharing choice in the evaluation record alongside the prompt and resulting report. Onboarding and gameplay instructions
Autopilot vs Claude Code and browser test tools
Claude Code supports code investigation and edits; Playwright supports web automation and testing. Autopilot documents gameplay through engine integrations and an MCP connection for coding agents. Choose according to the target you need to control and the result you need to inspect. Claude Code can also act as an Autopilot client, so these tools can form parts of the same workflow. Claude Code README, Playwright README, Autopilot MCP setup
| Choice | Why consider it | Supporting evidence |
|---|---|---|
| Autopilot with Claude Code | Combine coding assistance with Autopilot's gameplay interface | Autopilot documents its MCP connection and one-time setup. |
| Claude Code alone | Investigate, explain or edit project code | Its README describes a terminal coding tool that understands a codebase and handles development tasks. |
| A dedicated decision service inside Autopilot | Assess component-provider choice as a procurement requirement | Issue #45 proposes additional decision providers and reports fork experiments. |
| Playwright | Write browser checks with expected behavior or give an agent browser control | Its README documents web automation and testing across Chromium, Firefox and WebKit. |
The README's one-time Claude Code connection is autopilot connect claude, after installation and onboarding. The own-key route requires a TypeSafe API key. Issue #45 concerns replacing the decision component, a separate choice from the coding assistant. Its author reports replaying one harness through four providers on a fork; the maintainer distinguishes dedicated decision interfaces from ordinary language-model use. Connection instructions, Provider discussion
Which Autopilot this review covers
This autopilot review covers Aura Autopilot at ramenvr/autopilot, the game-playtesting project. The supplied search results also include Tesla driving assistance, Microsoft's personal agent, advertising discussions and an investing app. Use the repository name when looking for setup reports or alternatives so that advice about another Autopilot product stays separate from this game's development workflow. Tesla result, Microsoft result, Advertising discussion, Investing discussion
Questions people ask
Godot setup requires the plugin in addons/autopilot and activation in Project Settings > Plugins. The README then directs users through account or own-key onboarding and an agent-service readiness check. The questions below come from the supplied issue synthesis and cover setup, access, integrations, billing and startup. Their answers point to the relevant instructions or discussion. Godot setup instructions
How do I set up and use ramenvr/autopilot for autonomous Godot playtesting?
The README specifies Godot 4.7 or later. Download the Godot build, place the plugin in addons/autopilot, enable it in Project Settings > Plugins and complete onboarding. After the service-ready message, submit a task; the 10-minute exploration run is optional. Godot installation instructions
Why does Aura registration crash or say my device is already registered?
Issue #8's reporter restored access by registering again in Edge without ad-blocking extensions. The successful sequence included the welcome email, sign-in and recognition by the Godot plugin. Registration workaround
What should I do if Aura says it sent a password reset but no email arrives?
The maintainer suggested fresh registration in a browser without ad blockers, then contacting support@tryaura.dev if that failed. Account recovery advice
How can I privately report an Autopilot security vulnerability?
GitHub private vulnerability reporting is the channel the maintainer confirmed enabling in issue #7. Use it for sensitive details. Maintainer confirmation
Can Autopilot playtest Bevy games?
Issue #11 requests Bevy support through a Rust plugin and Python adapter. The maintainer invited a pull request. Bevy integration proposal
Where is the protocol documentation needed to build another engine plugin?
The maintainer says docs/protocol.md was withheld because it contains private-repository details and suggests studying autopilot/bridge.py to derive documentation. Protocol documentation discussion
Can I estimate an Autopilot run's cost before starting it?
Issue #46 proposes estimate commands; its command examples are feature-design proposals. The maintainer raises concerns about underestimating long runs. Estimate proposal
Can I set a spending limit for unattended Autopilot runs?
Issue #46 requests a maximum-dollar setting and reports that no setting stops a run that can cost too much. Spending-control request
Can I choose a decision model other than Jev?
Issue #45 proposes additional providers and reports four experimental runs on a fork. The maintainer welcomes dedicated decision-model interfaces and rejects ordinary Anthropic models for this role. Decision-provider discussion
Why does MCP start_run remain pending when sandbox startup fails?
Issue #3 reports that the external-agent startup worker raises SandboxError while the MCP request stays pending after bubblewrap exits. Pending startup report
What we have not tested yet
Our run covered source installation, build and tests, rather than an upstream editor session finding a known game defect. Native downloads, multiplayer behavior, retained-memory quality and improvement across repeated runs remain outside our measurements. The Godot account report ends at readiness to test. The Phaser experiment supplies one run per provider on a fork, rather than an independent bug-detection benchmark or provider ranking.
The generic recipe's PyPI autopilot 0.3.0 installation does not identify that distribution as the intended Aura engine workflow, and its missing-snippet failure does not demonstrate broken gameplay. The sandbox image labels do not establish actual Python or operating-system versions. Raw toolchain logs and pytest output are needed to resolve runtime versions and the inconsistent test total. The audit is not a comprehensive security assessment, and optional sharing controls do not establish local-only execution.
Issue #3's closed label comes without a demonstrated repair or successful rerun in the supplied extract. The account workaround does not prove a universal authentication cause or repair to the original account's reset-email delivery. The provider proposal does not establish shipped upstream alternatives, and the reference bill neither establishes typical costs nor explains how its categories reconcile. The proposed spending controls are not demonstrated implementations.
Aura's announcement claims 43% greater accuracy, almost twice the speed and an advantage over Claude Code, without supplied comparative methodology. Those vendor claims do not determine this verdict. A useful next evaluation is a supported plugin run against a known defect: record the prompt, inspect whether the report identifies the defect and keep the bill beside the result. Aura announcement


