Three public tasks test game repair through runtime behavior
GamePhanes has 3 cases packaged for Harbor, each checking a repaired Godot project at runtime beyond files, commands, and exit codes. One task covers a phase jump and relay sequence. Another deals with deterministic rollback. The hardest public example exercises platforming details such as swept collision, input buffering, moving-platform carry, and crush handling. The subject is engineering work, not a bot learning to play.
The benchmark is compact enough to inspect. Our checkout contained 223 files, about 4,823 lines of source, and occupied 41 MB before installation. Each Harbor package keeps the instruction, starter project, external tests, container definition, and maintainer solution in named locations. That structure makes the acceptance boundary readable, which is helpful for researchers checking why an agent passed. It also means the public tasks cannot double as secret competition cases without private variants and hidden evaluators.
What happened when we ran it
Our sandbox installed commit 16e6197 in 6 seconds. npm added 0 packages, and the checkout plus installed state used 51 MB on disk. The repository declares Node.js 22 or newer and has no third-party runtime dependencies in its package manifest. There was no build script or build target, so we skipped that step rather than treating a missing command as a success. Those results cover the Node-side repository tools, not the Godot containers or a model-driven Harbor trial.
Node's test runner finished in 10 seconds with 25 passed and 0 failed. npm audit found 0 known vulnerabilities across all severity levels. The repository also has a tests directory and 2 CI workflow files. One workflow runs the unit suite and validates an example task on Node 24; the other publishes the static site. This is a clean baseline for schema and command behavior, though the project's own guide says static validation cannot certify runtime gameplay.
A 6-second npm install is only the first setup layer
The 6-second npm install checks only the local Node tools; complete evaluation needs a larger toolchain. The contributor guide requires Node.js 22 or newer and Godot 4.x for end-to-end playtests. The hosted-service guide adds Linux, Docker, the Harbor version selected by the evaluator, and the engine pinned by each task. Running a demo also requires GAMEPHANES_GODOT or an explicit Godot path. None of those tools was exercised by our 6-second dependency-free install.
A live agent introduces credentials and service configuration. GamePhanes includes a host-side adapter for Harbor 0.22.x, but the operator must provide an OpenAI-compatible base URL, API key, and model name. The worker must isolate the candidate workspace, keep hidden tests outside it, disable unapproved network access, and redact stored trajectories. On our 3-CPU, 8 GB sandbox, we verified the repository's Node path only. Budget a separate rehearsal for the exact engine image, model gateway, and verifier you will use.
Public solutions make the 3 tasks inspectable, not leakage-resistant
Every calibrated package follows Harbor schema 1.1 and includes a solution/solve.sh file for maintainers. That is a sensible proof that a task can be solved from its starter state. It also puts the answer beside the public task. The README draws the boundary clearly: starter projects, scoring principles, and public reports stay visible, while sealed variants, hidden evaluators, and private trajectories belong to a service layer. Anyone running a leaderboard must build and protect that private side.
The catalogue has 3 calibrated tasks, while 20 remain an ambition
The v0.1.0 README separates 6 playable reference projects from 3 calibrated Harbor tasks. That distinction matters. A playable demo is an example feedback surface; it is not another scored repair case. The README describes a 20-task production slate across gameplay, engine work, interface behavior, content systems, and delivery quality. It also says planned tasks are not measured scores until a versioned package and repeatable run exist. Base a current study on the 3 published cases.
August activity is current, but there is no issue history yet
GitHub showed 540 stars, 22 forks, and 0 open issues or pull requests on August 31, 2026; the repository was created August 21 and last pushed August 25. The issue API returned no closed entries either. That is current development and visible interest, but it offers no maintenance record for bug reports, contributor discussions, or compatibility fixes. The repository moved from GamePhanes/GamePhanes to GamePhanesStudio/GamePhanes, and GitHub redirects the old address.
Versioning is similarly young. A v0.1.0 tag exists, while GitHub's releases endpoint returns no published release. The 2 CI workflows and all 25 passing Node tests are useful signs for a new repository; neither tells us how upgrades to Harbor or Godot will be handled over time. A team adopting GamePhanes now should pin the commit, engine, container, and evaluator versions in every run record. That makes today's comparison reproducible even if the project changes quickly.
GamePhanes fits Godot studies better than general agent rankings
GamePhanes v0.1.0 is the narrow choice for Godot repair; Terminal-Bench 1 covers general terminal tasks. SWE-bench is aimed at resolving real repository issues, and Windows Agent Arena evaluates agents acting through desktop applications. GamePhanes earns its place when the test must combine source repair with executable Godot behavior. Its 3 public tasks offer more runtime detail than a patch-only score, but much less breadth than those wider evaluation programs.
The low trial cost makes that narrow use easy to justify. Our run took 6 seconds to install, added 0 packages, and passed 25 tests in 10 seconds. Use those results as evidence that the repository tools are tidy. For the benchmark itself, insist on the full Harbor record the maintainers prescribe: pinned versions, a clean container run, verifier output, Oracle result, and a baseline. Without that record, you have validated the package format rather than the agent's ability to repair a game.