mrkeyoor.com_
Fri 16 Oct 21:34 UTC
Automationevaluationupdated 16 Oct 2026

artemis review

Artemis lets a coding assistant or a plain-language request operate Android apps on a connected phone or emulator. It collects screenshots, device logs and execution traces so developers can investigate what happened during an automated journey.

Verdict

Our Artemis run installed 208 packages and built successfully, then tests stopped after 8 seconds because OpenCV could not load libGL.so.1. Try it for supervised Android exploration and diagnostic traces if you can maintain the device host and model configuration. Require explicit verification for release decisions, especially when using Flash, whose documented workflow omits checkpoint checks and a final report.

We ran it

Lab card: what happened when we ran artemisScreenshot of artemis (github.com/google/artemis)
Install✓ · 46s208 packages · 1174 MB
Build✓ · 4s
Tests✗ · 8sran, no count parsed
Known vulns0(pip-audit)
Repo817 files~179,268 lines of source · 36.6 MB · 1 CI workflows · Dockerfile · tests dir

Answers from our run

Does artemis build from source?

Dependencies installed in 46 seconds (208 packages), and the build succeeded in 4 seconds. We cloned commit 351ca84 into a clean Debian container with 3 CPUs and no project-specific setup.

Do artemis's tests pass?

The test command failed in our container, and its output did not report a pass or fail count.

Does artemis have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use artemis?

Teams expecting a green fresh-container check immediately: our tests stopped after 8 seconds because OpenCV could not load libGL.so.1.

What are the alternatives to artemis?

Appium, Maestro. Our Artemis run installed 208 packages and built successfully, then tests stopped after 8 seconds because OpenCV could not load libGL.

Setup2/5208 packages installed, but tests stopped on missing libGL.so.1
Docs4/5Clear device, MCP, profile and helper instructions
Community4/5October issue and PR activity follows the October 1 push
Maturity3/5Useful diagnostic design, no published release and lab tests blocked

Who it’s for

Android developers who want an agent to explore a changed app and return screenshots and device logs.
Teams with a device host, model-provider access and time to inspect execution traces.
Claude Code and other MCP users who want Android device actions available inside their development workflow.

Who it’s NOT for

Teams expecting a green fresh-container check immediately: our tests stopped after 8 seconds because OpenCV could not load libGL.so.1.
Release gates using Flash as a complete verification process: its documented omissions include checkpoints, a task plan and a final report.
iOS teams choosing a supported device runner today: the README lists iOS expansion in its roadmap.
Managed phones that prohibit an automatically installed accessibility helper: the default device path installs one, though an opt-out and UIAutomator2 route exist.

Setup reality

Our sandbox installed commit 351ca84 in 46 seconds, adding 208 packages and using 1,174 MB. The build passed in 4 seconds. Tests failed with exit code 4 after 8 seconds because importing cv2 could not find libGL.so.1. Pip-audit reported 0 known vulnerabilities.

You need a connected Android phone with USB debugging or an emulator, plus model-provider credentials configured for the selected backend. The launcher documents toolchain setup and offers global MCP and agent-rule configuration. Those runtime steps were not established by our package build.

The lab used Python 3.12 on Debian with 3 CPUs, 8 GB of RAM and no secrets. Native libraries still matter after the Python install. The default device path also installs an accessibility helper, with documented opt-out and fallback settings.

The Python build passes before device automation is ready

Artemis built successfully in 4 seconds in our sandbox, then its tests stopped while importing OpenCV. That's a useful introduction to this Android automation project: getting the Python code packaged leaves native libraries and a connected device to deal with. The attractive part comes after that work. You describe a task in ordinary language, and Artemis observes a phone's interface, chooses actions and records what happened.

The README describes cross-app workflows, accessibility-based targeting and visual fallbacks for custom interfaces. Screenshots and Logcat output can help a coding assistant investigate an app after it makes a change. I'd consider it for exploratory checks where the useful result is a trace you can inspect. A release requirement with an exact expected outcome still needs an explicit assertion, regardless of how convincingly an agent describes its actions.

What happened when we ran it

Our run installed Artemis commit 351ca84 in 46 seconds, adding 208 packages and occupying 1,174 MB on disk. The build succeeded in 4 seconds. We used the Python 3.12 Bookworm uv image in an unprivileged Debian sandbox with 3 CPUs, 8 GB of RAM and no secrets on October 16, 2026. These figures describe repository setup, without establishing whether Artemis can complete a phone workflow.

The test command failed with exit code 4 after 8 seconds. The supplied traceback reaches artemis.context, then the video utilities, then cv2. OpenCV's native import raises ImportError because libGL.so.1 cannot be found. That identifies the immediate blocker in this container. It does not establish a failing application assertion, and the log provides no passing-test count or evidence of what would happen after supplying the library.

Pip-audit reported 0 known vulnerabilities. The checkout contained 817 files, roughly 179,268 source lines and 36.6 MB before installation. Our scan found a Dockerfile, a tests directory and 1 CI workflow file. None of those signals overrules the import failure. The defensible finding is that Python installation and build completed, while the available environment could not get through the test command's native-library import.

The launcher also touches device and agent configuration

The quick start requires an Android device with USB debugging enabled or an emulator. Its launcher is documented to detect and install toolchains such as ADB, scrcpy and FFmpeg, then offer global MCP configuration and agent rules. Model-provider credentials live in environment configuration. Those are meaningful workstation changes, so I would read the prompts and select the clients I actually use before accepting global setup.

Our 208-package installation does not validate that launcher, an attached phone or authenticated model calls. A missing libGL.so.1 in the lab is evidence that the Python environment alone was insufficient. It isn't evidence that the documented startup script fails identically. For a deployment image, reproduce the package setup and resolve the native import before spending time diagnosing phone permissions or model behavior. Otherwise, unrelated setup failures become hard to separate.

Flash omits the checks that Pro adds

Artemis requires Python 3.12 or later and exposes Flash and Pro execution profiles. The README says Flash has no task plan, checkpoint verification or final report, and no ADB shell. Pro adds a maintained plan, checks before actions and a Checker for the requested goal. Pick the profile by the evidence your task requires. A quick navigation request and a release-signoff investigation should not inherit the same verification expectations.

The 8-second lab failure leaves both profiles unvalidated here. I would begin with a disposable test account and a narrow journey whose destination is easy to inspect, then examine the captured state myself. The README's default Flash loop is unbounded, although a turn setting exists. Set a task limit deliberately when connecting a model-backed agent to a device, particularly for unattended work that could keep retrying an unavailable target.

The accessibility helper changes the phone's environment

The first device task installs an accessibility helper according to the README. It reads layout information, appears in Accessibility settings and shows a running notification. Other automation using UiAutomation can suppress it unless configured to preserve accessibility services. Artemis documents helper inspection and removal, an opt-out of automatic installation, and a UIAutomator2 alternative. That matters if your lab already has another tool controlling the same phone.

The 1,174 MB host environment is therefore only part of the operating footprint. Device state and connection setup deserve their own checklist. Open issue 184 reports that the wireless panel connects but cannot perform initial pairing. Its documented workaround is a terminal adb pair before using the panel. A proposed fix exists as an open pull request, which should not be treated as a shipped capability.

October discussions continue without a published release

GitHub listed 12,033 stars and 139 open issues and PRs, with the last push dated October 1, 2026. The latest-release endpoint returned no published release. Discussion and proposed fixes continued in October, including wireless pairing and local model support. That combination describes an active development project without a published release to anchor adoption. It doesn't justify calling the repository abandoned, or treating an open contribution as merged functionality.

Appium is the stronger starting point when you want explicit WebDriver scripts and platform drivers. Maestro suits teams that want readable YAML flows and repeatable assertions, and it also has an MCP path. Artemis earns a trial when exploratory cross-app work and diagnostic traces justify a model-driven operator. Before giving it a permanent CI role, clear the native-library import and require a successful representative device journey with an inspectable result.

Alternatives

ProjectWhat it isPick it when
Appium gh↗A WebDriver-based automation framework with platform-specific drivers.pick this instead when you want explicit scripted actions and assertions across supported platforms.
MaestroA UI automation framework built around readable YAML command flows.pick this instead when repeatable authored flows matter more than a model choosing the journey.

What people are saying

  1. [github-trending] google/artemis
  2. [theverge] NASA dumped over 800GB of Artemis II data online

Sources

  1. Artemis README, profiles and accessibility helper
  2. Artemis model credential template
  3. Artemis Python requirements
  4. Wireless ADB panel pairing issue
  5. Proposed wireless pairing change
  6. Artemis release history
  7. Appium WebDriver framework documentation
  8. Maestro YAML flows and MCP documentation

More automation reviews

microduck · supersidian · genspark-claw · orc-claude-plugin · tiktok-views-bot · Ydisks-Xianyu-Helper · the whole board →