mrkeyoor.com_
Thu 12 Nov 02:19 UTC
Automationevaluationupdated 24 Oct 2026

skypilot review

SkyPilot is infrastructure orchestration software for AI workloads. Its README describes one YAML task format or Python API for Kubernetes, Slurm, and 20+ clouds, with provisioning, code sync, setup commands, job execution, and log streaming. Our recorded run checked local setup steps; it did not establish that workloads run successfully across those targets.

+19stars / 7d
Verdict

SkyPilot merits evaluation when a shared task interface across different infrastructure is the requirement. Our October 1, 2026 run at a72a881 recorded successful install and build steps, but the test command stopped during configuration import because UID 1000 could not be resolved. That result establishes a local setup obstacle, not a broken scheduler or verified cloud support. Before adoption, check the installed CLI, run sky check, and launch a small representative workload on each intended target. Review the dated audit findings and the reported secret-handling and naming behavior against your chosen revision.

We ran it

Lab card: what happened when we ran skypilotScreenshot of skypilot (skypilot.ai)
Install✓ · 171s381 packages · 1681 MB
Build✓ · 11s
Tests✗ · 16sran, no count parsed
Known vulns12(pip-audit)
Repo2624 files~583,675 lines of source · 101.5 MB · 29 CI workflows · Dockerfile · tests dir

Answers from our run

Does skypilot build from source?

Dependencies installed in 171 seconds (381 packages), and the build succeeded in 11 seconds. We cloned commit a72a881 into a clean Debian container with 3 CPUs and no project-specific setup.

Do skypilot's tests pass?

No. The project's own test command exited unsuccessfully after 16 seconds, and its output did not expose an individual test count.

Does skypilot have known vulnerabilities in its dependencies?

pip-audit flagged 12 known advisories in the dependency tree at the time of our run.

Who should not use skypilot?

Teams seeking evidence here of successful GPU launches, provider compatibility, or cost savings: our recorded run measured none of those.

What are the alternatives to skypilot?

dstack, Ray. SkyPilot merits evaluation when a shared task interface across different infrastructure is the requirement.

Setup2/5The October 1, 2026 run at a72a881 recorded a 171-second install step and an 11-second build step, but no installed-CLI check. Test configuration import stopped on UID lookup; provider setup and launches remain unverified.
Docs3/5The installation extract gives uv/pip setup, pip requirements, credentials, quotas, and remote-server instructions. Python minimum versions differ between instruction paths, and this review did not validate those paths end to end.
Community2/5Supplied issue histories show contributor discussion, maintainer guidance, and closure of issue 10713. They do not establish adoption, response-time guarantees, or support quality across the project.
Maturity3/5The README and v0.13.0 notes document scheduling, team deployment, diagnostics, and lifecycle features. Our run did not validate workloads or reliability, and supplied open reports warrant revision-specific checks.

Discussed on

  1. hnSkyPilot: One system to use and manage all AI compute (K8s, 20 clouds, Slurm)15 points
  2. hnChat with your PDFs – Self-hosted LocalGPT on any cloud6 points
  3. hnGuide: Finetune Llama 3.1 on your infra5 points
  4. hnShow HN: Run LLaMA LLM chatbots on any cloud with one click4 points
  5. hnRun Llama2 in your cloud, completely privately3 points

Who it’s for

AI teams evaluating a common task interface across clouds, Kubernetes, and Slurm.
Infrastructure teams prepared to operate a shared API server and configure access to their compute.
Operators who want resource requirements, code sync, setup, and execution described together.
Teams willing to manage credentials, quotas, networking, and bills in their own accounts.

Who it’s NOT for

Teams seeking evidence here of successful GPU launches, provider compatibility, or cost savings: our recorded run measured none of those.
Teams unwilling to manage infrastructure access and account quotas, even when using a common launch interface.
Container users who need the supplied test command to work unchanged with an unresolved numeric UID: our run stopped during test configuration loading because UID 1000 had no passwd entry.
Teams that intend to pass secrets through --env while requiring those values to be redacted from stored request records. Issue 10741 reports that behavior; a maintainer says users should use --secret.

Setup reality

In our recorded run on October 1, 2026, at commit a72a88173f8ac438e9ef21d88549f6fdc43ca41e, the harness install step exited 0 after 170.609 seconds, rounded to 171 seconds. It recorded 381 dependencies and a 1,681 MB dependency footprint. The available install tail identifies Python 3.12.12 but does not demonstrate an installed sky executable check.

The harness build step exited 0 after 11.274 seconds, rounded to 11 seconds. Its supplied tail contains Python syntax warnings; the record does not identify the full build command or establish wheel creation. The test command exited 4 after 15.739 seconds, rounded to 16 seconds, while loading tests/conftest.py. Importing SkyPilot reached getpass.getuser(), then failed because getpwuid() could not resolve UID 1000. No passing-test count or completed workload is established.

Pip-audit recorded 12 advisories across four packages. Audit coverage, package names, severities, and exploitability are not supplied. The 381-dependency count is not an audited-package count.

The installation guide supports uv and pip. Its uv instructions include pip because SkyPilot needs it to build wheels for remote setup. Local operation also needs the selected provider dependencies and credentials. A client can instead connect to a team API server with sky api login, avoiding local provider dependencies. That server still needs configuration and operation.

Portable tasks still need configured infrastructure

SkyPilot can give a team one way to describe a job while leaving infrastructure ownership with that team. The README describes YAML or Python tasks containing resource requirements, data to sync, setup commands, and execution commands. It lists Kubernetes, Slurm, and 20+ clouds behind that interface.

The same README describes provisioning with capacity failover, workdir sync, dependency setup, log streaming, queueing, and automatic cleanup of idle resources. These are project claims, not capabilities demonstrated by our local run. They make SkyPilot relevant when infrastructure placement changes often, but do not establish compatibility with every workload or savings over an existing deployment.

SkyPilot is bring-your-own-cloud: resources launch within your accounts, VPCs, and clusters. Credentials, quotas, network access, and billing therefore remain part of the adoption decision. A stable single-cluster team should first identify which of SkyPilot's scheduling, development, or management features it needs. Portability alone does not settle that choice.

What our recorded run establishes

Our stored run began on October 1, 2026, at 03:24:56.993 UTC and used commit a72a88173f8ac438e9ef21d88549f6fdc43ca41e. The recorded configuration was 3 CPUs, 8 GB of memory, and image ghcr.io/astral-sh/uv:python3.12-bookworm. The install output identifies Python 3.12.12.

Recorded step Observation Limit
Install Exit 0 after 170.609 seconds, rounded to 171 seconds The available tail does not verify an installed sky executable or identify dependency-download time separately.
Build Exit 0 after 11.274 seconds, rounded to 11 seconds The full command is absent; its tail contains Python syntax warnings. Wheel creation is not established.
Tests Exit 4 after 15.739 seconds, rounded to 16 seconds The command stopped while importing test configuration; no completed suite or passing-test count is established.
Dependencies 381 recorded dependencies; 1,681 MB footprint This is the recorded environment, not a minimum installation size.
Audit Pip-audit recorded 12 advisories across four packages Coverage, identities, severities, and exploitability are unknown.

The test traceback runs from tests/conftest.py through an import of sky into user-hash generation. Python's getpass.getuser() calls pwd.getpwuid(os.getuid()) and raises KeyError: 'getpwuid(): uid not found: 1000'. This is an observed user-lookup failure during configuration loading. It does not show failing workload assertions, a cloud integration failure, or how the suite behaves after user lookup succeeds.

The repository measurement recorded 2,624 files, about 583,675 lines of source, and 101.5 MB, plus 29 CI workflow files, a Dockerfile, and a tests directory. Those signals describe the checkout; they do not prove test coverage or operational reliability. No new execution, GPU workload, provider launch, or savings benchmark was performed for this correction.

From local setup to a useful launch

The installation guide supports uv and pip. It requires pip inside the uv environment for remote wheel building. Its stable uv instructions specify Python 3.9 through 3.13, while some pip and other examples specify 3.7 through 3.13. Check the instructions and provider requirements for the route you choose rather than treating every example as the same compatibility statement.

For local operation, install the dependencies for the providers you plan to use and run sky check to inspect credential configuration. The guide warns that first-time cloud accounts may have zero or low instance quotas, especially for GPUs. The README's example requests eight A100 GPUs and explicitly requires access to those instances.

For a team deployment, sky api login connects a client to a remote API server without local provider dependencies. This changes where configuration lives; it does not remove the need to operate the server and configure its infrastructure access. The next useful evaluation is a small representative launch, including setup, logs, and cleanup, on each intended target. Our run did not complete that evaluation.

What v0.13.0 adds to evaluate

The supplied v0.13.0 release notes describe Sky Batch, Hugging Face storage, lifecycle hooks, sky debug-dump, per-instance hourly cost caps, and GKE Autopilot support. Sky Batch distributes data processing across worker pools and reuses workers between jobs. Lifecycle hooks run custom logic around autostop, preemption, and shutdown events.

The hourly cost cap limits which instance prices are considered during optimization. It is not evidence of a total spending limit or measured savings. The debug-dump command collects logs, state, and configuration into a zip, and the release notes say tasks automatically receive API-server credentials by default. Review those behaviors when deciding who can submit tasks and access diagnostics. Our run did not exercise these release features or establish that its checkout was the v0.13.0 release artifact.

Issue status matters as much as the reproduction

Issue 10713 reported that sky status -o json printed prose before its JSON payload for an absent cluster. The supplied issue history shows it closed as completed on October 7, 2026, referencing commit 4a68bc1 and PR 10763. Our October 1 run predates that closure and did not test this command. The closure supports updating the issue's status; it does not establish that every JSON-emitting command is clean or that the fix is included in a particular release.

Issue 10741 remains open in the supplied extract. It reports that credentials passed through --env persist in plaintext request records, while construction-time redaction handles --secret. On September 25, a maintainer said this was intended because users should use --secret for secrets. Treat this as reported behavior with maintainer guidance, not a security finding from our sandbox.

Issue 10742 also remains open in the supplied extract. Its author reports that distinct cluster names can normalize to the same cloud name and share resources, with a reproduction attributed to master at 6cf931f. We did not reproduce it. Check naming behavior on the revision you intend to deploy, particularly if automation generates names that differ only by case, dots, underscores, or hyphens.

Compare orchestration with orchestration

dstack is the direct orchestration comparison supported by the supplied sources. Its README describes GPU provisioning across clouds, Kubernetes, and on-prem clusters, with a server required before CLI or API use. Compare both tools using the same intended infrastructure and representative workload; this review contains no dstack measurements.

Ray answers a different question. Its README describes a distributed runtime and AI libraries, including tasks and actors for distributed execution. Consider Ray when application execution needs to scale. It can be a complement to an infrastructure orchestration layer, so treating it as a smaller replacement for SkyPilot would obscure the decision.

How to read the scores

These are editorial evidence ratings, not benchmark results. The shared scale is 1 for substantial documented obstacles, 2 for partial support with major unresolved checks, 3 for a documented path with meaningful limits, 4 for broad evidence including relevant validation, and 5 for comprehensive evidence across the intended scope. Setup considers prerequisites and recorded execution; docs considers actionable guidance and consistency; community considers visible support and contribution evidence; maturity considers documented operating features and validated behavior.

The community score reflects limited evidence in this review, not a measurement of adoption or support quality. Stars, release features, and the presence of CI cannot establish production reliability. Choose SkyPilot for a requirement you can name, then validate that requirement on your infrastructure.

Alternatives

ProjectWhat it isPick it when
dstackA GPU provisioning and orchestration control plane for clouds, Kubernetes, and on-prem clusters.Evaluate it as a direct orchestration alternative when you want a server-and-CLI workflow for provisioning compute and submitting development, training, or inference workloads. Its README requires a server before CLI or API use.
Ray gh↗A distributed runtime and AI libraries for scaling Python applications.Choose it when the problem is distributing application execution through tasks, actors, or AI libraries. It addresses a different layer from SkyPilot's infrastructure interface and may complement it rather than replace it.

Sources

  1. SkyPilot README
  2. SkyPilot installation guide
  3. SkyPilot v0.13.0 release notes
  4. Issue 10713: JSON output contains prose
  5. Issue 10741: env credentials in request records
  6. Issue 10742: cloud cluster name collisions

More automation reviews

hummingbot · spotify-downloader · bbot · switch · Horizon · ai-goofish-monitor · the whole board →