mrkeyoor.com_
Tue 01 Sept 15:07 UTC
Automationevaluationupdated 31 Aug 2026

prefect review

Prefect is a Python workflow system that adds scheduling, retries, caching, state tracking, and a web dashboard to ordinary functions. It helps data teams run scripts repeatedly, see what failed, and move the same workflow from a laptop to workers without rewriting it as a separate configuration language.

+13 / 1dstars / 7d
Verdict

Our Prefect run installed 128 packages in 53 seconds and built in 12 seconds, but tests stopped before collection because numpy was missing. Prefect is a strong choice when Python workflows have outgrown cron and a team will use the run history, retries, parameters, and worker model. Keep cron for quiet single-host scripts, and test child-deployment failure plus API-outage behavior before trusting Prefect with a critical chain.

We ran it

Lab card: what happened when we ran prefectScreenshot of prefect (prefect.io)
Install✓ · 53s128 packages · 148 MB
Build✓ · 12s
Tests✗ · 30sran, no count parsed
Known vulns0(pip-audit)
Repo5366 files~913,182 lines of source · 87.3 MB · 43 CI workflows · Dockerfile · tests dir

Answers from our run

Does prefect build from source?

Dependencies installed in 53 seconds (128 packages), and the build succeeded in 12 seconds. We cloned commit 33ccf3d into a clean Debian container with 3 CPUs and no project-specific setup.

Do prefect's tests pass?

The test command failed in our container, and its output did not report a pass or fail count.

Does prefect have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use prefect?

Teams whose workflows are not authored in Python: the README presents Prefect as a Python data-pipeline framework.

What are the alternatives to prefect?

Apache Airflow, Dagster, Temporal Python SDK. Our Prefect run installed 128 packages in 53 seconds and built in 12 seconds, but tests stopped before collection because numpy was missing.

Setup3/5Quick local start; production adds a server, deployments, and workers
Docs5/5Clear quickstart plus deployment, self-hosting, and API references
Community5/523,732 stars with same-day pushes, issues, and pull requests
Maturity4/5Eight years of releases, though our contributor suite did not start

Discussed on

  1. hnShow HN: Prefect – A workflow engine designed for modern data engineering59 points
  2. hnThe Prefect Workflow Engine Is Now Open Source18 points
  3. hnPerfect – Data workflow automation with Python6 points
  4. hnShow HN: Prefab – A generative UI framework for Python5 points
  5. hnPrefect: Data Engineering Workflow System4 points

Who it’s for

Python data teams whose scheduled jobs need retries, parameters, logs, and visible run history.
Developers who want to keep branching and task logic in Python instead of defining a static scheduler file.
Platform teams choosing between a self-hosted control plane and Prefect Cloud.
Organizations prepared to run workers, deployments, and the orchestration service as production infrastructure.

Who it’s NOT for

Teams whose workflows are not authored in Python: the README presents Prefect as a Python data-pipeline framework.
A small script that cron already runs reliably: Prefect adds a server, deployment process, workers, state, and an operating surface.
Contributors expecting the full test suite after the base install: our run stopped while loading tests/conftest.py because numpy was unavailable.
Parent flows that must automatically fail when a separately deployed child fails: open issue 19479 says run_deployment() can leave the parent successful after a child runtime error.
Environments that need documented client retry settings to cover an unreachable API at flow startup: issue 20462 reports that the flow can fail before orchestration retries exist.
Jobs that only call a remote Prefect API: the README recommends the lighter prefect-client package for that case.

Setup reality

Our sandbox installed commit 33ccf3d in 53 seconds, adding 128 packages and using 148 MB. The build passed in 12 seconds. Tests exited with code 4 after 30 seconds while loading tests/conftest.py because numpy could not be imported. Pip-audit found 0 known vulnerabilities.

Prefect needs Python 3.10 or newer. A local trial can run a decorated function, start the server, and open the UI on port 4200. Scheduled deployments also need a process that serves or polls for work. Prefect Cloud replaces the self-hosted control plane but adds an account and workspace configuration.

The measured checkout had 5,366 files, about 913,182 source lines, 43 CI workflows, a Dockerfile, and a tests directory. Production setup depends on where code runs, how workers obtain it, which secrets they need, and whether the orchestration API is self-hosted or managed.

A Python decorator adds state, retries, and a run record

Prefect turns a normal function into a flow with @flow, then lets selected operations become tasks with @task. The code remains Python, including loops and branches, while the orchestration layer records runs and can apply retries, caching, parameters, schedules, and event-triggered actions. That is a comfortable migration path for a data script whose business logic already works but whose cron entry gives poor answers after a failure.

The quickstart is genuinely short. Install the package, decorate the functions, and execute the file. prefect server start opens a local dashboard on port 4200 so the run can be inspected. Turning the same flow into a deployment adds a named serving process and a schedule. That second step is where Prefect stops being a library inside one process and becomes infrastructure that must stay reachable.

A deployment adds a control plane and somewhere to run work

A production flow needs more than its decorator. The Prefect API stores orchestration state, while a worker or serving process obtains scheduled runs and starts code in the chosen environment. Users can operate the server themselves or send run metadata to Prefect Cloud. Either path still leaves the workflow's compute, packages, network access, and secrets under the team's control.

This separation is useful when jobs move among local processes, containers, or Kubernetes. It also introduces failure modes that a single python job.py never had. The API must be reachable, workers need matching configuration, deployments must point at runnable code, and version changes touch several components. The lighter prefect-client package exists for short-lived processes that only need to communicate with a remote server and do not need the full SDK.

What happened when we ran it

Our sandbox installed commit 33ccf3d in 53 seconds, adding 128 packages and occupying 148 MB. The build finished in 12 seconds. Pip-audit reported 0 known vulnerabilities in the installed Python packages. The checkout itself was much larger: 5,366 files, roughly 913,182 source lines, and 87.3 MB before dependencies. This is a platform codebase, not a small scheduling helper.

The test command failed with exit code 4 after 30 seconds. Pytest was loading tests/conftest.py when import numpy raised ModuleNotFoundError. No tests completed, so there is no passed or failed test count to report. The log does not show why numpy was absent. It shows that the complete suite did not start in our fresh Python 3.12 environment after the measured install.

The repository scan found a tests directory, Dockerfile, and 43 CI workflow files. The project plainly has extensive internal automation. That does not turn our failed local suite into a pass, and it matters to contributors who expect a documented environment to reproduce CI. Application users can still note that installation and build both succeeded, while maintaining a separate acceptance test for their own flows, server, storage, and worker type.

Child deployment failures do not automatically fail the parent

Open issue 19479 describes run_deployment() submitting a child flow whose runtime error does not propagate to the parent. The parent can finish successfully because submission worked, even while the child later shows as failed. The requested change is an opt-in setting that waits and raises on a failed, crashed, or cancelled child. Until that behavior exists, nested deployment users need to inspect the returned state and define failure propagation themselves.

This detail changes alerting. A green parent cannot be the only success condition for a batch that starts separate child deployments. Test a real child exception, confirm what appears in the UI, and make the outer workflow or monitoring system wait for the result you care about. Prefect supports branching and dependencies, but deployment submission creates an asynchronous boundary with semantics that deserve an explicit integration test.

Flow retries begin too late for one reported API outage case

Issue 20462 reports a related boundary at startup. In the submitter's reproduction, an unreachable Prefect API caused a RuntimeError before the flow run was created. Flow-level retries therefore never began, and the client's configured retry settings did not provide the expected several-minute connection window. The issue asks for a startup retry setting or clearer documentation around the initial handshake.

That report does not prove identical behavior for every Prefect 3 setup, but it gives operators a useful failure drill. Stop the API, launch a scheduled and an ad hoc flow, then measure what the process and worker do. A retry policy inside the flow cannot recover work that orchestration never registered. External process supervision and API health checks may still be needed around the framework's own availability boundary.

Version 3.8.4 is active inside a large issue queue

GitHub recorded 23,732 stars and 866 open issues and pull requests combined on August 31, 2026. The last push was that same day, and newly opened reports were already receiving linked fixes. Release 3.8.4 arrived on August 25 with scheduling changes, worker fixes, UI corrections, dependency updates, and test maintenance. The dated activity supports a healthy project even though the open queue is large.

Prefect has been developed since 2018 and now spans an SDK, server, UI, workers, deployment tooling, and integrations. That breadth is the attraction for a growing Python data platform and the tax for a one-file job. Adopt it after testing one representative deployment end to end, including a child failure and an API restart. Our 30-second test stop is a contributor-setup warning; the operational decision rests on whether the control plane removes more toil than it adds.

Alternatives

ProjectWhat it isPick it when
Apache Airflow gh↗A widely deployed scheduler for authoring and monitoring batch workflows as Python DAGs.pick this instead when an established batch-scheduling ecosystem and Airflow operator compatibility outweigh Prefect's lighter function-first style.
DagsterA data orchestrator organized around observable data assets and their dependencies.pick this instead when asset lineage, partitions, and data-product modeling should drive the workflow design.
Temporal Python SDKA Python SDK for durable application workflows backed by a Temporal service.pick this instead when long-running business processes and durable application state matter more than data-pipeline ergonomics.

What people are saying

  1. [github-trending] PrefectHQ/prefect

Sources

  1. Prefect README
  2. Prefect 3.8.4 release
  3. Issue 19479: child deployment failure propagation
  4. Issue 20462: API startup retry behavior
  5. Prefect repository facts
  6. Apache Airflow repository
  7. Dagster repository
  8. Temporal Python SDK repository

More automation reviews

OpenCLI · web-access · Karabiner-Elements · WiiUDownloader · Telegram-Media-Downloader · rewards-farmer · the whole board →