PyPIAI / MLupdated 05 Aug 2026

mlflow

Item: mlflow
Rating: 4.5
Author: Keyoor

MLflow is the most widely deployed open source platform for tracking machine learning work: it logs experiments (params, metrics, artifacts, models), stores models in a registry with versioning and aliases, and serves a web UI to compare runs. Since the 3.x line it also went hard on LLM tooling: tracing built on OpenTelemetry, LLM evaluation, a prompt registry, and an AI gateway. It runs anywhere Python runs, from a laptop with a local folder to a Databricks-hosted backend.

mlflow · v3.15.1 · 9.7M/wk · ★ 27,376 · pushed 05 Aug 2026 · mlflow/mlflow · docs · reviewed by Keyoor

Verdict

MLflow earned its position: it is the default self-hosted answer for experiment tracking and model registry, with serious corporate backing and a real migration path to Databricks. Accept that you are adopting a platform, not a library; if your need is one metric chart or only LLM traces, smaller tools cost less to run.

API stability4/5Core tracking APIs (start_run, log_metric, log_model) have been stable for years and 3.x kept them mostly intact, though log_model moved from artifact_path to name and the newer GenAI modules still shift between releases

Docs4/5Docs are extensive with quickstarts, a live demo, and per-framework tracing guides; the sheer surface area means finding the current blessed way (classic ML vs GenAI variants of the same feature) takes effort

Maintenance5/5Pushed the day of this review, backed by Databricks with a large contributor base and a fast release cadence; about 2097 open issues and PRs reflects heavy traffic, not neglect

Ecosystem5/5Autologging for the major ML frameworks, one-line tracing for 60+ LLM frameworks per the README, OpenTelemetry compatibility, and native integration in Databricks, SageMaker, and Azure ML deployment paths

Use it if

You are running many training experiments and need params, metrics, and artifacts recorded automatically; mlflow.autolog() covers sklearn, xgboost, pytorch, and more with one line
You need a model registry with staged rollout semantics: version numbers, aliases like champion/challenger, and a load-by-alias URI that decouples training from serving
You want tracing for LLM apps without a SaaS contract: one-line autolog instrumentation for openai, langchain, and dozens of frameworks, stored on infrastructure you control
Your team is on Databricks, where MLflow is the native tracking layer and the hosted version removes the server-management work

Skip it if

You just want quick metric curves for a solo project: the full install drags in Flask, gunicorn, docker, alembic, pandas, pyarrow, and matplotlib, and running a tracking server is real overhead next to tensorboard or a CSV
You want a polished hosted experience without running servers: the open source server has minimal access control and no real multi-tenant story, which is where wandb or the paid Databricks platform come in
You only need LLM observability: purpose-built tools like langfuse are lighter to deploy and their UIs are built around traces, not around a decade of ML-experiment concepts
You dislike fast-moving surfaces: the 3.x GenAI APIs (tracing, evaluation, prompt registry) reshuffle names and modules release to release, and blog posts from the 2.x era routinely show APIs that moved

Setup reality

pip install mlflow is a heavyweight tree (Flask, gunicorn, docker, alembic, graphene, pandas, pyarrow, matplotlib); use mlflow-skinny in clients and containers where you only log. With no configuration, runs land in a local ./mlruns folder, which works until two machines are involved. A real setup means running mlflow server with a database backend (sqlite for one box, postgres for a team) and an artifact store like S3, and note the server ships with essentially no authentication out of the box. Migrations between mlflow versions on the same database go through alembic and deserve a backup first.

Patterns

Log params and metrics to a runtracking-quickstart

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("churn-model")

with mlflow.start_run(run_name="baseline"):
    mlflow.log_param("max_depth", 6)
    mlflow.log_metric("auc", 0.91)
    mlflow.log_metric("loss", 0.4, step=1)

Without set_tracking_uri everything writes to a local ./mlruns folder relative to the working directory, which is why runs seem to vanish when you launch from a different folder.

Automatic logging for a training frameworkautolog-sklearn

import mlflow
from sklearn.ensemble import RandomForestClassifier

mlflow.autolog()
with mlflow.start_run():
    model = RandomForestClassifier().fit(X_train, y_train)
    model.score(X_test, y_test)  # logged as a metric

Call mlflow.autolog() before importing or at least before fitting. It logs params, metrics, and the model itself; disable the model artifact with log_models=False if artifacts are large.

Log a model and load it backlog-and-load-model

import mlflow

with mlflow.start_run():
    info = mlflow.sklearn.log_model(model, name="model")

loaded = mlflow.pyfunc.load_model(info.model_uri)
preds = loaded.predict(X_test)

In MLflow 3 the parameter is name, not the old artifact_path, and log_model returns ModelInfo whose model_uri you should keep. pyfunc.load_model gives a framework-agnostic predict().

One-line tracing for LLM callsllm-tracing-autolog

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("agent-debug")
mlflow.openai.autolog()

# every OpenAI client call now produces a trace in the UI

There is one autolog per integration (mlflow.openai, mlflow.langchain, mlflow.anthropic, and so on). Traces appear under the active experiment, so set the experiment first or they pile into Default.

Trace your own functions with a decoratortrace-custom-function

import mlflow

@mlflow.trace
def retrieve(query: str) -> list[str]:
    return search_index(query)

@mlflow.trace
def answer(query: str) -> str:
    docs = retrieve(query)
    return llm_call(query, docs)

Nested decorated calls become child spans automatically, which is how you get a tree instead of a flat list. Inputs and outputs are captured, so do not decorate functions that take secrets.

Register a model and deploy by aliasmodel-registry-alias

import mlflow
from mlflow import MlflowClient

mv = mlflow.register_model(info.model_uri, "churn-model")
client = MlflowClient()
client.set_registered_model_alias("churn-model", "champion", mv.version)

prod = mlflow.pyfunc.load_model("models:/churn-model@champion")

Aliases replaced the old Staging/Production stages, which are deprecated. Serving code that loads models:/name@champion never changes; promoting a model is just moving the alias.

Query past runs as a DataFramesearch-runs

import mlflow

df = mlflow.search_runs(
    experiment_names=["churn-model"],
    filter_string="metrics.auc > 0.9",
    order_by=["metrics.auc DESC"],
)
print(df[["run_id", "metrics.auc", "params.max_depth"]].head())

The filter string is MLflow's own mini-language, not SQL: metrics., params., and tags. prefixes are required, and string values need single quotes.

Start a tracking server with a real backendrun-tracking-server

# one box: sqlite backend, local artifacts
mlflow server \
  --backend-store-uri sqlite:///mlflow.db \
  --artifacts-destination ./mlartifacts \
  --host 0.0.0.0 --port 5000

The default file-based store cannot back the model registry properly; use a database URI. Anything multi-user also needs artifact storage every client can reach, usually S3 or equivalent.

Attach files, figures, and dicts to a runlog-artifacts

import mlflow

with mlflow.start_run():
    mlflow.log_dict({"features": cols}, "features.json")
    mlflow.log_figure(fig, "roc_curve.png")
    mlflow.log_artifact("data/holdout.parquet")

log_artifact uploads an existing file; log_dict and log_figure serialize for you. Artifacts go to the artifact store, so a misconfigured one fails here first, not at log_metric time.

Package custom inference logic as a modelcustom-pyfunc-model

import mlflow

class Wrapper(mlflow.pyfunc.PythonModel):
    def load_context(self, context):
        self.model = load(context.artifacts["weights"])

    def predict(self, context, model_input):
        return self.model.run(preprocess(model_input))

with mlflow.start_run():
    mlflow.pyfunc.log_model(name="model", python_model=Wrapper(),
                            artifacts={"weights": "weights.bin"})

pyfunc is the escape hatch when your inference is not a plain framework model. Pin dependencies via pip_requirements, or the auto-inferred environment may not reproduce at serving time.

Alternatives

Package	Registry	Pick it when
wandb	PyPI	You want hosted experiment tracking with a slicker UI and zero server ops
tensorboard	PyPI	Solo work where metric curves are enough and you want nothing to deploy
clearml	PyPI	You want experiment tracking plus orchestration and data management in one self-hostable stack
langfuse	PyPI	You only need LLM tracing and evaluation, not classic ML experiment tracking