One server now covers experiments and agent traces
MLflow began as experiment tracking, and that foundation remains: log parameters and metrics, compare runs, register models, evaluate them, and connect deployment targets. The current README puts agents and language models beside that older workflow. It adds OpenTelemetry-based tracing, prompt versions, evaluation, monitoring, and an OpenAI-compatible gateway. That breadth can consolidate a real platform. It can also turn a simple tracing need into a much larger adoption.
The basic local path is pleasantly short. uvx mlflow server starts the service, an application points at http://localhost:5000, and an autologging integration records calls. The README also offers an agent setup command that installs skills and launches a supported coding agent. Those commands prove that first contact is easy. They do not settle storage, access control, artifact retention, or provider policy for a shared deployment.
More than 60 integrations increase both coverage and surface area
The integration list spans agent frameworks, model providers, gateways, and tools across Python, TypeScript, and Java. OpenTelemetry support lets other languages send data without a native MLflow integration. MCP appears as a supported protocol, and Claude Code is listed among tool integrations. A team using several frameworks can therefore keep traces and evaluations in one interface instead of teaching each group a different system.
Automatic instrumentation still needs verification against the calls your application makes. A provider SDK update, streamed response, async path, or unusual framework wrapper can produce a trace different from the one expected. The recently active queue included a bug about empty Responses Agent output streams and a matching fix. That is normal project work, but it is a useful reminder to test trace completeness before using the data for incident review or cost reporting.
What happened when we ran it
Our sandbox installed MLflow in 49 seconds, adding 116 packages and consuming 601 MB. The build succeeded in 11 seconds. Pip-audit found 0 known vulnerabilities in the installed Python environment. commit 10ca0a8 occupied 384.9 MB before dependencies and contained 7,791 files with about 1,479,658 lines of source.
The test command failed with exit code 1 after 242 seconds. Pytest reported 273 passed, 8 skipped, and 200 collection or setup errors, then stopped at its failure cap. Its summary says 12,927 items were collected with 187 errors during collection. Those counts overlap different phases of the same run, so they should not be added into a fictional total. No assertion failures were reported in the supplied summary.
The log tail shows several agent CLI tests exceeding pytest-timeout's 120-second limit. Named cases covered launching opencode, finding a git root, behavior when git is absent, fallback outside a repository, and bundled skills. The log does not identify why they timed out. Our finding is limited: this checkout did not complete its suite in the fresh Debian container, even though 273 tests had passed.
A 601 MB install precedes databases and artifacts
Local evaluation can use the default server and filesystem, but a team service needs durable state. Tracking metadata belongs in a backed-up database, while model artifacts and evaluation outputs need storage with an explicit lifecycle. Authentication and network exposure matter because traces may include prompts, tool inputs, outputs, errors, and usage details. The AI Gateway additionally holds provider credentials and applies routing rules.
The repository provides no root Dockerfile in our scan, despite 64 CI workflow files and a tests directory. That does not mean MLflow cannot run in containers or on cloud services. It means an operator should use the project's documented deployment path and inspect the exact published image rather than infer container behavior from this checkout. The installed 601 MB also excludes whatever database, object store, and model artifacts the deployment accumulates.
The gateway is useful only with owned policy
MLflow's gateway offers an OpenAI-compatible interface, provider routing, rate limits, fallbacks, credential management, guardrails, and traffic splitting. Centralizing those controls helps when several applications call several providers. It also creates a service whose outage or bad rule can affect every caller. Teams should define who can change routes, how credentials rotate, which logs retain content, and how direct provider access is prevented.
Evaluation has similar limits. Built-in metrics and model judges make repeatable checks easier, but a judge is still a model with cost and behavior that must be pinned and reviewed. Release v3.15.2, published August 26, added immutable evaluation dataset versions and a scorer ensemble primitive, alongside fixes. Dataset immutability is the kind of mundane control that makes comparisons more defensible than a dashboard alone.
August 27 activity supports a serious trial
GitHub recorded the last push on August 27, 2026, one day after v3.15.2. The repository had 27,688 stars and 2,081 combined open issues and pull requests. The open count is not a defect count. Together with current releases and same-day pull requests, it shows active maintenance around a large scope. Apache-2.0 covers the repository.
MLflow earns a trial when a team already has both model-lifecycle and agent-observability problems. Start with one application and validate that traces, metrics, artifacts, and permissions match the operational promise. Our successful 49-second install and 11-second build lower the cost of that trial. The 200 setup errors make a clean local test strategy part of adoption, not cleanup to postpone until after deployment.