mrkeyoor.com_
Fri 04 Sept 10:34 UTC
LLM Toolsevaluationupdated 04 Sept 2026

LlamaFactory review

LlamaFactory is a Python toolkit for adapting more than 100 large language and multimodal model families to your own data. It brings command-line tools, a browser interface, multiple training methods, experiment tracking, and serving options into one project, reducing the amount of separate training plumbing a team must assemble.

trackingstars / 7d
Verdict

Our build completed in 35 seconds, but tests ended with 3 collection/setup errors after 318 tests passed. LlamaFactory is a compelling shortlist choice when breadth matters: it combines many model families, training methods, interfaces, and inference paths in one active project. Use it only after reproducing the suite in your own accelerator environment and reviewing the 45 dependency vulnerabilities, not as a blind install on a production trainer.

We ran it

Install✓ · 212s161 packages · 6319 MB
Build✓ · 35s
Tests✗ · 798s318 passed · 0 failed · 23 skipped · 3 errors of 321 (pytest)
Known vulns45(pip-audit)
Repo596 files~63,637 lines of source · 13.8 MB · 8 CI workflows · tests dir

Answers from our run

Does LlamaFactory build from source?

Dependencies installed in 212 seconds (161 packages), and the build succeeded in 35 seconds. We cloned commit 4451765 into a clean Debian container with 3 CPUs and no project-specific setup.

Do LlamaFactory's tests pass?

Yes: 318 of 321 passed when we ran the project's own test command (pytest), with 3 collection errors. Some failures need services or credentials a bare container does not have.

Does LlamaFactory have known vulnerabilities in its dependencies?

pip-audit flagged 45 known advisories in the dependency tree at the time of our run.

Who should not use LlamaFactory?

Beginners expecting a tiny one-command install, because our environment grew to 161 packages and 6,319 MB

What are the alternatives to LlamaFactory?

Axolotl, TRL, Unsloth. Our build completed in 35 seconds, but tests ended with 3 collection/setup errors after 318 tests passed.

Setup2/5161 packages, 6,319 MB, and test collection errors
Docs4/5Broad guides and examples, though official docs are marked WIP
Community4/574,570 stars and same-day activity, but 1,139 open issues
Maturity3/5Large passing suite offset by setup errors and audit findings

Who it’s for

ML engineers who need one interface for full tuning, LoRA, QLoRA, preference tuning, or reward modeling
Research teams comparing training approaches across several supported model families
Practitioners who value both command-line workflows and a Gradio-based interface
Teams that already understand GPU environments and can audit a large Python dependency tree

Who it’s NOT for

Beginners expecting a tiny one-command install, because our environment grew to 161 packages and 6,319 MB
Security-sensitive teams unwilling to triage dependency findings, because pip-audit reported 45 known vulnerabilities
Teams requiring an entirely clean upstream test run before adoption, because collection/setup ended with 3 errors in our sandbox
CPU-only users expecting to validate real training performance from this review, because our run did not benchmark training

Setup reality

Our run installed successfully in 212 seconds, adding 161 packages and occupying 6,319 MB, and the build succeeded in 35 seconds. The test command then exited 1 after 798 seconds: pytest reported 318 passed, 23 skipped, 3 expected failures, 4 unexpected passes, and 3 collection/setup errors. That is materially heavier and rougher than the README's easy-start framing. The project is approachable by large-model training standards, but it still expects comfort with Python environments, accelerator-aware dependencies, and diagnosing setup failures.

It turns a fragmented training toolchain into one project

LlamaFactory tackles fragmented large-model training. It gathers choices under a Python package with a zero-code CLI and a Gradio-based LlamaBoard. Its README claims support for more than 100 models, and the supplied repository snapshot has 596 files and about 63,637 lines of source. This is a substantial framework, not a thin script collection.

The breadth is the main reason to consider it. Listed model families include LLaMA, LLaVA, Mistral, Mixtral-MoE, Qwen3, Qwen3-VL, DeepSeek, Gemma, GLM, and Phi. Training choices cover continued pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, and ORPO. Resource options range from 16-bit full tuning and frozen layers to LoRA and 2, 3, 4, 5, 6, or 8-bit QLoRA.

The best fit is an experienced team that wants breadth

This is most attractive to ML engineers who already know why they would choose full tuning over adapters, or DPO over supervised fine-tuning. Version v0.9.5 and the Apache-2.0 license make it straightforward to evaluate, while the command line remains suitable for repeatable jobs. Researchers also get several monitors, including LlamaBoard, TensorBoard, Wandb, MLflow, and SwanLab.

The README includes multi-turn dialogue, tool use, image understanding, visual grounding, video recognition, and audio understanding. It also lists an OpenAI-style API plus vLLM and SGLang workers for inference. LlamaFactory can cover more of the journey than a training-only library, although the evidence here does not establish production throughput or model quality.

What happened when we ran it

Our run used commit 4451765 in a fresh unprivileged Debian container with 3 CPUs, 8 GB of RAM, Python 3.12, and no secrets. Installation succeeded in 212 seconds, but it pulled in 161 packages and occupied 6,319 MB on disk. The separate build step succeeded in 35 seconds. Those results confirm that the project can be installed and built in a clean environment, while also showing that the dependency footprint is far from lightweight.

Tests were the rough edge. The command ran for 798 seconds and exited with status 1. Pytest reported 318 passed, 23 skipped, 3 expected failures, 4 unexpected passes, 18 warnings, and 3 errors. The errors occurred while collecting or setting up scripts/api_example/test_image.py, scripts/api_example/test_toolcall.py, and tests_v1/plugins/data_plugins/test_converter.py.

That distinction matters: no test assertion was reported as failed, yet the suite still did not complete cleanly. On our box, a warning also said pin_memory was enabled without an accelerator. This was not a GPU training benchmark, and the measurements support no speed or model-quality claims. They do show a large amount of working code alongside environment-sensitive edges that prospective users should reproduce on their own hardware before committing.

Its range is unusually practical, but complexity comes with it

LlamaFactory's strongest design choice is that its options map to real constraints. Users can choose full tuning, frozen layers, LoRA, or quantized adapters, then add named techniques such as FlashAttention-2, Unsloth, Liger Kernel, RoPE scaling, NEFTune, or rsLoRA. The README also advertises Day 0 support for several model families and Day 1 support for others.

The cost of this range showed up clearly in our 6,319 MB environment. Pip-audit found 45 known vulnerabilities across the installed dependencies. That number is not proof that LlamaFactory itself is exploitable, but it is a concrete triage burden for any team putting the stack near private data or a network service. Our repository scan found 8 CI workflow files and a tests directory, both good maintenance signals, but it reported no Dockerfile even though the README includes a Docker build path.

Documentation is extensive in topic coverage, with installation, data preparation, CLI, GUI, Docker, inference, model hubs, and logging all represented. The README labels the official documentation as work in progress, an honest warning for a project with this many combinations. Colab, PAI-DSW, AMD GPU Cloud, ModelScope, and Novita entry points make exploration easier, but also widen the matrix maintainers must keep current.

The project is active, while its support queue is large

The repository had 74,570 stars and 1,139 open issues in the supplied snapshot. Popularity alone does not prove support quality, and a four-digit issue count can make finding a relevant answer noisy. At the same time, the last push was on 2026-09-04, the date of our review, so the codebase is plainly active. The latest listed release was v0.9.5 on 2026-05-30. One release date is not enough to calculate a cadence, but the same-day push prevents that older tag from being read as abandonment.

In a real stack, LlamaFactory belongs between prepared datasets and the application that consumes a trained artifact. Use its training interfaces and monitors for experimentation, then expose results through its CLI, Gradio UI, or OpenAI-style API with a listed vLLM or SGLang worker. Version v0.9.5 offers enough surface to consolidate several tools, but it does not remove the need for dataset governance, dependency review, accelerator provisioning, evaluation criteria, or service monitoring. Those remain the adopting team's responsibility.

Choose it for consolidation, after a local qualification run

LlamaFactory earns a trial when one team expects to move among several supported model families or training methods and wants fewer hand-built integrations. Axolotl is the closest broad alternative, TRL is a sharper choice for post-training algorithm work, and Unsloth may suit users prioritizing a local UI. Our result is encouraging but not clean: 318 tests passed, the build succeeded, and 3 collection/setup errors still stopped the suite. Treat it as capable infrastructure that must pass your own environment gate, not a turnkey appliance.

Alternatives

ProjectWhat it isPick it when
AxolotlA configuration-driven framework for fine-tuning language and multimodal models.Pick this instead when your team prefers YAML-centered pipelines and Axolotl's deployment options.
TRLA focused library for post-training transformer language models with preference and reinforcement-learning methods.Pick this instead when post-training algorithms matter more than an all-in-one GUI and serving surface.
Unsloth gh↗A local interface and toolkit for running and training language and diffusion models.Pick this instead when a local-first UI and its narrower workflow match your hardware and model choices.

What people are saying

  1. [velocity-scout] hiyouga/LlamaFactory

Sources

  1. LlamaFactory GitHub repository
  2. LlamaFactory documentation

More llm tools reviews

verl · skills-hub · aidlc-workflows · notebooklm-py · a2ui · web-llm · the whole board →