mrkeyoor.com_
Wed 16 Sept 00:57 UTC
AI Toolsevaluationupdated 26 Aug 2026

llm review

LLM is a Python command-line tool and library for sending prompts to hosted or local language models through one workflow. It also stores conversations in SQLite, handles attachments and structured output, runs tools, creates embeddings, and grows through provider and feature plugins.

+33stars / 7d
Verdict

Our LLM install used 60 packages and 67 MB, but its tests stopped in 8 seconds because llm_echo was missing, so the source checkout is easier to use than to verify from our measured setup. LLM is the best fit here for an engineer who wants one scriptable command, durable SQLite history, and a broad plugin vocabulary across hosted and local models. Use LiteLLM for a shared application gateway, Ollama for local model serving, or Open WebUI for a team-facing chat interface.

We ran it

Lab card: what happened when we ran llmScreenshot of llm (llm.datasette.io)
Install✓ · 59s60 packages · 67 MB
Build✓ · 8s
Tests✗ · 8sran, no count parsed
Known vulns0(pip-audit)
Repo116 files~36,017 lines of source · 1.9 MB · 4 CI workflows · tests dir

Answers from our run

Does llm build from source?

Dependencies installed in 59 seconds (60 packages), and the build succeeded in 8 seconds. We cloned commit a463c63 into a clean Debian container with 3 CPUs and no project-specific setup.

Do llm's tests pass?

The test command failed in our container, and its output did not report a pass or fail count.

Does llm have known vulnerabilities in its dependencies?

pip-audit found none in the dependency tree at the time of our run.

Who should not use llm?

People who want a polished graphical chat product: the core product is a CLI and Python library.

What are the alternatives to llm?

LiteLLM, Ollama, Open WebUI. Our LLM install used 60 packages and 67 MB, but its tests stopped in 8 seconds because llm_echo was missing, so the source checkout is easier to use than to verify from our measured setup.

Setup4/5Fast 60-package install; providers still need plugins and keys
Docs5/5Deep CLI, Python, plugin, logging, schema, and tool documentation
Community5/512,409 stars, release-day pushes, and busy plugin contributions
Maturity4/5Long feature history, with our test environment missing a fixture plugin

Discussed on

  1. hnLLM-hacker-news: LLM plugin for pulling content from Hacker News92 points

Who it’s for

Developers who want to call several model providers from shell scripts without learning a new CLI each time.
Researchers who want searchable local records of prompts, responses, tools, and token usage.
Python developers building a model or tool plugin around a documented interface.
Data workers extracting schemas or embeddings from files at the command line.

Who it’s NOT for

People who want a polished graphical chat product: the core product is a CLI and Python library.
Teams expecting every provider in the base install: Gemini, Anthropic, Ollama, and many others require separate plugins, credentials, or local model services.
Privacy-sensitive users who will not manage prompt history: logging stores prompts and responses in SQLite unless configured otherwise.
Contributors who need the documented test command to work from the measured install: our suite stopped before collection because llm_echo was missing.
Windows users relying on interactive chat key bindings without testing: issue #1639 reports a pyreadline3 binding error and has an active fix discussion.
Organizations that cannot audit third-party Python plugins before installation: the plugin system is the main route to additional models and tools.

Setup reality

Our sandbox installed 60 packages in 59 seconds, using 67 MB on disk, and built the project in 8 seconds. Tests then exited 4 after 8 seconds before collection: tests/conftest.py imported llm_echo, which was missing. pip-audit reported 0 known vulnerabilities.

The base CLI can call an OpenAI-compatible endpoint, while named providers usually need API keys and plugins such as llm-gemini or llm-anthropic. Local models need their own runtime and weights, for example Ollama plus its LLM plugin. Logging, default models, templates, and keys live in the user configuration directory.

The checkout had 116 files, about 36,017 source lines, 4 CI workflows, a tests directory, and no Dockerfile. Plugins are Python packages with their own dependencies and release cadence, so pin and audit the exact set used in automation.

LLM 0.33 makes model access feel like a Unix command

LLM turns a prompt into a shell command and a Python call. The base tool can use OpenAI models or an arbitrary OpenAI-compatible endpoint; plugins add Anthropic, Gemini, Ollama, and many other hosted or local backends. Inputs can come from arguments, pipes, files, images, audio, or video. Responses can stream to the terminal, continue a conversation, follow a JSON schema, or feed the next command.

This is a good shape for developers who already live in a terminal. A code file can be piped into a system prompt, a template can preserve recurring options, and a shell script can select a model without embedding each provider's SDK. The Python API exposes the same concepts for applications that outgrow shell composition. It is less suitable for colleagues who expect accounts, a shared browser, and an administrator dashboard.

Release 0.33 arrived on 2026-08-22. It moved to the OpenAI Python library 3.x, added per-call keys for embedding commands and APIs, preserved server-side tool results in logs, allowed repeated templates to combine in order, and tightened schema errors. These changes show the project's value and its maintenance burden: provider behavior, reasoning metadata, tools, and attachments keep changing beneath a supposedly simple command.

SQLite history is useful and sensitive

Prompt and response logging is a first-class feature. LLM stores conversations in SQLite, supports search and JSON output, records model and tool information, and can expose the database through Datasette. That makes experiments reproducible enough to compare, revisit, or turn into a dataset. Version 0.33 also distinguishes server-executed tool results from local ones in its logs.

The same database can accumulate source code, pasted documents, personal questions, tool output, and provider payload details. The documentation includes controls for turning logging off, changing the storage directory, and backing up the database. Use them deliberately. A laptop backup or shared home directory can leak more than an API key if years of prompts sit unencrypted inside it.

Keys have several routes. llm keys set stores named credentials, a --key option passes one for a call, and environment variables can supply them. Plugins decide which provider names and model options exist. Automation should choose one route, restrict permissions on the configuration directory, and avoid printing secrets through shell tracing or captured job logs.

What happened when we ran it

Our run at commit a463c63 installed 60 Python packages in 59 seconds and used 67 MB on disk. The project build completed in 8 seconds. The checkout contained 116 files, about 36,017 lines of source, and occupied 1.9 MB, with 4 CI workflow files, a tests directory, and no Dockerfile.

The test suite did not reach test collection. It exited with status 4 after 8 seconds while loading tests/conftest.py; line 5 attempted to import llm_echo, and Python raised ModuleNotFoundError: No module named 'llm_echo'. That is the full cause shown in the supplied tail. We will not guess whether a development extra, fixture package, or harness step was intended to provide it.

pip-audit reported 0 known vulnerabilities in our installed environment. That number applies to the 60 packages our lab installed for a463c63, not to every optional plugin or local model runtime a user may add. Each plugin can expand both dependencies and permissions. Re-run an audit after locking the exact provider and tool packages used in production.

Plugins solve coverage by moving trust outward

The plugin directory is central, covering remote APIs, local models, embedding providers, tools, template loaders, fragments, and extra commands. Developers get documented hooks for synchronous and asynchronous models, schemas, attachments, tool execution, token usage, and provider metadata. A provider can evolve outside the core release rather than waiting for LLM itself.

That separation is productive, though one command such as llm install still installs Python code into the tool's environment. A model plugin may receive prompts and API keys. A tool plugin may run local actions selected through model output. Pin versions, read the package source, limit operating-system permissions, and keep experimental tools away from valuable files. The core Apache-2.0 license does not establish the quality or terms of every plugin.

Provider parity also has limits. Attachments, reasoning, schemas, server-side tools, local tools, and token accounting depend on what a provider and plugin implement. Release 0.33 added attachment validation before a conversation prompt, which is better than discovering the mismatch after a paid call. Still, a script intended to switch models should test its exact feature set against each candidate.

The 676-item queue mixes issues and pull requests

The repository was pushed and released on 2026-08-22, and active work continued in issue and pull-request updates through 2026-08-26. GitHub's 676 open count combines issues and pull requests. Recent activity includes Windows chat key bindings, local-time log display, log deletion, API-key validation, plugin-directory additions, MIME detection, and dependency declarations. This is a busy interface surface, not 676 confirmed defects.

Documentation is unusually deep for a CLI. The generated README links into setup, model providers, tools, schemas, templates, fragments, embeddings, plugins, the Python API, SQLite tables, and every command's help. The density can make the product look larger than it feels. Start with one endpoint and logging choice, then add plugins only when a real workflow needs them.

LLM is a strong personal workbench for an engineer comparing models or turning prompts into scripts. Our build succeeded and the audit found zero known advisories, but the missing llm_echo import prevented any test from running. That contributor rough edge should be fixed or documented; it does not diminish how direct the installed CLI is for ordinary prompt, schema, embedding, and log work.

Alternatives

ProjectWhat it isPick it when
LiteLLM gh↗A Python SDK and proxy that normalizes many model APIs behind an OpenAI-style interface.pick this instead when applications need a shared provider gateway, budgets, and routing more than an interactive CLI and local logbook.
Ollama gh↗A local model runner with a concise CLI and HTTP API for downloaded models.pick this instead when local model management is the whole job and hosted-provider plugins are unnecessary.
Open WebUI gh↗A self-hosted browser interface for local and hosted models with user accounts.pick this instead when non-technical users need a shared graphical chat service.

What people are saying

  1. [github-trending] Shubhamsaboo/awesome-llm-apps
  2. [hackernews] Why your local LLM feels dumber than it is
  3. [hackernews] Clean up Claude 5's token vomit with a separate LLM
  4. [github-trending] mnfst/awesome-free-llm-apis
  5. [hackernews] What happens when an LLM never sees material beyond fifth grade?
  6. [hackernews] Debian has begun voting on the future of AI/LLM contributions

Sources

  1. LLM repository README
  2. LLM repository facts
  3. LLM 0.33 release
  4. Windows chat key binding issue
  5. LLM documentation

More ai tools reviews

LocalMiniDrama · agents-towards-production · Marinara-Engine · AI-Engineering-Coach · sdf-js · TradingAgents-astock · the whole board →