Document chat works without choosing one model vendor
AnythingLLM packages a familiar chat interface around document ingestion, retrieval, agents, and shared workspaces. The operator can connect hosted APIs or local runtimes, then choose separate providers for chat, embeddings, speech, and vector storage. That separation is useful for a company that wants Anthropic for one workspace, a local llama.cpp model for another, and documents kept on its own machine. The product also exposes a developer API and an embeddable chat widget for Docker deployments.
The provider list is long enough to create its own work. The README names OpenAI, Anthropic, Gemini, Bedrock, Ollama, LM Studio, LocalAI, and many OpenAI-compatible services. A model choice also brings a context window, tool-call format, key, rate limit, and attachment support. AnythingLLM gives these systems one control surface, but it cannot make their behavior identical. Issue 6162 is a concrete example: its reporter can switch models yet cannot register 2 local AI endpoints with separate context settings.
Desktop is the easy path; multi-user hosting adds state
The desktop build is aimed at a single person on macOS, Windows, or Linux. That is the cleanest route for private document questions because the app, local files, and optional local model can stay on one workstation. Docker is the relevant edition for multiple users, permissions, the website widget, and a shared service. It needs persistent storage and a backup plan for documents, embeddings, workspace settings, conversations, and stored provider configuration.
Development is more involved than the product pitch. The README tells contributors to run yarn setup, fill several generated environment files, and start the server, frontend, and collector separately. Our measured checkout at commit 72aabbd had 5,733 files, around 267,027 source lines, and 58.4 MB before dependencies. There was no build target in the tested package, so a successful install should not be mistaken for a production bundle check.
What happened when we ran it
Our sandbox installed 288 npm packages in 52 seconds and occupied 49 MB on disk. No build script or target existed, so we skipped that step rather than inventing a build result. Npm audit reported 0 known vulnerabilities across the installed dependency tree. The run used Node 22 on a fresh unprivileged Debian container with 3 CPUs, 8 GB of RAM, and no secrets.
Tests exited 1 after 14 seconds. Jest counted 180 passed and 2 failed out of 182 tests, while the suite line reported 34 failed and 10 passed out of 44. The visible failure at server/__tests__/utils/helpers/modelPricing/index.test.js:255 expected pricing.isCacheStale to be false. The log tail does not explain why many suites were marked failed despite only 2 failed tests, so that mismatch should remain a reported observation rather than a guessed diagnosis.
Retrieval defaults may be too fixed for mixed documents
Uploading files and receiving cited answers is AnythingLLM's central appeal. It supports common document formats, workspaces, drag-and-drop imports, watched documents, and several vector databases. Release 1.16.0, published August 13, 2026, improved large folder handling with lazy loading and preserved folder hierarchy in the upload panel. It only imports top-level files for now; the release notes describe recursive importing as later work, so buyers should not treat it as present.
Retrieval quality depends on choices made before the model writes an answer. Issue 5753 asks for sentence-based, semantic, recursive, and document-aware chunking because the current controls do not cover those needs. That request matters for code, tables, long manuals, and mixed reports where a generic text split can separate a claim from its context. Test a representative document set and inspect citations. A polished chat response is weak evidence that the right passages were retrieved.
Privacy depends on configuration and provider choice
AnythingLLM is MIT licensed and can run with local models, but local installation alone does not guarantee that data stays local. A cloud chat model receives prompts and retrieved passages. External embedding, speech, search, or agent tools receive the inputs required for their calls. Azure users should note issue 6172, which says the current Azure OpenAI provider requires a static API key and requests Entra ID managed identity support. That may conflict with an organization's credential policy.
Anonymous telemetry is enabled by default. The README says it records installation type, document add or remove events, provider and model tags, vector database type, and chat events without message or document contents. Operators can set DISABLE_TELEMETRY=true or turn it off in the interface. The same section lists other possible outbound connections, including configured providers, a model mirror CDN, and GitHub-hosted files. Verify network behavior against your own policy instead of treating the switch as a general offline mode.
Agents and MCP widen both usefulness and permissions
Beyond document questions, AnythingLLM has agent flows, scheduled jobs, model routing, memory, web browsing tools, image generation, and Model Context Protocol compatibility. Version 1.16.0 added /img, tool toggles during an agent session, scheduled-job tooling, and fixes around MCP definitions. These features turn a knowledge interface into software that can take actions, which calls for a stricter review of tool credentials, workspace access, and what an agent may send outside the instance.
GitHub recorded 65,259 stars, 327 combined issues and pull requests, and a push on August 27, 2026. Those numbers show current work, not a count of confirmed defects or a guarantee of stable provider behavior. AnythingLLM is worth trying because 52 seconds and 49 MB made our basic install inexpensive. The failed test run and open configuration requests are reasons to stage the exact retrieval and identity setup before using it as a shared source of answers.