llama-index review
LlamaIndex 0.14.24 is a Python framework for turning files and other data sources into retrievable nodes, then passing selected context to an LLM through query engines, chat engines, or agent workflows. The `llama-index` distribution is a starter metapackage: it installs core plus the OpenAI LLM and embedding integrations and NLTK. Other model providers, readers, and vector databases arrive as separate `llama-index-*` packages. Version 0.14.24 mainly repairs existing behavior. Ingestion upserts retain every node for a document, MMR search accepts a threshold of zero, multiblock chat history persists correctly, citation nodes receive distinct IDs and offsets, and `LLMRerank` now has an async method.
LlamaIndex 0.14.24 installed in 1.5 seconds but occupied 186 MB across 68 packages in our sandbox, so the starter bundle fits teams building document retrieval systems and is excessive for simple provider calls. pip-audit found zero known vulnerabilities, yet the failed top-level import probe and missing `py.typed` marker are real costs for teams expecting a conventional typed package.
We installed it
| Install | ✓ · 1.5s | 68 packages on disk · 186 MB · 1 deprecation warning |
| Import | ✗ | import llama-index · pure Python · requires Python <4.0,>=3.10 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does llama-index install cleanly?
Yes. In a fresh container with an empty cache, pip install llama-index finished in 2 seconds, leaving 68 packages and 186 MB on disk. pip-audit reported no known vulnerabilities. The install printed 1 deprecation warning.
What does llama-index need to run?
Python <4.0,>=3.10, and nothing compiled: it is pure Python. In our run import llama-index failed, so it needs extra system packages.
llama-index or langchain: which should you use?
langchain: Choose it when model tools and LangGraph agent state matter more than LlamaIndex's document and node abstractions. LlamaIndex 0.14.24 installed in 1.5 seconds but occupied 186 MB across 68 packages in our sandbox, so the starter bundle fits teams building document retrieval systems and is excessive for simple provider calls.
When should you not use llama-index?
Your application only sends prompts to one model API; the 0.14.24 starter installed 68 packages and occupied 186 MB in our sandbox.
Use it if
- You are building retrieval over PDFs, support articles, contracts, or another corpus and want ingestion, chunking, retrieval, and answer synthesis under one API.
- Your design needs to exchange LLM, embedding, or vector-store providers without rewriting the retrieval layer around each vendor SDK.
- You need source nodes, metadata filters, citation-oriented responses, or persisted indexes rather than a single prompt sent to a hosted model.
- Your team accepts a framework and separate integration packages in return for readers and retrievers that already follow the same node model.
- Your application only sends prompts to one model API; the 0.14.24 starter installed 68 packages and occupied 186 MB in our sandbox.
- You expect `import llama-index` to work after installing the PyPI name; our import probe failed, while the README documents imports under the `llama_index` namespace.
- You cannot pin a 0.x dependency family and test upgrades together; core and integrations publish as separate packages with bounded version ranges.
- You want local models without extra setup; the starter depends on the OpenAI LLM and embedding packages, while Ollama and Hugging Face require separate installs and configuration.
- You need static type completeness as a release gate; our installed wheel was pure Python but did not contain `py.typed`.
Setup reality
We installed llama-index 0.14.24 in a fresh Python 3.12 Bookworm container with 3 CPUs and 8 GB of RAM. The install succeeded in 1.5 seconds, produced one deprecation warning, and left 68 packages using 186 MB. PyPI declares four direct dependencies and Python 3.10 through 3.x. The wheel was pure Python, pip-audit found no known vulnerabilities, and our installed metadata did not expose a license value.
The starter selects OpenAI for both generation and embeddings, so its shortest indexing example needs OPENAI_API_KEY. PyPI lists llama-index-core, both OpenAI integrations, and NLTK as the four direct requirements. Ollama, Hugging Face embeddings, Chroma, and other integrations need their own llama-index-* distributions. For production, installing core plus named integrations makes the dependency list easier to review and prevents an unused OpenAI default from becoming accidental configuration.
Our top-level import probe failed and captured no error text. The distribution name contains a hyphen, while documented Python imports use llama_index.core or an integration path such as llama_index.llms.openai. The installed package exposed no py.typed marker. Version pins deserve attention because 0.14.24 constrains core below 0.15 and the OpenAI integration families below their next minor lines.
Indexes use memory unless you persist their storage context or connect a vector store. Disk persistence is convenient for a small local index, but changing the embedding model means rebuilding stored vectors. Async applications should choose the aquery, achat, arun, and async reranking paths instead of wrapping synchronous calls in an event loop. Version 0.14.24 fixes several cases where stored state was incomplete, including ingestion upserts, multiblock chat history, streaming response text, and citation-node identity.
Patterns
Install core and only the integrations you use install-selected-components
pip install llama-index-core \
+ llama-index-llms-openai \
+ llama-index-embeddings-openaiThe `llama-index` starter installed 68 packages in our sandbox. Selecting core and named integrations avoids pulling every starter default into production.
Index files and ask a question index-directory
import os
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex
os.environ["OPENAI_API_KEY"] = "..."
documents = SimpleDirectoryReader("./documents").load_data()
index = VectorStoreIndex.from_documents(documents)
answer = index.as_query_engine().query("What are the cancellation terms?")
print(answer)The starter's default LLM and embedding model are OpenAI integrations. This path needs `OPENAI_API_KEY` before index construction.
Replace both OpenAI defaults use-local-models
from llama_index.core import Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.ollama import Ollama
Settings.llm = Ollama(model="llama3.1", request_timeout=360.0)
Settings.embed_model = HuggingFaceEmbedding(
model_name="BAAI/bge-small-en-v1.5"
)Install `llama-index-llms-ollama` and `llama-index-embeddings-huggingface` first. `Settings` supplies defaults to components created after these assignments.
Set chunk size during indexing control-chunks
from llama_index.core.node_parser import SentenceSplitter
splitter = SentenceSplitter(chunk_size=512, chunk_overlap=64)
index = VectorStoreIndex.from_documents(
documents,
transformations=[splitter],
)Chunking changes the units stored and retrieved. Rebuild the index after changing these values so old and new nodes do not mix.
Write an index to local storage persist-index
index.storage_context.persist(persist_dir="./storage")The README states that index data stays in memory by default. Persistence writes the storage context so a later process can reload it.
Reload persisted storage reload-index
from llama_index.core import StorageContext, load_index_from_storage
storage = StorageContext.from_defaults(persist_dir="./storage")
index = load_index_from_storage(storage)
query_engine = index.as_query_engine()Loading requires the same embedding and index assumptions used when the files were written. Change the embedding model only with a rebuilt index.
Restrict retrieval by document metadata filter-metadata
from llama_index.core.vector_stores import MetadataFilter, MetadataFilters
filters = MetadataFilters(
filters=[MetadataFilter(key="tenant_id", value="acme")],
)
query_engine = index.as_query_engine(filters=filters, similarity_top_k=5)
response = query_engine.query("Which invoices are overdue?")Version 0.14.24 defaults a missing `MetadataFilters.condition` to AND. Confirm that the selected vector-store integration supports the operator and value type you use.
Inspect the nodes behind an answer inspect-retrieval
response = query_engine.query("What changed in the renewal clause?")
for source in response.source_nodes:
print(source.score)
print(source.node.metadata)
print(source.node.get_content()[:240])`source_nodes` exposes retrieved content and scores. Check them before blaming answer synthesis when a response misses a fact.
Stream generated response text stream-answer
query_engine = index.as_query_engine(streaming=True)
response = query_engine.query("Summarize the incident report")
for text in response.response_gen:
print(text, end="", flush=True)Retrieval completes before generated text is yielded. Version 0.14.24 also fixes the response text stored when a streaming chat result is written to memory.
Run a query without blocking an async service query-async
async def answer(question: str):
query_engine = index.as_query_engine()
return await query_engine.aquery(question)Use `aquery` inside an existing event loop. Calling the synchronous `query` method there can block other requests while retrieval and generation run.
Give an agent a typed Python tool build-function-agent
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.llms.openai import OpenAI
def lookup_order(order_id: str) -> str:
"""Return the status for one order ID."""
return order_store[order_id]
agent = FunctionAgent(
tools=[lookup_order],
llm=OpenAI(model="gpt-5-mini"),
)
result = await agent.run("Where is order A-104?")`FunctionAgent.run` is async and derives the tool description from the function signature and docstring. Validate authorization inside the function itself.
Return citations with an answer build-citation-query
from llama_index.core.query_engine import CitationQueryEngine
citation_engine = CitationQueryEngine.from_args(
index,
similarity_top_k=5,
citation_chunk_size=512,
)
response = citation_engine.query("Which section permits termination?")Version 0.14.24 assigns each citation node its own ID and offsets. Applications still need to render and verify the cited source nodes.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| langchain | PyPI | Choose it when model tools and LangGraph agent state matter more than LlamaIndex's document and node abstractions. |
| haystack-ai | PyPI | Choose it when explicit pipeline components and inspectable connections suit your production retrieval flow. |
| txtai | PyPI | Choose it for an embeddings database with search and optional generation in a smaller conceptual surface. |
More ai / ml guides
openai · mcp · huggingface-hub · @modelcontextprotocol/sdk · scikit-learn · tiktoken · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.