mrkeyoor.com_
Tue 01 Sept 17:40 UTC
Dataevaluationupdated 28 Aug 2026

lance review

Lance is a columnar file and table format for datasets that mix embeddings with text, images, audio, or video. It gives data teams versioned tables, random row access, vector and full-text indexes, and SQL-style filtering without requiring a separate vector store for each dataset.

+27 / 3dstars / 7d
Verdict

Our Lance build completed in 715 seconds, but its tests were still compiling when our 900-second limit expired, so adopting the source tree demands patience even before data validation starts. Use Lance when embeddings and source media belong in the same object-storage table and random access is central to the workload. Pin a stable storage version, test compaction against your schemas, and prefer LanceDB if what you need is an application database rather than a format.

We ran it

Lab card: what happened when we ran lanceScreenshot of lance (lance.org)
Install✓ · 48s917 packages
Build✓ · 715s
Tests✗ timed out · 900sran, no count parsed
Repo1913 files~874,829 lines of source · 54.1 MB · 29 CI workflows

Answers from our run

Does lance build from source?

Dependencies installed in 48 seconds (917 packages), and the build succeeded in 715 seconds. We cloned commit 29c4d59 into a clean Debian container with 3 CPUs and no project-specific setup.

Do lance's tests pass?

We could not finish them: the suite was still running after 15 minutes in our container.

Who should not use lance?

Teams that need a quick source build and test loop: our 3-CPU build took 715 seconds, and the test command was still compiling after its 900-second limit.

What are the alternatives to lance?

Apache Iceberg, Delta Lake, Apache Hudi. Our Lance build completed in 715 seconds, but its tests were still compiling when our 900-second limit expired, so adopting the source tree demands patience even before data validation starts.

Setup2/5715-second build; tests still compiling at 900 seconds
Docs4/5Clear quick start and storage-version warnings
Community5/56,984 stars and same-day code and issue activity
Maturity4/5Stable format contract, with serious compaction reports open

Discussed on

  1. hnShow HN: Lance – Open lakehouse format for multimodal AI datasets4 points

Who it’s for

Data and ML teams storing embeddings beside the source media or features they describe.
Engineers who need vector search, full-text search, and filtered scans against one versioned dataset.
Python, Rust, or Java teams willing to adopt a newer format and pin its storage version.
Lakehouse operators who want object-storage tables that work with Arrow-based tools.

Who it’s NOT for

Teams that need a quick source build and test loop: our 3-CPU build took 715 seconds, and the test command was still compiling after its 900-second limit.
Operators who cannot pin writers in a mixed-version fleet: the README says older Lance releases may not understand newer storage versions and tells production users to set data_storage_version.
Workloads that depend on binary-copy compaction with reordered physical columns today: open issue 8501 reproduces wrong column mapping and possible silent data corruption on v11.0.0-beta.4.
Buyers seeking a ready-made hosted database: this repository supplies a file format, table format, catalog specification, and SDKs, so serving, access control, and object storage remain your responsibility.

Setup reality

Our sandbox installed 917 packages in 48 seconds. The build succeeded in 715 seconds, but tests hit the 900-second limit before producing results. The log tail only shows Rust crates still compiling, including AWS S3, DataFusion Substrait, Hugging Face Hub, Lance tools, examples, and compression crates; it does not show a failed assertion.

The local Python quick start needs no account or service. Object-storage deployments need the chosen provider's credentials and URI configuration, while production writers should set a stable data_storage_version. The README warns that the next alias is experimental.

This is a Rust core with Python and Java bindings, and the checkout contains 1,913 files and about 874,829 source lines. The repository has 29 CI workflow files and a compose file, but no Dockerfile or top-level tests directory. Building from source is a different commitment from installing the published Python package.

Lance keeps vectors and source data in one table

Lance is for datasets where an embedding is only one part of the record. A row can carry text, an image, audio, video, or features beside the vector used to retrieve it. The same table supports vector similarity, BM25 full-text search, SQL predicates, and secondary indexes. That removes the synchronization job created when a lakehouse stores the source while a separate vector database stores a partial copy.

The repository contains a file format, a table format, and a catalog specification. Its Rust core feeds Python and Java bindings, while the README lists integrations with Arrow, Pandas, Polars, DuckDB, Spark, Ray, Trino, and Flink. Our checkout at commit 29c4d59 contained 1,913 files and about 874,829 lines of source, so this is a storage system with several language surfaces, not a thin serializer.

Stable storage versions require deliberate writer settings

Every dataset records a data_storage_version. The project promises that future Lance releases will keep reading stable versions, which is the compatibility guarantee that matters for durable data. SDK and API compatibility is separate and follows semantic versioning. Older clients may fail to understand a version introduced by a newer writer, so mixed fleets should choose a stable version explicitly rather than accepting whatever the newest process writes.

The README reserves the next storage alias for experiments and says never to use it for production data. Preview packages are also published more frequently than full releases and remain available for at least 6 months, but that promise concerns package availability rather than a stable file layout. A staged rollout should pin both the package and the storage version, write sample datasets, then prove that every reader in the fleet can open them.

What happened when we ran it

Our sandbox installed 917 packages in 48 seconds and completed the build in 715 seconds. The clean checkout occupied 54.1 MB before those dependencies. Installation and compilation both succeeded, which confirms that commit 29c4d59 can get through its build in an unprivileged Debian container with 3 CPUs and 12 GB of RAM. It also shows why editing the Rust core is a heavier job than the short pip install pylance quick start suggests.

Tests did not finish inside our 900-second limit. The final log lines show compilation still moving through aws-sdk-s3, datafusion-substrait, hf-hub, Lance namespace packages, lance-tools, lance-examples, and bit-packing crates. There is no assertion failure or error in that tail, and no test count was produced. The honest result is a timeout during compilation, not a failed behavior check and not a passing suite.

Local files are easy; shared object storage adds ownership

The README's smallest example converts a Parquet dataset in /tmp, reopens it through PyArrow, and reads it into Pandas. That path needs no hosted account. A production lakehouse changes the job: the team must supply object-storage credentials, settle URI and catalog configuration, choose who may write, and coordinate storage versions across services. Lance provides ACID transactions, version history, tags, and branches, but it does not operate your identity or storage controls.

The repository has 29 CI workflow files and a compose file, with no Dockerfile and no top-level tests directory in our scan. Those signals fit a multi-language library that is tested through several targeted workflows. They do not give a one-container production recipe. Teams building Rust, Python, and Java consumers should test the exact binding and object-store path they will deploy, particularly when separate services may upgrade on different schedules.

Compaction reports deserve a pre-production regression test

Open issue 8501 provides a public Rust reproducer for binary-copy compaction on files whose physical column order differs from the table schema. The report says v11.0.0-beta.4 can record the compacted mapping in schema order while copying pages in physical order. A debug build panics on the mismatched data type; a release build may return wrong values. That is a narrow set of conditions, but silent corruption is serious enough to test directly.

Another open report, issue 8281, describes a full scan failing after adding a column and compacting a dataset on storage version 2.3. Its author could not reduce the problem to the simpler combinations they tried, so it should not be generalized to every compaction. Together, the 2 reports point to a practical acceptance test: reproduce your schema evolution, merge-insert, and compaction sequence, then compare every value before and after optimization.

Same-day work and v10.0.0 show an active project

GitHub showed a push on August 28, 2026, 6,984 stars, and 1,062 combined open issues and pull requests. Recent activity included fixes and performance work across encoding, HNSW, namespaces, full-text search, and Java callbacks. The large combined queue is not a defect count. It matches a repository with many bindings and active development, and same-day updates show maintainers and contributors are still working through it.

The latest stable release fetched from GitHub was v10.0.0, published August 7, 2026. Its notes include breaking changes, format-decoder safety fixes, new indexes, and many storage fixes. That pace is useful if Lance solves your access pattern, but upgrades deserve migration review. The format is a strong fit for multimodal training or retrieval tables; an ordinary analytics lake with broad SQL-engine requirements may be better served by Iceberg, Delta Lake, or Hudi.

Alternatives

ProjectWhat it isPick it when
Apache IcebergA widely adopted table format centered on large analytic datasets and engine interoperability.pick this instead when compatibility across established SQL engines matters more than native vector and multimodal operations.
Delta LakeA transaction layer for data lakes with mature Spark-oriented workflows.pick this instead when your platform already runs on Spark and Delta tables.
Apache HudiA lakehouse table format built around incremental ingestion and change processing.pick this instead when upserts, incremental pipelines, and existing Hudi engine support drive the choice.
LanceDB gh↗A database layer built on Lance for application-facing vector and multimodal retrieval.pick this instead when you want database APIs and a serving layer rather than adopting the storage format directly.

What people are saying

  1. [github-trending] lance-format/lance

Sources

  1. Lance README
  2. Lance v10.0.0 release
  3. Issue 8501: binary-copy compaction column-order corruption
  4. Issue 8281: full scan after schema change and compaction

More data reviews

turso · TrackersListCollection · dash · getcontact-cli · awesome-zhuiju-free · iggy · the whole board →