The data team has to define what an answer means
Before giving nao access to a warehouse, decide what a question such as "revenue last month" should count. The project combines a context-building CLI with a chat application that turns questions into analysis. Its context principles call for one canonical definition per metric and documented joins. Choose nao if someone can own those definitions after the demo.
Our checkout contained 2,412 files and about 331,042 lines of source in 45.6 MB. This is a workspace monorepo with a TypeScript application and a separate Python CLI. That scope suits a data team serving colleagues more than someone querying a local table.
What happened when we ran it
Our run installed commit 5bde830 in 16 seconds on October 11, 2026, adding 1,875 packages and occupying 2,045 MB on disk. The unprivileged lab-node:22 sandbox had 3 CPUs, 8 GB of RAM, and no secrets. The lab identified a Node project using Bun. Its build and tests were skipped because it found no script or target for either step.
Npm audit reported 51 known vulnerabilities: 7 critical, 13 high, 15 moderate, and 16 low. The scan found 12 CI workflow files, a Dockerfile, and a compose file, but no tests directory. The installation succeeded, but these measurements provide no application build or agent accuracy result. A missing detected test target also doesn't prove there are no checks elsewhere in the workspaces.
The audit findings deserve attention before warehouse access
The 7 critical and 13 high findings make a production trial conditional on dependency review. The supplied lab summary does not identify the packages, affected paths, or whether deployed code can reach them. The totals establish a review requirement, without proving that an attacker could exploit the deployed app.
The 51-vulnerability result belongs to the installed environment at commit 5bde830. A published container may have a different dependency tree. Ask for the detailed audit and inspect the version you intend to ship. A team with a policy against unresolved critical advisories has enough evidence to postpone adoption until that review is complete.
The Python quick start differs from source development
The documented CLI route uses nao init to create a project, then nao debug and nao sync before starting chat. The configuration guide separates database access from model-provider settings and repository sources. Initialization can defer configuration, but useful warehouse questions still need a working data connection and model endpoint.
Our 2,045 MB installation measured the repository environment, rather than timing that Python quick start. Source development also needs nvm, Bun, and uv, an authentication secret, the public app URL, and a project path. The contributor guide pins Node and npm versions. The local browser app and packaged chat use different documented addresses, so copy the configuration for your chosen route instead of mixing examples.
Self-hosting still needs durable storage and a model policy
The deployment guide uses PostgreSQL for the application database and tells operators to keep BETTER_AUTH_SECRET stable across restarts. It also warns that container-local attachment storage is ephemeral on platforms such as Cloud Run. Choose a durable volume or object storage before users start saving exports.
A 16-second install with no secrets could not exercise your authentication, warehouse permissions, or provider configuration. The chat guide says external data transmission depends on configuring an external LLM provider. Self-hosting the interface therefore does not by itself keep model calls inside your network. Also read the actual license: Enterprise-marked files have commercial terms separate from the mostly Apache-licensed code.
The evaluation suite needs your own reference SQL
The evaluation workflow asks you to write questions and expected SQL in YAML. It then compares the agent's returned data with the reference query's result. That can catch context changes that damage business answers. It requires someone who already knows the right query, and the reference itself needs review when a metric definition changes.
The lab's 1,875-package installation produced no measured test result, so the presence of this evaluation feature cannot support a claim that nao answered correctly for us. Start your own acceptance work with questions whose results you can inspect. The project also publishes context-engineering skills for Claude Code and other coding agents, but generated rules and evaluation cases still need a data owner's approval before becoming the standard for an answer.
Saved stories can retain an outdated metric definition
Open issue 1852 reports that stories retain compiled SQL after the underlying MetricFlow metric definition changes. The request asks for existing stories to resolve the current definition or be refreshable. It gives teams relying on centrally governed metrics a reason to delay replacement of their established reporting setup.
The 51 audit findings and this stale-definition report address different adoption risks: dependencies need review, while saved analytics need a maintenance policy. The MCP endpoint lets external clients delegate questions or directly browse context and execute SQL. Calls run as the token owner. Extending access into another assistant increases the importance of agreeing which definitions and results users should trust.
October changes show active work without settling readiness
The last push was October 9, 2026, and GitHub listed 267 open issues and PRs combined. Release v0.3.22 arrived on October 7 with provider fixes and changes to project-file access. Recent issue activity includes the semantic-layer request. Those dates show maintenance, without settling the audit or saved-metric concerns.
For fixed reports, compare Metabase or Apache Superset before adopting the 1,875 packages we measured. Vanna is another option when you want to build a SQL assistant into your own application. nao earns a trial when your data team wants both editable context and a shared chat interface. Start with a warehouse question your analyst can answer exactly, then change its metric definition and check whether the saved story follows.