mrkeyoor.com_
Wed 11 Nov 06:22 UTC
LLM Toolsevaluationupdated 25 Oct 2026

baml review

BAML (Basically A Made-up Language) is BoundaryML's programming language for agents. Its README describes runtime types, statically analyzed errors, concurrency, built-in tests and incremental adoption through calls from TypeScript, Python, Go, C# and Java. The supplied testing documentation covers the legacy BAML v0 DSL and explicitly distinguishes it from BAML v1, a separate programming language in public beta.

+18stars / 7d
Verdict

BAML merits a bounded evaluation if you want an agent programming language that can be called from existing applications. First choose the intended version: the supplied v0 testing guide does not document v1. Our October 13, 2026 source-workspace run at 4c21adc installed dependencies, then encountered missing uv during both build and test commands. It establishes an environment prerequisite, not that BAML's tests or released CLI are broken. Before adoption, verify the released installation path and test your actual extraction cases, especially record preservation and request compatibility.

We ran it

Lab card: what happened when we ran bamlScreenshot of baml (boundaryml.com/explore)
Install✓ · 117s2528 packages · 3740 MB
Build✗ · 12s
Tests✗ · 9sran, no count parsed
Repo10973 files~2,385,899 lines of source · 152.3 MB · 53 CI workflows

Answers from our run

Does baml build from source?

Dependencies installed in 117 seconds (2528 packages), and the build failed. We cloned commit 4c21adc into a clean Debian container with 3 CPUs and no project-specific setup.

Do baml's tests pass?

No. The project's own test command exited unsuccessfully after 9 seconds, and its output did not expose an individual test count.

Who should not use baml?

Teams that need a verified production recommendation from this review: our recorded run did not establish a working installed CLI or application behavior.

What are the alternatives to baml?

Instructor, Guardrails AI. BAML merits a bounded evaluation if you want an agent programming language that can be called from existing applications.

Setup2/5Under the visible evidence rubric, setup has a material verification gap: the October 13, 2026 dependency install succeeded, but build and test commands encountered missing uv. No released CLI installation or successful invocation was checked.
Docs2/5The supplied testing guide documents v0 and distinguishes it from v1 public beta. It provides useful legacy examples but cannot substantiate a v1 setup walkthrough.
Community2/5Contributor responses dated September 22 and 23, 2026 are visible in the supplied issues. They establish engagement with those reports, not broad adoption, support speed or community health.
Maturity2/5The supplied documentation labels v1 public beta. Issue #4590 documents a record-loss report, but its behavior at the October measured commit was not tested; the intentional request change in #4585 was confirmed and closed.

Discussed on

  1. hnShow HN: BAML - A new programming language to get structured data from LLMs18 points
  2. hnBaml: The Programming Language for Agents8 points
  3. hnBAML is a language that helps you get structured data from LLMs4 points

Who it’s for

Developers evaluating a dedicated language for agent applications who can check documentation against their intended BAML version.
Teams interested in calling BAML functions from existing TypeScript, Python, Go, C# or Java applications, as described in the README.
Structured-extraction teams willing to test whether parsing preserves every expected record before adopting a release.

Who it’s NOT for

Teams that need a verified production recommendation from this review: our recorded run did not establish a working installed CLI or application behavior.
Contributors expecting pnpm dependency installation alone to prepare the full source workspace: our build required uv, which was unavailable in the recorded environment.
Extraction workflows that cannot tolerate missing records without first checking the parsing behavior reported in issue #4590 against their chosen version.
Developers expecting legacy v0 testing examples to establish v1 syntax or compatibility.

Setup reality

In our recorded run on October 13, 2026, at commit 4c21adc8107a8afb47409e602abf7867cfd280bd, the pnpm dependency-install step exited successfully after 117 seconds. The record reports 2,528 dependencies and a 3,740 MB installed footprint. The checkout measured 152.3 MB, 10,973 files and approximately 2,385,899 source lines.

The install log also warned that it could not create a baml-cli binary link because the target file was missing. Successful dependency installation therefore does not establish that the CLI was installed or runnable.

The build command exited with code 1 after approximately 12 seconds. The test command exited with code 1 after approximately 9 seconds. Both logs identify @baml/language-client-python running uv run maturin develop --uv --manifest-path Cargo.toml, followed by sh: 1: uv: not found. This is a concrete missing prerequisite in that environment, not proof of a product defect or failing test assertions.

The recorded configuration was 3 CPUs, 8 GB memory and image lab-node:22. The image label does not verify the runtime version. We did not perform a new run, a released-binary installation check or a model workload.

BAML's adoption decision starts with choosing a version. The supplied testing guide documents the legacy v0 DSL, while the current repository presents BAML as a programming language for agents. Mixing those descriptions would give readers an unreliable setup guide.

What BAML offers, and what remains a README claim

BAML stands for Basically A Made-up Language. BoundaryML's README describes runtime types, statically analyzed errors, green threads, built-in tests and an agent standard library. It says BAML can run standalone or be adopted incrementally through functions called from TypeScript, Python, Go, C#, Java and other languages.

Those are upstream descriptions. Our recorded run did not measure compiler speed, agent accuracy, concurrency, interoperability or model-output quality. The README's speed comparisons are not benchmarks from this review.

For a team considering BAML, incremental adoption is the useful proposition to investigate: try one function from an existing application before committing to a wider language migration. Whether that helps your application remains a pilot question.

Keep v0 documentation separate from v1

The supplied testing guide identifies itself as documentation for BAML v0, the legacy DSL, and describes v1 as a separate programming language in public beta. Its examples cover test blocks, image and audio inputs, PDFs, checks, assertions and command-line testing. These examples establish documented v0 facilities, not verified v1 compatibility.

Choose the version before copying syntax or commands. This review does not provide a verified v1 installation or testing walkthrough. The supplied release page identifies BAML Language 0.17.1-nightly.20260824.b; it does not establish that this is the latest release or the version represented by every package in our October checkout.

What our recorded source-workspace run established

Our recorded run took place on October 13, 2026, at commit 4c21adc8107a8afb47409e602abf7867cfd280bd, whose recorded commit date is October 12, 2026. The configuration was 3 CPUs, 8 GB memory and image lab-node:22. The image name alone does not establish the actual runtime version.

The pnpm dependency-install step succeeded in 117 seconds. The record reports 2,528 dependencies and a 3,740 MB installed footprint. The repository measured 152.3 MB, with 10,973 files and approximately 2,385,899 source lines. These are source-workspace measurements, not the download size or setup cost of a released BAML installation.

Installation included a warning that a baml-cli binary link could not be created because its target was missing. We have no successful installed-binary check. Dependency resolution and installation must therefore remain separate from a claim that the CLI worked.

The build command exited with code 1 after approximately 12 seconds. Its Python language-client task attempted uv run maturin develop --uv --manifest-path Cargo.toml and reported sh: 1: uv: not found. Rust channel synchronization for 1.98.0-x86_64-unknown-linux-gnu also appears in the log, but it is not the reported cause of this task's failure.

The test command exited with code 1 after approximately 9 seconds at the same Python build prerequisite. A bridge test task was scheduled in the visible tail, but no individual test results or complete suite count were supplied. Turbo's successful-task counts are task counts, not passing-test counts. We cannot conclude that all tests were skipped, that assertions failed or that a particular number passed.

No new execution, model workload, throughput benchmark or security audit supports this review. The next source-build step is to satisfy the missing uv prerequisite and rerun the commands. That has not been done here.

Parsing needs record-preservation checks

Issue #4590 reports that schema-aligned parsing dropped an uncoercible list element while returning success. The author identifies BAML 0.17.0 with wrapper 0.2.0 and later reports the same behavior on 0.17.1-nightly.20260824.b. This is the reporter's reproduction, not our sandbox result.

In a September 23, 2026 comment, a contributor acknowledged the behavior and said the team was still deciding how to expose the information. The supplied extract shows the issue as open. It does not establish whether the October checkout still has the same behavior.

If each extracted row becomes a downstream record, include cases with a required field missing inside one array element. Check that every expected record survives or produces a visible failure. A successful exit code alone would not detect the reported loss.

Request changes need version-specific fixtures

Issue #4585 reports that text-only OpenAI-generic requests changed from a content-part array in 0.15.0 to a plain string in 0.17.0. The report also describes prompt newline differences between the two syntaxes.

On September 22, 2026, a contributor confirmed that the request change was intentional to improve compatibility with completions-style inference backends. The issue was closed as completed that day. It should not be presented as an unresolved accidental regression.

For applications that record or byte-compare outgoing requests, retain captured-request fixtures across a migration. The contributor's explanation establishes intent; it does not guarantee byte stability across future releases.

When a library is enough

Instructor demonstrates structured extraction through Pydantic models and a library API, including automatic retries after validation failures. It is a relevant alternative when typed extraction is the immediate requirement.

Guardrails AI describes input/output guards and structured data generation. It is relevant when explicit validators are central to the application. Neither comparison is a measured performance judgment, and neither tool was run for this review.

How to read the scores

Scores are editorial assessments of the supplied evidence, not measured product quality. The five-point rubric is: 1, a documented blocker for the stated use; 2, a material limitation or substantial verification gap; 3, a documented fit with bounded limitations; 4, broad corroboration plus relevant successful execution; 5, extensive independent validation across the stated scope. Unknown adoption or compatibility cannot justify a high score.

Setup receives 2 because dependency installation succeeded but source build and test commands encountered missing uv, and no released CLI was checked. Documentation receives 2 because the supplied testing reference is useful for v0 but cannot support a current v1 walkthrough. Community receives 2 because dated contributor responses are visible, while adoption and support responsiveness remain unmeasured. Maturity receives 2 because the supplied documentation labels v1 public beta and the evidence leaves record-preservation behavior at our measured commit unverified.

Start with one version-pinned function and representative inputs. Verify installation, invocation, output preservation and any request fixtures that your application depends on before expanding the pilot.

Alternatives

ProjectWhat it isPick it when
InstructorA structured-output library whose Python examples use Pydantic models, validation and automatic retries.Choose it when your immediate task is extracting typed data through a library API. Its supplied README demonstrates that workflow directly; it is an application-library alternative, not a replacement for BAML's source-build tooling.
Guardrails AIA Python framework for input/output guards and structured data generation.Choose it when your application needs explicit validators on model inputs or outputs. Evaluate its guard workflow separately from adopting an agent programming language; this review did not run Guardrails.

What people are saying

  1. [github-trending] BoundaryML/baml

Sources

  1. BAML README
  2. BAML repository
  3. BAML 0.17.1 nightly release
  4. BAML testing guide
  5. Silent array element loss report #4590
  6. OpenAI-generic request change report #4585

More llm tools reviews

mcp-server-guide · memorilla · 12-factor-agents · gemini-api-cli · MCPJungle · octomind · the whole board →