PyPIDataupdated 06 Aug 2026

dbt-core

Item: dbt-core
Rating: 4
Author: Keyoor

dbt is a transformation framework for data warehouses. You write SELECT statements as .sql files, have them reference each other with {{ ref('other_model') }}, and dbt works out the dependency graph, wraps each SELECT in the right CREATE TABLE or CREATE VIEW for your warehouse, and executes them in order. Around that it adds declarative data tests, snapshots for slowly changing dimensions, generated docs with lineage, and Jinja macros for reuse. What it deliberately does not do is move data or decide when to run: dbt assumes the rows are already in the warehouse and that something else invokes it on a schedule.

dbt-core · v1.12.0 · 23.2M/wk · ★ 13,586 · pushed 06 Aug 2026 · dbt-labs/dbt-core · docs · reviewed by Keyoor

Verdict

Still the default way to organise warehouse SQL, and the surrounding ecosystem of adapters, packages, and hiring pool is the real reason to pick it. Go in knowing the Python implementation is now the older of two dbt engines, and budget for a migration decision rather than assuming 1.x is forever.

API stability3/5Minor releases keep moving the surface: adapters were split out of core in 1.8, the schema key tests was renamed data_tests, snapshots gained a YAML form in 1.9, and a v2.0 alpha with a stricter language spec is now on main.

Docs5/5docs.getdbt.com is versioned, covers every config, materialization, and adapter with worked examples, and the guides are good enough that most teams onboard analysts straight from them.

Maintenance3/5Commits land daily and 1.12.0 shipped in July 2026, but the Python line sits on a 1.latest maintenance branch while new engineering goes into the Rust v2 rewrite, and there are 1252 open issues (1486 counting PRs).

Ecosystem5/5Adapters for every mainstream warehouse, dbt Hub packages such as dbt_utils and codegen, first-party Airflow and Dagster integrations, plus testing and observability tools built specifically against dbt artifacts.

Use it if

Your transformation logic is SQL living in a warehouse (Snowflake, BigQuery, Databricks, Redshift, Postgres) and you want it in git with code review, tests, and a dependency graph instead of a folder of scheduled queries
You want incremental models, snapshots, and environment-specific schemas handled by the framework rather than by hand-written MERGE statements per table
Analysts on your team write SQL but not Python, and you need them shipping models without owning a pipeline runtime
You want column and model documentation plus lineage generated from the same files that define the transforms, so it stays roughly honest

Skip it if

You are choosing a long-term foundation today: dbt Labs moved v1 Python development to a 1.latest branch, the main branch is now a v2.0 alpha rewritten in Rust that underpins the Fusion engine, and the project's own getting-started text steers new users toward Fusion instead
Your real problem is getting data into the warehouse: dbt has no extract or load story at all, so dlt, Airbyte, Fivetran, or your own loaders sit in front of it either way
You need scheduling, retries, and backfills: dbt Core is a CLI you invoke, and everything about when it runs belongs to cron, Airflow, Dagster, or dbt Cloud
You want to install it next to Airflow in one environment: 24 pinned runtime dependencies including protobuf, pydantic, jinja2, and agate make shared virtualenvs a recurring resolver fight, so plan on isolation or containers
The project is small: three transformation queries do not justify a project directory, profiles.yml, an adapter package, and a Jinja dialect on top of SQL
Your logic is genuinely Python: dbt is SQL plus Jinja, and Python models only exist on a subset of adapters such as Snowflake, Databricks, and BigQuery

Setup reality

pip install dbt-core gives you a CLI that cannot reach any database. Since 1.8 the adapters ship separately, so you also install dbt-postgres, dbt-snowflake, dbt-bigquery or whichever warehouse you have, and version compatibility between core and adapter is a thing you will check again after every upgrade. dbt init scaffolds the project, but credentials live in a profiles.yml under ~/.dbt outside the repo, keyed by a profile name that must match dbt_project.yml or nothing runs. dbt debug is the command that tells you which of those two files is wrong. Anonymous usage statistics are on by default (snowplow-tracker is a hard dependency) and you switch them off with send_anonymous_usage_stats: false. Then there is Jinja: your models are templates, so half of debugging is reading target/compiled to see what SQL actually got sent. Python 3.10 or newer.

Patterns

Install dbt with a warehouse adapterinstall-with-adapter

python -m venv .venv && source .venv/bin/activate
pip install dbt-core dbt-postgres
# or: dbt-snowflake, dbt-bigquery, dbt-databricks, dbt-redshift
dbt --version

dbt-core on its own connects to nothing. Adapters were decoupled from core in 1.8, and dbt --version prints core and adapter versions separately so you can see when they have drifted.

Point the project at a warehouseconfigure-profile

# ~/.dbt/profiles.yml
analytics:
  target: dev
  outputs:
    dev:
      type: postgres
      host: localhost
      port: 5432
      user: analytics
      password: "{{ env_var('DBT_PASSWORD') }}"
      dbname: warehouse
      schema: dbt_dev
      threads: 4

The top-level key must equal the profile: value in dbt_project.yml. Run dbt debug after any change; it is the only command that reports which of the two files disagrees.

Build a model on another modelmodel-with-ref

-- models/staging/stg_orders.sql
{{ config(materialized='view') }}

select
    id as order_id,
    customer_id,
    status,
    created_at
from {{ source('shop', 'orders') }}

-- models/marts/orders_daily.sql
select
    date_trunc('day', created_at) as day,
    count(*) as orders
from {{ ref('stg_orders') }}
group by 1

ref() and source() are the whole point: they build the DAG and let dbt rewrite the schema per target, so a hardcoded table name silently breaks dev and prod separation.

Declare raw tables and their freshnessdeclare-sources

# models/staging/_sources.yml
version: 2

sources:
  - name: shop
    schema: public
    loaded_at_field: _synced_at
    freshness:
      warn_after: { count: 12, period: hour }
      error_after: { count: 24, period: hour }
    tables:
      - name: orders
      - name: customers

dbt source freshness is a separate command from dbt run, so it only protects you if your scheduler actually calls it before the build.

Attach data tests to columnsadd-tests

# models/staging/_models.yml
version: 2

models:
  - name: stg_orders
    columns:
      - name: order_id
        data_tests:
          - unique
          - not_null
      - name: customer_id
        data_tests:
          - relationships:
              to: ref('stg_customers')
              field: customer_id
      - name: status
        data_tests:
          - accepted_values:
              values: ['placed', 'shipped', 'returned']

This key was tests: before 1.8 and is data_tests: now. Older blog posts use the old spelling, which still parses but emits a deprecation warning.

Append only new rows on each runincremental-model

{{ config(
    materialized='incremental',
    unique_key='event_id',
    incremental_strategy='merge'
) }}

select *
from {{ source('events', 'raw_events') }}

{% if is_incremental() %}
where ingested_at > (select coalesce(max(ingested_at), '1900-01-01') from {{ this }})
{% endif %}

The is_incremental() block is skipped on the first build and on dbt run --full-refresh. Which incremental_strategy values exist depends on the adapter, so merge is not available everywhere.

Track slowly changing dimensionssnapshot-changes

# snapshots/customers.yml
snapshots:
  - name: customers_snapshot
    relation: source('shop', 'customers')
    config:
      schema: snapshots
      unique_key: id
      strategy: timestamp
      updated_at: updated_at

YAML snapshots arrived in 1.9; projects older than that use a {% snapshot %} block in a .sql file with target_schema instead of schema. Snapshot tables hold history you cannot rebuild, so they need real backups.

Factor repeated SQL into a Jinja macrowrite-macro

-- macros/cents_to_dollars.sql
{% macro cents_to_dollars(column_name, scale=2) %}
    round(1.0 * {{ column_name }} / 100, {{ scale }})
{% endmacro %}

-- usage in a model
select
    order_id,
    {{ cents_to_dollars('amount_cents') }} as amount_usd
from {{ ref('stg_orders') }}

Macros emit text, not typed SQL, so a mistake surfaces as a warehouse syntax error against generated code rather than as a dbt error against your macro.

Pull in community macro packagesinstall-packages

# packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: [">=1.0.0", "<2.0.0"]

# then
dbt deps

dbt deps vendors packages into dbt_packages/, which belongs in .gitignore; commit packages.yml and, if you want reproducible builds, package-lock.yml.

Run only part of the graphselect-and-build

dbt build                          # run and test everything in DAG order
dbt build --select stg_orders+     # this model and everything downstream
dbt build --select +orders_daily   # this model and everything it depends on
dbt build --select tag:nightly
dbt run --full-refresh --select raw_events_incremental

dbt build interleaves tests with model runs so a failing test blocks its downstream models; dbt run followed by dbt test does not, and happily builds on bad data.

Read the SQL dbt actually sentinspect-compiled-sql

dbt compile --select orders_daily
cat target/compiled/analytics/models/marts/orders_daily.sql

# the full statement including the materialization wrapper
cat target/run/analytics/models/marts/orders_daily.sql

target/compiled shows your query after Jinja; target/run shows it wrapped in the create or merge statement. Almost every confusing dbt failure is explained by one of these two files.

Turn off anonymous usage statisticsdisable-telemetry

# ~/.dbt/profiles.yml
config:
  send_anonymous_usage_stats: false

# or per invocation
export DBT_SEND_ANONYMOUS_USAGE_STATS=false

Telemetry is on by default and snowplow-tracker is a hard dependency of dbt-core, so on locked-down networks this is worth setting before your first run rather than after a proxy alert.

Alternatives

Package	Registry	Pick it when
sqlmesh	PyPI	You are starting fresh and want column-level lineage and cheap virtual environments rather than rebuilding a dev warehouse.
dlt	PyPI	Your bottleneck is loading data into the warehouse, which dbt does not attempt.
dagster	PyPI	You need asset-aware scheduling, backfills, and observability around the transforms, with or without dbt running inside it.