Skip to main content

dbt Integration

A dbt model is not SQL. It is a Jinja template that becomes SQL, and that is the whole difficulty: you cannot format the template as SQL, because it isn't SQL, and you cannot format the compiled SQL and hand that back, because every {{ ref() }} the author wrote would be gone.

dfmt does neither. It evaluates the template itself, formats the SQL that comes out, and then puts the template back over the formatted text.

No dbt compile, no Python, no dbt install — dfmt has its own Jinja evaluator and its own dbt compiler, and it bundles the standard adapter macros.

How it works

When dfmt finds dbt_project.yml, it:

  1. Loads the projectdbt_project.yml, models, macros, packages, and the adapter type.
  2. Evaluates the template with a dbt-shaped context: ref(), source(), config(), var(), is_incremental(), your macros, and the bundled adapter macros.
  3. Formats the compiled SQL — one parse, one layout pass, in the dialect the adapter implies.
  4. Re-imposes the template on the formatted text by position, so every construct returns to the place it was written, character for character.
  5. Formats branches that were not taken. A {% if %} whose condition was false contributed no SQL to step 3, so its body gets its own pass and is spliced back in. Otherwise the code you cannot see today would be the code that never gets formatted.

What survives

Every Jinja construct is reproduced verbatim — the text between {% and %} is never rewritten:

ConstructExample
Expressions{{ ref('users') }}
Filters{{ col | upper }}
Config blocks{{ config(materialized='table') }}
Conditionals{% if is_incremental() %}…{% endif %}
Loops{% for c in cols %}…{% endfor %}
Set blocks{% set q %}…{% endset %}
Macro calls{{ dbt_utils.star(ref('m')) }}
Comments{# … #}
Split identifiers{{ schema }}.{{ table }}

dbt features

  • ref() and source()
  • config()
  • var() with defaults
  • is_incremental()
  • project and package macros (dbt_utils, dbt_expectations, …)
  • adapter dispatch, and {% materialization %} blocks
  • env_var() — resolved to a placeholder, since the value cannot change the layout

Packages must be installed: run dbt deps first. If packages.yml lists packages that are not in dbt_packages/, dfmt stops with an error rather than expanding the missing macros to nothing and formatting whatever that produces.

What is not formatted

dfmt skips directories that are not your code: dbt_packages, target, node_modules, .venv, venv, .git.

dbt_packages is the important one. It is what dbt deps writes and what the next dbt deps overwrites — the dbt equivalent of node_modules. Formatting it produces a diff against files you do not own and will not keep; on one 1,196-model project it accounted for 162 files.

This holds whether or not you have a dlab.json, because "include": ["**.sql"] — which is what dfmt init writes — would otherwise mean "and the dependencies too". To format one of them anyway, name it in include:

{ "include": ["**.sql", "dbt_packages/**.sql"] }

Naming it is the opt-in. A catch-all is not.

Configuration

dlab.json is optional in a dbt project — the dialect comes from the adapter type. Add one to set formatting options or to narrow which files are included:

{
"include": ["models/**/*.sql", "macros/**/*.sql"],
"dfmt": {
"keywordCase": "lower",
"maxLineWidth": 100
}
}
dfmt write .                    # every model
dfmt --model dim_customers # one, by name
dfmt write --changed-since main # only what this branch touched
dfmt write --no-jinja . # treat the files as plain SQL instead

When dfmt will not format a model

Two safeguards, and both leave the file untouched rather than guessing.

A template with no dbt project is refused. dfmt searches upward for dbt_project.yml from the file, not from your working directory. If there is none, the file is reported as a warning and skipped. This is not caution for its own sake: {% if x %} handed to a SQL parser does not fail, it lexes as operators and identifiers and yields a confident parse of a query nobody wrote.

A construct the reconstruction cannot place makes dfmt hand back the original. The template layer checks its own work — every construct present in the input must be present in the output, in the same order — and if that check fails, the model is emitted exactly as it came in, with a warning. Nothing is half-formatted.

Both outcomes count as not assessed, which fails dfmt check. A file nobody could assess is not a file that passed.

What "correct" is checked against

Formatting only moves whitespace, and dfmt verifies exactly that: the tokens of the output must be the same tokens, in the same order, as the tokens of the input. A run that would drop or reorder your SQL fails instead of writing.

The one exception is deliberate. clarifyStructure adds parentheses — a OR b AND c becomes a OR (b AND c) — which changes the token stream on purpose. Because adding a token is only safe if the parse tree really describes the input, it is switched off for any file where the parser had to recover, and the verification tolerates added parentheses and nothing else.