Runners
A runner is the program that reads your SQL files and sends them to the database: dbt, Flyway,
psql, SQL*Plus, Qlik. Each one reads a file its own way. psql treats \set as a command, Flyway
fills in ${schema}, dbt renders Jinja. dfmt reads the file the way its runner does, because a
file formatted the way nothing will read it can look valid and still not run.
How dfmt picks the runner
In order:
- The
"runner"of the folder indlab.json, when you set one. A runner you write down is never second-guessed. - A runner's project file above the file:
dbt_project.yml,flyway.tomlorflyway.conf,schemachange-config.yml,bqetl_project.yaml, or an extension's<name>.control. dfmt walks up from the file it is formatting, not from your working directory, and the nearest one wins. - The dialect's own client: psql for PostgreSQL, sqlcmd for SQL Server, SQL*Plus for Oracle, and so on.
{
"sources": [
{ "path": "transform", "runner": "dbt" },
{ "path": "db/migration", "dialect": "postgres", "runner": "flyway" },
{ "path": "scripts", "dialect": "postgres", "runner": "psql" },
{ "path": "adhoc", "dialect": "postgres", "runner": "none" }
]
}
The first entry names a folder holding dbt_project.yml, so dbt says the rest: which folders are
its files and which database it is for. The others are folders, so they name their dialect. Every
runner a dialect can have is listed in dfmt setup, and dfmt setup <path> says which runner a
file got and why.
dfmt setup writes these entries for you. A file with Jinja and no runner that renders Jinja is
refused, not guessed at: {% if x %} handed to an SQL parser does not fail, it lexes as operators
and names and yields a confident reading of a query nobody wrote.
dbt
A model is a template that becomes SQL. dfmt evaluates it with its own Jinja engine and dbt
compiler, in the model's own context (ref(), source(), config(), var(), is_incremental(),
project and package macros). It lays out the SQL that results and puts every tag back where it was
written. Both arms of an {% if %} are formatted. dbt Integration covers
what survives, what is refused, and how.
Flyway
Flyway parses its migrations itself, and dfmt reads the project the way Flyway does:
- Configuration from
flyway.toml(read strictly: what dfmt does not implement, it refuses rather than skips) orflyway.conf(key=value, with${ENV}expansion). That includesflyway.locations, where aclasspath:location is reported as unreachable rather than read as empty, and a per-migration<migration>.sql.confsidecar. - The dialect from the JDBC url. An engine dfmt does not know gives no dialect, never ANSI.
- Which files are migrations, by name, as Flyway's own resolver reads them.
V2_3__desc.sqlis versioned (underscores are dots, andV2_3comes beforeV2_10),R__desc.sqlis repeatable,Uis undo,afterMigrate__x.sqland its siblings are callbacks. Anything else is not a migration. - Placeholders. A placeholder like
${schemaName}can only stand where one value stands, so dfmt masks it with a stand-in of the same width, formats the file, and puts the placeholder back. Only clear text is masked:'${x}'inside a string literal is content.flyway.placeholderReplacement = falseturns masking off. - Statement boundaries are Flyway's, so
statementTerminatoris refused in a Flyway project.
Client scripts
Most dialects reach their engine through a client program, and a file for that client is a
script. Its commands sit between the statements: psql's \timing, sqlcmd's GO, SQL*Plus's /,
mysql's DELIMITER, db2's --#SET TERMINATOR. dfmt formats the statements and leaves the
commands where they are. The options that act between statements (linesBetweenStatements,
blankLines) act on the boundaries the client sees.
In a psql script, :schema is a variable and \copy keeps its line:
\set schema reporting
select * from :schema.orders where id = :'order_id';
\copy (select 1) to 'out.csv'
In SQL*Plus, &®ion is a substitution and / runs the block before it, which formats as
code. Statement Terminator is the one option that exists for
scripts alone.
The clients dfmt reads are psql, sqlcmd, mysql, SQL*Plus, snowsql, the db2 command line, duckdb, sqlite3, trino, athena and beeline.
Every other runner
| Runner | "runner" | What dfmt does with its files |
|---|---|---|
| schemachange | schemachange | Renders the Jinja with the vars from schemachange-config.yml, like a dbt model, and reads the result as Snowflake. |
| bqetl (Mozilla's BigQuery ETL) | bqetl | Renders each query's Jinja in the context the bqetl tool gives it, found from bqetl_project.yaml. |
| PostgreSQL extension | pg-extension | Reads the scripts CREATE EXTENSION runs as the server does: @extschema@ and its siblings are kept, and only \echo lines are a client's. |
| Qlik | qlik | Keeps every dollar-sign expansion ($(vDays), $(=Today())) as written and formats the SQL around it. Nothing marks a folder Qlik reads from, so you name it in dlab.json. |
| none | none | The database reads the file whole, with no client's commands, and its grammar decides where each statement ends. |
What counts as a project's own files
dfmt skips dbt_packages, target, node_modules, .venv, venv and .git unless an include
pattern names them. dbt_packages matters most: dbt deps writes it and the next dbt deps
overwrites it, so formatting it produces a diff against files you do not own.