Skip to main content

Runners

A runner is the program that reads your SQL files and sends them to the database: dbt, Flyway, psql, SQL*Plus, Qlik. Each one reads a file its own way. psql treats \set as a command, Flyway fills in ${schema}, dbt renders Jinja. dfmt reads the file the way its runner does, because a file formatted the way nothing will read it can look valid and still not run.

How dfmt picks the runner​

In order:

  1. The "runner" of the folder in dlab.json, when you set one. A runner you write down is never second-guessed.
  2. A runner's project file above the file: dbt_project.yml, flyway.toml or flyway.conf, schemachange-config.yml, bqetl_project.yaml, or an extension's <name>.control. dfmt walks up from the file it is formatting, not from your working directory, and the nearest one wins.
  3. The dialect's own client: psql for PostgreSQL, sqlcmd for SQL Server, SQL*Plus for Oracle, and so on.
{
"sources": [
{ "path": "transform", "runner": "dbt" },
{ "path": "db/migration", "dialect": "postgres", "runner": "flyway" },
{ "path": "scripts", "dialect": "postgres", "runner": "psql" },
{ "path": "adhoc", "dialect": "postgres", "runner": "none" }
]
}

The first entry names a folder holding dbt_project.yml, so dbt says the rest: which folders are its files and which database it is for. The others are folders, so they name their dialect. Every runner a dialect can have is listed in dfmt setup, and dfmt setup <path> says which runner a file got and why.

dfmt setup writes these entries for you. A file with Jinja and no runner that renders Jinja is refused, not guessed at: {% if x %} handed to an SQL parser does not fail, it lexes as operators and names and yields a confident reading of a query nobody wrote.

dbt​

A model is a template that becomes SQL. dfmt evaluates it with its own Jinja engine and dbt compiler, in the model's own context (ref(), source(), config(), var(), is_incremental(), project and package macros). It lays out the SQL that results and puts every tag back where it was written. Both arms of an {% if %} are formatted. dbt Integration covers what survives, what is refused, and how.

Flyway​

Flyway parses its migrations itself, and dfmt reads the project the way Flyway does:

  • Configuration from flyway.toml (read strictly: what dfmt does not implement, it refuses rather than skips) or flyway.conf (key=value, with ${ENV} expansion). That includes flyway.locations, where a classpath: location is reported as unreachable rather than read as empty, and a per-migration <migration>.sql.conf sidecar.
  • The dialect from the JDBC url. An engine dfmt does not know gives no dialect, never ANSI.
  • Which files are migrations, by name, as Flyway's own resolver reads them. V2_3__desc.sql is versioned (underscores are dots, and V2_3 comes before V2_10), R__desc.sql is repeatable, U is undo, afterMigrate__x.sql and its siblings are callbacks. Anything else is not a migration.
  • Placeholders. A placeholder like ${schemaName} can only stand where one value stands, so dfmt masks it with a stand-in of the same width, formats the file, and puts the placeholder back. Only clear text is masked: '${x}' inside a string literal is content. flyway.placeholderReplacement = false turns masking off.
  • Statement boundaries are Flyway's, so statementTerminator is refused in a Flyway project.

Client scripts​

Most dialects reach their engine through a client program, and a file for that client is a script. Its commands sit between the statements: psql's \timing, sqlcmd's GO, SQL*Plus's /, mysql's DELIMITER, db2's --#SET TERMINATOR. dfmt formats the statements and leaves the commands where they are. The options that act between statements (linesBetweenStatements, blankLines) act on the boundaries the client sees.

In a psql script, :schema is a variable and \copy keeps its line:

\set schema reporting
select * from :schema.orders where id = :'order_id';
\copy (select 1) to 'out.csv'

In SQL*Plus, &&region is a substitution and / runs the block before it, which formats as code. Statement Terminator is the one option that exists for scripts alone.

The clients dfmt reads are psql, sqlcmd, mysql, SQL*Plus, snowsql, the db2 command line, duckdb, sqlite3, trino, athena and beeline.

Every other runner​

Runner"runner"What dfmt does with its files
schemachangeschemachangeRenders the Jinja with the vars from schemachange-config.yml, like a dbt model, and reads the result as Snowflake.
bqetl (Mozilla's BigQuery ETL)bqetlRenders each query's Jinja in the context the bqetl tool gives it, found from bqetl_project.yaml.
PostgreSQL extensionpg-extensionReads the scripts CREATE EXTENSION runs as the server does: @extschema@ and its siblings are kept, and only \echo lines are a client's.
QlikqlikKeeps every dollar-sign expansion ($(vDays), $(=Today())) as written and formats the SQL around it. Nothing marks a folder Qlik reads from, so you name it in dlab.json.
nonenoneThe database reads the file whole, with no client's commands, and its grammar decides where each statement ends.

What counts as a project's own files​

dfmt skips dbt_packages, target, node_modules, .venv, venv and .git unless an include pattern names them. dbt_packages matters most: dbt deps writes it and the next dbt deps overwrites it, so formatting it produces a diff against files you do not own.