Getting started¶
You need Python 3.11 or newer, or Docker. You do not need dbt installed or any warehouse credentials.
Try the demo first¶
pip install openpipeline
openpipeline demo
Open http://127.0.0.1:8484. The demo uses a copy of the jaffle_shop example, so it never touches your files. It already has an agent proposal waiting in the Proposals tab. See the demo for options.
Install¶
pip install openpipeline
openpipeline serve path/to/your/dbt/project
Optional extras:
pip install 'openpipeline[ai]' # AI explanations (your own Anthropic key)
pip install 'openpipeline[preview]' # sample rows, run locally with DuckDB
uv works too: uvx openpipeline serve path/to/project.
docker run --rm -p 8484:8484 -v "$PWD:/project" ghcr.io/pawanpatil19/openpipeline
Run this from your dbt project folder. The image includes the sample rows
and AI extras. For AI explanations, pass your
key with -e ANTHROPIC_API_KEY.
The container listens on all interfaces, so saving changes needs a token. It
prints one when it starts (docker logs <container>), with a link that opens
the UI ready to edit. To choose it, add -e OPENPIPELINE_TOKEN=.... For a
server that refuses every change, add -e OPENPIPELINE_READ_ONLY=1. See
Write access and security.
Runs the web UI and the MCP server together:
DBT_PROJECT_DIR=~/code/my-dbt-project docker compose up
The UI is on http://localhost:8484 and the MCP server on
http://localhost:8485/mcp. Without DBT_PROJECT_DIR it uses the bundled
example. Set OPENPIPELINE_TOKEN so both services share one token:
OPENPIPELINE_TOKEN=choose-something-long docker compose up.
Point it at your dbt project¶
Give OpenPipeline the folder that holds dbt_project.yml:
openpipeline serve ~/code/my-dbt-project
If you leave the path out, it uses the current folder, or the folder in the
OPENPIPELINE_PROJECT_DIR environment variable.
The server notices when you edit a SQL, YAML, or CSV file and reads the project again on the next request. The page fetches fresh data when you switch back to its browser tab, so you can keep your editor open next to it.
Manifest or files¶
OpenPipeline can read your project two ways. The top bar of the UI says which one it used ("read from manifest" or "read from files").
target/manifest.json |
Plain files | |
|---|---|---|
| When it is used | Whenever target/manifest.json exists |
When there is no manifest |
| Jinja and macros | Already resolved by dbt | ref(), source(), and config() are resolved; macro calls are shown by name; only the first branch of an if is read |
| Stories | Written from dbt's compiled SQL when the manifest has it | Written from the file with Jinja simplified |
| Setup needed | Run dbt compile |
None |
| Best for | Projects that use macros, packages, or var() |
A quick look, or a repo without dbt installed |
If your project uses macros, run dbt compile first and OpenPipeline will pick up
the manifest on its own, and use the compiled SQL in it for the stories and column
lineage. (dbt parse also writes a manifest, but without compiled SQL, so
stories fall back to the files.) Re-run it when the project changes, because the
manifest is a snapshot.