External dbt

Run your Git-based dbt project against the Embrasure warehouse and catalog from your own runner.

Keep your dbt project, Git repository, and scheduler. The dbt-embrasure adapter sends SQL to Embrasure's public API. Embrasure runs the queries and manages storage and catalog updates. You do not need AWS credentials or physical catalog names.

Install the adapter from PyPI. Access to the private product repository is not required.

Supported versions and scope

Use dbt Core 1.11 and Python 3.12 for the quickstart. The adapter supports Python 3.10 through 3.13. dbt Cloud and Fusion are not supported by this release.

Supported operations include SQL tables, views, ephemeral models, CSV seeds, data tests, docs generation, and incremental append and merge. Existing ref, source, Jinja, and packages still work subject to supported SQL and adapter macros. SQL or macros specific to another warehouse may need changes.

1. Prepare the warehouse and credentials

Create your Embrasure warehouse and output databases first, for example analytics_dev and analytics_prod. Writes use the default schema; dbt does not provision databases or custom output schemas.

For an unattended runner, have a workspace admin create an expiring personal access token for the same workspace with write scope, which includes read access. Do not grant admin scope to the runner. PATs must be enabled by workspace policy, and the creating user must retain at least editor access for warehouse writes.

Store the token in your CI secret manager as DBT_ENV_SECRET_EMBRASURE_TOKEN. Store the workspace UUID separately as EMBRASURE_WORKSPACE_ID. Never commit a token to profiles.yml or paste it into a command, screenshot, or job log.

The adapter does not refresh tokens. A short-lived CLI or browser session token is useful for an interactive test, not a stable credential for scheduled jobs. PATs are user-bound and workspace-scoped, not database-scoped: separate dev and production databases prevent naming collisions but are not an access boundary.

2. Point your existing project at Embrasure

Install into your project's virtual environment:

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install 'dbt-embrasure==0.1.0'

Keep your existing dependency lock process for reproducible production installs. Set profile: embrasure in your existing dbt_project.yml, then save this secret-free profile as .dbt/profiles.yml in the project:

embrasure:
  target: dev
  outputs:
    dev: &embrasure
      type: embrasure
      endpoint: https://api.embrasure.ai
      workspace_id: "{{ env_var('EMBRASURE_WORKSPACE_ID') }}"
      token: "{{ env_var('DBT_ENV_SECRET_EMBRASURE_TOKEN') }}"
      database: analytics_dev
      schema: default
      threads: 4
      query_timeout: 1800
    prod:
      <<: *embrasure
      database: analytics_prod

Inject the two environment variables through your secret manager, then run:

dbt deps
dbt debug --profiles-dir .dbt --target dev
dbt build --profiles-dir .dbt --target dev
dbt docs generate --profiles-dir .dbt --target dev

Use logical names from Embrasure's catalog for sources:

version: 2
sources:
  - name: crm
    database: raw
    schema: salesforce
    tables:
      - name: account

{{ source('crm', 'account') }} reads raw.salesforce.account. A model named accounts writes analytics_dev.default.accounts in dev and analytics_prod.default.accounts in prod. Ingestion destinations and outputs owned by managed Embrasure pipelines cannot be overwritten by external dbt.

3. Schedule a GitHub Actions job

In your dbt repository, create a GitHub environment called dbt-production. Add the token as an environment secret named DBT_ENV_SECRET_EMBRASURE_TOKEN and the workspace UUID as an environment variable named EMBRASURE_WORKSPACE_ID. Restrict that environment to your trusted main branch. Required environment reviewers will make scheduled jobs wait for approval, so choose that policy deliberately.

With your dbt project at the repository root, save this as .github/workflows/dbt.yml. Adjust the branch, cron, and paths for your project.

name: Build Embrasure models
on:
  workflow_dispatch:
  schedule:
    - cron: '17 6 * * *' # Daily at 06:17 UTC; GitHub may delay scheduled jobs.

permissions:
  contents: read

concurrency:
  group: embrasure-analytics-prod
  cancel-in-progress: false

jobs:
  dbt:
    if: github.ref == 'refs/heads/main'
    runs-on: ubuntu-latest
    timeout-minutes: 60
    environment: dbt-production
    steps:
      - uses: actions/checkout@93cb6efe18208431cddfb8368fd83d5badbf9bfd # v5
        with:
          persist-credentials: false
      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
        with:
          python-version: '3.12'
      - run: python -m pip install 'dbt-embrasure==0.1.0'
      - run: dbt deps
      - name: Build production models
        env:
          EMBRASURE_WORKSPACE_ID: ${{ vars.EMBRASURE_WORKSPACE_ID }}
          DBT_ENV_SECRET_EMBRASURE_TOKEN: ${{ secrets.DBT_ENV_SECRET_EMBRASURE_TOKEN }}
        run: |
          dbt debug --profiles-dir .dbt --target prod
          dbt build --profiles-dir .dbt --target prod

GitHub concurrency groups only coordinate jobs in the same repository. Use the same group for every workflow writing these outputs, and coordinate other repositories or schedulers separately. Do not run concurrent full builds against the same destination tables. This example does not run untrusted pull request code with production credentials or upload dbt artifacts automatically; compiled SQL and artifacts can contain sensitive information.

Your runner owns scheduling, Git checkout, job dependencies, overlap prevention, notifications, and retry decisions. Embrasure owns each query's authorization, execution, limits, history, and catalog reconciliation.

Rotate credentials without breaking jobs

Before expiration, create a replacement PAT in the same workspace with the same minimal scopes. Record its expiration in your team's normal reminder system.
Update the runner secret. Test dbt debug with the replacement and run a small build in a separate dev database before the next production job.
Allow jobs using the old token to finish, then revoke it through Access Embrasure → API → Personal access tokens. If it was exposed, revoke it immediately and investigate instead of waiting.

A 401 can mean an invalid, expired, or revoked token. A 403 can mean disabled PATs, insufficient scope, or changed workspace membership. Check these before retrying. Do not print the secret while diagnosing authentication failures.

Failure handling and initial limits

Tables and seeds rebuild with drop/create, not an atomic swap. A failed rebuild can leave a missing or partially populated destination.
Incremental append is the default; merge requires unique_key. Use --full-refresh for schema changes. Automatic schema evolution and snapshots are not supported.
Custom output schemas, renames, Python models, enforced contracts, SQL grants, persist_docs, and multi-statement transactions are not supported. Hooks autocommit.
query_timeout defaults to 1,800 seconds, with a server maximum of 3,600 per query. The existing scan limit still applies. A job timeout is separate from a query timeout; stopping a runner does not guarantee an immediate server cancellation.
After a lost submission response, inspect Embrasure query history before retrying, especially an append. The adapter deliberately does not replay writes automatically, because the first submission may already have succeeded.

Use a dedicated dev database to verify an existing project before moving its production schedule to Embrasure.