Replicate Snowflake tables into the managed warehouse with Snowflake streams, including updates and deletes.
Snowflake ingestion copies the selected tables into the managed warehouse, then checks for changes every 15 minutes. Embrasure captures inserts, updates, and deletes with a Snowflake stream on each table. Checks that find no changes use Snowflake metadata only, so they do not resume your warehouse.
To query Snowflake in place instead of copying its tables, connect it as an existing warehouse.
Use a dedicated service user and role, and an auto-suspending warehouse for extraction. An X-Small warehouse with a 60-second auto-suspend suits most tables. Snowflake bills at least 60 seconds each time a warehouse resumes.
The ingestion role needs:
USAGE on the warehouse, the source database, and the selected schemasSELECT on the selected tablesUSAGE, CREATE TABLE, and CREATE STREAM on the EMBRASURE_INGESTION schema in the source databaseEMBRASURE_INGESTION holds the streams and staging tables Embrasure creates. Create the schema in advance, or allow the role to create it.
Each table owner must also enable change tracking on the selected tables:
-- Replace the database, schema, role, and table names with your own.
CREATE SCHEMA IF NOT EXISTS ANALYTICS.EMBRASURE_INGESTION;
GRANT USAGE, CREATE TABLE, CREATE STREAM ON SCHEMA ANALYTICS.EMBRASURE_INGESTION TO ROLE EMBRASURE_INGESTION;
ALTER TABLE ANALYTICS.PUBLIC.ORDERS SET CHANGE_TRACKING = TRUE;The read-only role used to query Snowflake in place is not enough for replication, because it cannot create streams.
In Embrasure, open Warehouse → Ingestion. Select Connect warehouse source, or New sync if the workspace already has ingestion syncs, then choose Snowflake.
Enter the account identifier (the part before .snowflakecomputing.com), user, warehouse, and database. Each sync reads from one source database. Role is optional; generated setup uses EMBRASURE_INGESTION when it is blank.
Generated key-pair setup is recommended. Embrasure creates an RSA key pair, keeps the private key encrypted, and shows one SQL block for an administrator. That block creates the role and user, grants read access and the EMBRASURE_INGESTION privileges, and installs the public key. A role-restricted programmatic access token also works.
Choose the tables and a destination database, then run preflight. Preflight blocks tables the role cannot see, tables with change tracking turned off, and unsupported columns. Start the sync once every selected table is ready. A missing EMBRASURE_INGESTION grant shows up as an error on the first sync.
The first sync copies every row. Later syncs copy only rows that changed since the previous sync.
Each table gets a dedicated standard stream. Embrasure identifies rows by Snowflake's stream row ID, stored as embrasure_source_row_id. Tables without a primary key, and tables with duplicate rows, replicate correctly. Updates replace the previous version of a row. Deleted rows are retained as tombstones with _embrasure_deleted = true; filter on it to read only current rows.
When a check finds changes, Embrasure copies that interval into a staging table in EMBRASURE_INGESTION with one atomic statement, using your warehouse. It pages through the staging table and advances only after the destination write succeeds. Copying an interval may take up to 40 minutes. If a very large initial copy needs longer, the sync asks for a larger warehouse; an X-Small warehouse copied 400 million narrow rows in about five minutes. A failed or retried sync repeats the same interval. It never skips changes or captures them twice.
The initial copy commits up to 100,000 rows at a time and sustains about 2,000 rows per second per table. In our tests a 6-million-row table took about an hour to copy the first time. Several selected tables copy at the same time. After the initial copy, each sync writes only the rows that changed.
| Snowflake type | Warehouse representation |
|---|---|
NUMBER, DECIMAL | Decimal with the declared precision and scale |
FLOAT | Double |
BOOLEAN, DATE | Boolean, date |
TIMESTAMP_NTZ | ISO 8601 text with nanoseconds and no offset |
TIMESTAMP_LTZ, TIMESTAMP_TZ | ISO 8601 text with nanoseconds and an offset |
TIME | Text with nanoseconds |
VARIANT, OBJECT, ARRAY | Compact JSON text; SQL NULL array elements become null |
BINARY | Hexadecimal text |
VARCHAR, CHAR | Text |
Timestamps are kept as text because the warehouse timestamp type cannot hold nanoseconds. Columns of other types, such as GEOGRAPHY, and columns whose names need quoting must be excluded before syncing. Column names are lowercased in the warehouse.
New source columns are picked up at the next change interval. Existing rows keep null in a new column until they change; run a full resync to backfill it. Dropped, renamed, or incompatible columns wait for review in the sync.
A paused sync stops reading its streams. Snowflake marks a stream stale once it has gone unread longer than the table's retention period, which is extended to 14 days by default. Resuming after that asks for a full resync. Embrasure never silently skips the missed changes.
A full resync creates a new stream, copies the table again, and replaces the destination table once the copy completes.
Deleting a sync also drops the streams and staging tables it created in EMBRASURE_INGESTION. Objects belonging to other syncs, and your source tables, are not touched. If Embrasure cannot reach Snowflake, the sync is still deleted and Embrasure lists the objects for an administrator to drop.
Imported warehouse tables remain after the sync is deleted. Use the workspace's deletion controls or contact privacy@embrasure.ai to remove them, subject to the retention policy.
Embrasure stores the replicated tables and operational metadata: selected tables, checkpoints, row counts, sync status, errors, preflight results, and Snowflake query IDs for each read.