OpenLineage types

Type the OpenLineage events, datasets, column facets, batches, and delivery receipts accepted by Embrasure.

Embrasure accepts OpenLineage 2.x run, job, and dataset events. These TypeScript definitions describe the fields Embrasure reads; standard OpenLineage facets may include additional properties.

Event union

type OpenLineageEvent = RunEvent | JobEvent | DatasetEvent;

type OpenLineageEventType =
  | "START"
  | "RUNNING"
  | "COMPLETE"
  | "ABORT"
  | "FAIL"
  | "OTHER"
  | (string & {});

interface EventBase {
  eventType?: OpenLineageEventType;
  /** ISO-8601 timestamp with a timezone, for example 2026-08-05T18:30:00Z. */
  eventTime?: string;
  producer?: string;
  schemaURL?: string;
}

interface RunEvent extends EventBase {
  run: OpenLineageRun;
  job: OpenLineageJob;
  inputs?: OpenLineageDataset[];
  outputs?: OpenLineageDataset[];
}

interface JobEvent extends EventBase {
  run?: never;
  job: OpenLineageJob;
  inputs?: OpenLineageDataset[];
  outputs?: OpenLineageDataset[];
}

interface DatasetEvent extends EventBase {
  run?: never;
  job?: never;
  dataset: OpenLineageDataset;
}

If either job or run is present, job.namespace and job.name are required. A run event also requires run.runId. An event without a job or run must contain dataset.

Jobs, runs, and datasets

type OpenLineageFacets = Record<string, unknown>;

interface OpenLineageRun {
  runId: string;
  facets?: OpenLineageFacets;
}

interface OpenLineageJob {
  namespace: string;
  name: string;
  facets?: OpenLineageFacets;
}

interface OpenLineageDataset {
  namespace: string;
  /** Non-empty; at most 1,000 characters. */
  name: string;
  facets?: OpenLineageFacets & {
    columnLineage?: ColumnLineageFacet;
    symlinks?: DatasetSymlinksFacet;
  };
  version?: string;
}

interface DatasetSymlinksFacet {
  identifiers?: Array<{
    namespace: string;
    name: string;
    type: string;
  }>;
  [key: string]: unknown;
}

Across inputs and outputs, one event may contain at most 2,000 datasets. Embrasure uses each dataset's case-sensitive namespace and name for resolution. A symlink identifier with type: "TABLE" can supply an alternate table identity.

An input-only job event represents a service that reads the listed datasets. Embrasure identifies that service by job.namespace and job.name.

Column lineage

Attach columnLineage to an output dataset. Each input field must identify a dataset in the same event's inputs array.

interface ColumnLineageFacet {
  fields: Record<string, {
    inputFields: Array<{
      namespace: string;
      name: string;
      field: string;
      transformations?: Array<{
        type?: string;
        subtype?: string;
        description?: string;
        masking?: boolean;
        [key: string]: unknown;
      }>;
    }>;
    transformationDescription?: string;
    transformationType?: string;
    [key: string]: unknown;
  }>;
  [key: string]: unknown;
}

Embrasure classifies mappings as direct, expression, or aggregate from the transformation metadata. Unmatched input datasets or empty input field names are ignored rather than guessed.

Batch request

The batch endpoint accepts either a bare array or an object envelope:

type OpenLineageBatchRequest =
  | OpenLineageEvent[]
  | { events: OpenLineageEvent[] };

A batch contains 1 to 1,000 events. Every child must be a JSON object; non-object children are quarantined and counted in the receipt.

Delivery receipt

Both ingestion endpoints return 202 Accepted with the same receipt:

interface LineageDeliveryReceipt {
  delivery_id: string;
  state: "accepted" | "quarantined";
  duplicate: boolean;
  item_count: number;
  invalid_item_count: number;
}

accepted means the bytes were durably stored for asynchronous validation and projection; it does not mean every dataset resolved or every edge reached the served graph. duplicate is true when the same delivery was already accepted. For a batch, invalid_item_count reports children rejected at the transport boundary.

Receiver bounds

Constraint	Limit
HTTP request body	8 MiB
Batch events	1 to 1,000
Datasets per event	2,000 across inputs and outputs
JSON nesting depth	64
JSON values per event	100,000
Facet data per event	1 MiB
`Idempotency-Key`	1 to 600 characters
`provider_key`	1 to 500 characters

Duplicate JSON object keys and compressed request bodies are rejected. See ingest one event or ingest a batch for transport details.