Native Engine

The native engine runs Honeyframe recipes directly against the warehouse with no dbt toolchain. It is one of two transformation engines an organization can run on, selected by the org-level engine_profile column (dbt — the default — or native). On a native org, visual and SQL recipes are stored as target='sql' with Jinja-free inline SQL and materialized by compiling each recipe to a single statement: CREATE TABLE AS for tables, CREATE OR REPLACE VIEW for views, run straight against the tenant warehouse (Postgres and DuckDB).

Because there is no dbt binary in the loop, a native install skips the dbt pip-install, the generated profiles.yml, and the dbt-run timer entirely. The same recipes, lineage, and Flow canvas work on both engines — recipe authoring is identical — so most of the rest of the documentation is engine-agnostic. This page is the focused explainer for what changes when an org runs native.

GET /api/catalog/engine reports the active profile, and the SDK exposes Project.engine() / Project.uses_dbt() so automation can detect which engine it is talking to. See Developer → SDK.

dbt vs native

Both engines run the same visual and SQL recipes and produce the same datasets and lineage. The difference is the execution path and its scope.

	dbt (default)	native
Transformation engine	dbt — dbt run	Built-in SQL engine — CREATE TABLE AS / CREATE OR REPLACE VIEW
Warehouses	dbt-supported targets	Postgres + DuckDB
Materializations	Full incremental models, snapshots, macros, tests	Full-refresh tables and views only
Toolchain	dbt pip-install, profiles.yml, dbt-run timer	None
Lineage source	dbt manifest (DbtManifestCatalog)	honeyframe.datasets + honeyframe.dataset_edges (NativeCatalog)

Pick native for lean installs that do not want a dbt toolchain — fewer moving parts, no manifest, faster cold start. Pick dbt when you need anything beyond full-refresh tables and views (incremental models, snapshots, macros, tests) or when a team already owns a dbt project. Anything richer than full-refresh tables/views stays on the dbt engine.

See Deployment Tiers → Engine profile: dbt vs native for how the profile sits relative to the Cloud / Enterprise / Self-Hosted tiers (it is independent of the deployment tier).

How native builds run

POST /api/flow/build-native (requires project.edit, native orgs only) is the native equivalent of dbt run. It topologically sorts the project's target='sql' recipes by their dataset inputs and outputs, then runs each in dependency order through the recipe runner, stopping on the first failure. The plan resolver is pure, stable, and cycle-detecting — a cycle returns 400.

Three query options mirror dbt's most-used flags:

• ?select= — partial rebuild of a model's subgraph using dbt selector syntax: model, model+ (and downstream), +model (and upstream), +model+ (full lineage), comma-separated for several terms. The selected set is built in original order, so the topo sort stays deterministic. An unknown model is a 400 — a typo is surfaced loudly, not silently built as nothing. Omitting select builds the whole project.
• ?background=true — resolves the plan synchronously (engine, selector, and cycle errors still fail fast), records a native_build job run, then runs the ordered recipes on a fresh session and returns {run_id, status: "running"} immediately. Poll the existing GET /api/jobs/{run_id} for progress — there is no new polling endpoint.
• ?dry_run=true — resolves the plan (engine gate, selector, topo order) and returns {ordered, total, selected, dry_run: true} without touching the warehouse — the native dbt ls --select. Pairs with ?select= to preview exactly which recipes a partial rebuild would run, and in what order, before committing (for example before kicking off a background build). Dry-run is checked before background, so it wins if both are set.

Readiness and conversion

Two endpoints support moving an org onto native:

• GET /api/flow/recipes/native-readiness returns a ready/blocked verdict with per-recipe buckets, so you can see which recipes are not yet native-runnable.
• POST /api/flow/recipes/convert-to-sql bulk-converts existing recipes to native target='sql' form.

The intended loop is native-readiness → convert-to-sql → flip the engine → build-native.

Lineage without a manifest

On dbt orgs, lineage resolves through the dbt manifest. Native orgs have no manifest, so lineage is served by NativeCatalog, backed by:

• honeyframe.datasets — the per-tenant dataset registry (the model list),
• information_schema — column metadata for each dataset,
• honeyframe.dataset_edges — a first-class many-to-many table of dependency edges.

The edges are written automatically: successful recipe runs and Flow canvas saves upsert into honeyframe.dataset_edges, so a native org gets the DAG and upstream/downstream lineage with no admin step and no manifest. Edge-write failures are swallowed and logged rather than failing the run.

The catalog layer is a pluggable adapter — DbtManifestCatalog for dbt tenants and NativeCatalog for native tenants — behind the same ModelCatalog interface, so lineage reads, dataset listing, governance/PII summaries, and the DAG all work on both engines. Column-level lineage endpoints (for example GET /api/column-lineage/{model}) are a dbt-only artifact with no native equivalent today; on native tenants they short-circuit with a message rather than crashing. See Connectors → Catalog adapters for per-org adapter configuration.

Choosing the engine

The profile is stored in organizations.engine_profile with a CHECK (engine_profile IN ('dbt','native')) constraint and defaults to dbt, so existing installs are unchanged.

Born native at install (Self-Hosted). Set the profile at provisioning time with the standard installer:

./iaas/scripts/setup-customer.sh --engine native
# or, in install.conf:
#   engine_profile: native

The column is folded into the organizations CREATE TABLE (not just a migration), so it exists on fresh installs and --engine native cannot silently fall back to dbt. A native provision skips the dbt workspace seed, the dbt pip-install, the profiles.yml, and the dbt-run timer.

Born native on Cloud / Enterprise. POST /api/organizations accepts engine_profile (dbt | native, default dbt) at org-creation time, so a tenant can be created directly on the native engine.

Flipping an existing org. PUT /api/catalog/engine changes the profile of a live org. A flip to native is guarded by an org-wide recipe-readiness check: it returns 409 if any SQL-shaped recipe in the org is not native-runnable. Pass ?force to override the guard. Flipping back to dbt is unguarded.

Authoring on native

Recipe authoring is the same on both engines — you build visual and SQL recipes on the Flow canvas exactly as documented in Recipes. The difference is purely under the hood: on a native org, saving a recipe stamps target='sql' with Jinja-free inline_code, and the native engine materializes it honoring the author's view/table choice and target_schema (rather than emitting a dbt model).

From the SDK, author native transforms via the same recipe-save methods (for example save_prepare_steps() / save(recipe_type=…)); the server materializes through the native engine according to the org's engine profile, with no per-call flag. Project.engine() / Project.uses_dbt() let a single automation script drive either engine.

API reference

Endpoint	Description
GET /api/catalog/engine	The org's active engine profile (dbt | native).
PUT /api/catalog/engine	Flip the org's engine profile. To-native 409s unless recipes are ready (or ?force); to-dbt is unguarded.
POST /api/flow/build-native	Native dependency-order build (native orgs). Supports ?select=, ?background=true, ?dry_run=true.
GET /api/flow/recipes/native-readiness	Ready/blocked verdict (with per-recipe buckets) for converting an org to native.
POST /api/flow/recipes/convert-to-sql	Bulk-convert existing recipes to native target='sql'.
POST /api/organizations	Create an org; accepts engine_profile (dbt | native, default dbt).
GET /api/jobs/{run_id}	Poll a background native_build job started with ?background=true.