v0.0.67 — Prepare-recipe F12 + catalog AI Search

Released: 2026-05-13. Thirteen commits.

Recipes — F12 foundation

The prepare-recipe path got a substantial backend rewrite so Cobuild can author recipes step-by-step on a robust contract.

Sort step actually sorts

The UI exposed a Sort editor and persisted the step in JSONB, but the compiler had no sort case — client-side preview looked right, the materialized table came back unordered. The TypeScript and Python compilers now emit ORDER BY after dedup/unpivot, with multi-step accumulation supported.

Preview masking + server-side step validation

/flow/preview-sql was the one read path bypassing the masker. It now runs through apply_masking with name-inferred semantic types, mirroring datasets / dashboards / chat. The /flow/save-recipe-steps endpoint also gains a server-side validator (services/prepare_step_compiler.py, ported from the TS compiler) that returns step_index + field-level errors before anything reaches dbt.

Optimistic locking + transactional save

flow_recipes gains an integer version column with optimistic locking. The new POST /flow/recipes/save-prepare collapses the legacy 3-call save (generate-sql → PUT /dbt/file → save-recipe-steps) into one transactional endpoint. Stale-UI conflicts return 409 with the server version so the caller can reload and reconcile — same shape as Dataiku's stale-view toast.

Server-side per-step preview

PrepareRecipePage's 115-line client-side JS simulation is gone. POST /flow/recipes/preview-up-to-step now compiles steps [0..target], executes the SQL, applies masking, and returns the same rows the dbt-materialized recipe will produce. Selecting a step pins up_to_step_id so the preview reflects the data at that step. New header affordances: "updating preview" badge, "at step N" pill, "🔒 N masked cols" pill, error row.

Cobuild step CRUD tools

Closes F12 with five new tools the Cobuild planner and dashboard chat agent can use to edit prepare recipes step-by-step: get_recipe_steps, add_step, remove_step, move_step, patch_step_config. Mutations bump flow_recipes.version, so an in-flight UI save will 409 if the agent landed an edit in between. Read tool is approval-free; the four mutations are approval-gated. V1 is recipe_type='prepare' only.

Catalog — AI Search + Genie

Dataiku parity slice for the dataset catalog.

• AI Dataset Search. POST /api/datasets/ai-search wraps services.catalog_discovery.discover_datasets, now accepting project_id=None for cross-project NL search across the whole org. New AISearchPage renders ranked cards with rationale, primary columns, tags, row count, score. Entry-point button on DatasetListPage.
• Genie — Ask a dataset. POST /api/datasets/{name}/ask scopes chat_service.generate_sql to one dataset's schema. The allowed_schemas gate rejects cross-tenant joins even if the LLM hallucinates one. New GenieTab on DatasetDetailPage with quick-question chips, collapsible SQL, follow-up chips.
• Dataset detail drawer. Clicking an AI Search result opens a right-side drawer with a compact Genie input embedded. "Open full page →" forwards the technical dataset name (preserves the F7 friendly-vs-technical pin). Esc-to-close registered globally.

AI Search fail-closed fix

If the user landed on /projects/:slug/datasets/ai-search before useProjectKeySync had stamped selected_project_id, the dispatcher passed projectId=null to AISearchPage — silently widening the query to every project in the org while the URL still claimed project scope. The page now renders a brief "Loading project context…" placeholder when projectSlug is present but selected_project_id isn't.

Datasets — origin filter chip strip

Every DatasetListPage row gains an origin filter strip (uploaded / connected / derived / external) parallel to the existing layer chips. Empty buckets are pruned so upload-only or dbt-only projects see a tight strip. /api/datasets accepts ?origin= and returns an origins index alongside layers + tags.

Prepare profiler — F13 V1

New POST /api/flow/recipes/profile-columns returns per-column total / null_count / null_pct / distinct_count / min / max in one warehouse round-trip, sample-bounded to 100k rows. Step-aware (compiles + slices through up_to_step_id). PII masking mirrors the preview endpoint — min/max for pii_email / pii_phone columns are masked before they leave the server. Gated org.admin; 50 columns / call max.

Agent coordination docs

CLAUDE.md gains a Release & Tagging section; AGENTS.md is symlinked to it so Codex reads the same project rules. New "Working alongside another agent" section covers safe behavior when Claude Code and Codex are active on the same repo (no stash/reset to clear unexpected changes, commit at task boundaries).

Runbook fix

The cleanup-test-orgs runbook §3 missed two RESTRICT FK violations (dashboards_project_id_fkey, dashboards_owner_user_id_fkey). §3 is replaced with the full-fanout delete script that purged 9 orgs in one transaction, plus the FK introspection query for finding future tables that need adding.