Scheduler

The Scheduler is Honeyframe's single surface for time-driven automation: dataset refreshes, recipe builds, AI-agent runs, and email reports. It replaces three separate v0.0.37 surfaces (pipeline_schedules, report_schedules, and the per-project cron table) consolidated into one in v0.0.38.

Concepts

A schedule binds a trigger to a scenario. A scenario is a sequence of one or more steps.

Term	Meaning
Trigger	When the scenario fires. Cron expression, interval, or "on upstream success".
Scenario	The unit of work the scheduler executes. One scenario can have many steps.
Step	A single action — build a dataset, run a recipe, sync a folder, send a report, run a script.

Step types

Every step carries a type and a config object. The scheduler refuses a step whose type is not one of the types below (or a plugin-provided plugin:<plugin_id>:<component> step), so a typo fails loudly instead of being skipped.

Build and transform

Step	What it does
run_recipe	Run an existing Flow recipe with its own run configuration (writer, watermark, and so on).
build_dataset	Build one dataset's model (dbt run --select <model>) in the project's dbt workspace. Honors build-mode toggles (upstream / downstream / full lineage) and a forced rebuild. Retries transient failures.
build_zone	Build one or more Flow zones: resolves each zone's terminal datasets and runs them as a single build. Config zone_ids (or zone_id) plus optional force_build; zones from another project are rejected.
dbt_run	Run the project's full dbt build as one step. Retries transient failures.
backfill_dataset	Rebuild a partitioned dataset partition by partition, in order, over a start / end range.
clear_dataset	Truncate the table backing a registered dataset of this project.
swap_datasets	Atomically swap the backing tables of two datasets in the same project (blue/green publish).
propagate_schema	Walk a changed source schema downstream through the recipe graph, mark affected recipes stale and, with apply: true, re-baseline safe (additive) changes.
run_sql	Run a read-only SELECT scoped to the scenario's own project data.
set_project_vars	Set or override project variables for the rest of the run.
invalidate_cache	Drop cached column statistics for the project (or one dataset) so the next profile read recomputes them.

Ingestion and sync

Step	What it does
extract	Run one of the platform's bundled ingestion scripts in a subprocess.
extract_documents	Run the managed-folder document-extraction pipeline into a target table.
sync_sharepoint	Sync a SharePoint library into a managed folder.
sync_google_drive	Sync a Google Drive folder into a managed folder.
sync_knowledge_bank	Re-ingest (and optionally re-embed) a Knowledge Bank.
sync_dataset_catalog	Reconcile the project's dbt manifest into the dataset catalog.
catalog_refresh	Crawl one or more configured catalog adapters and refresh their assets.
trigger_orchestrator	Fire a run on an external orchestrator (Airflow or Dagster) through the org's connector.
update_git_refs	Fetch the project workspace's Git remote and, with pull: true, fast-forward the current branch.

Quality and monitoring

Step	What it does
verify_rules	Run the project's data-quality rules and report pass/fail counts.
compute_metrics	Compute a dataset's metrics and evaluate their threshold checks.
check_schema_drift	Fail the scenario when a dataset's live schema drifts from its captured contract.
sla_check	Evaluate the project's dataset freshness SLAs and notify on breaches.
detect_anomalies	Run the org's alert rules over recent run history and raise alerts. One rule type, cost_budget, flags a model whose recent average duration creeps above a multiple of its baseline.
classify_pii_columns	Run the PII classification scan; proposals go to steward review.
check_monitors	Poll configured external monitors and fail the step when the worst status crosses fail_on, so the scenario's reporters fire.
check_fleet_health	Evaluate fleet-health alert rules.
run_agent_review	Run a named agent review test set and fail below its threshold.
exec_pytest	Run a pytest suite from a managed folder. Org-admin only to author (it executes code).

Reports and delivery

Step	What it does
send_email_report	Render a dashboard and email it to a recipient list.
export_dashboard	Render a dashboard to PPTX or PDF, store it, and optionally email it.
snapshot_dashboards	Snapshot every dashboard in the project into its version history, skipping dashboards snapshotted within skip_hours.
export_notebook	Run a notebook headless and publish an HTML report into a managed folder.
create_wiki_export	Render the project wiki to a .docx in a managed folder.
create_flow_documentation_export	Render the project's Flow documentation (datasets, recipes, zones, lineage) to a .docx.
executive_digest	Compute a governed summary briefing with an agent and push it to the org's notification channels.
briefing	Assemble a scheduled digest from configured sections and deliver it per channel (send: false renders without delivering).
business_alert	Push a KPI alert when a configured threshold is crossed.

Control flow and deployment

Step	What it does
run_scenario	Run another scenario's steps inline.
kill_scenario	Abort a target scenario's running runs.
for_each	Run a list of body steps once per item (dataset rows or a static list), with the item's values bound as variables. Sequential; cannot be nested.
custom_script	Run a shell command from the install directory, with output streamed into the step log.
cobuild_prompt	Run a Cobuild prompt headless as a named user. Approval-gated writes pause the step exactly as they would interactively.
prepare_bundle	Snapshot the project as the next bundle version.
deploy_bundle	Push a bundle version to a remote node.
plugin:<plugin_id>:<component>	A scenario step provided by an installed plugin.

Steps run sequentially. Every step goes through the same executor, so every type gets the same per-step options: run_condition (no_prior_failed — the default — always, prior_failed, outcome_is, or a condition expression), max_retries (up to 10), and ignore_failure (continue the scenario even if this step fails). A step can also be switched off with enabled: false.

snapshot_dashboards (v0.0.39)

The dashboard editor's autosave only snapshots on operator edit (rate-limited to one per 30s). Dashboards untouched for a week have zero recovery points without scheduled checkpointing. The snapshot_dashboards step type closes that gap:

{
  "triggers": [
    {"trigger_type": "cron", "config": {"cron": "0 2 * * *", "timezone": "UTC"}}
  ],
  "steps": [
    {
      "type": "snapshot_dashboards",
      "config": {"skip_hours": 12, "max_dashboards": 500, "change_note": "Daily auto-snapshot"}
    }
  ]
}

0 2 * * * fires at 02:00 on the trigger's timezone — any IANA zone id works (UTC, Europe/Berlin, America/New_York, …); see Triggers.

Step config:

Field	Default	Meaning
skip_hours	12	If a dashboard was snapshotted within this many hours, skip it (idempotency on re-runs).
max_dashboards	500	Safety cap to avoid runaway scans.
change_note	"Daily auto-snapshot"	Note stored on each snapshot revision.
author_user_id	0	System author. Override to attribute the snapshot to a specific service user.

With this step active and a daily cron, every dashboard in the project has a recovery point within 24h regardless of edit activity. The Dashboards → History drawer shows scheduled snapshots alongside editor autosaves; the change_note distinguishes them.

send_email_report (v0.0.38)

Email reports are scenario steps (v0.0.38 folded the old standalone report schedules into the scheduler).

Config	Meaning
dashboard_id	Required. The dashboard to render. It must belong to the scenario's own project — a step cannot email another project's dashboard.
recipients	Required. A list of addresses, or one comma-separated string.
subject_prefix	Optional prefix for the email subject.

Delivery uses the org's configured mail server (see Connectors). A failed delivery fails the step, so the scenario's reporters and max_retries apply.

Scenarios list and reporters (v0.0.89, v0.0.93)

The Scenarios list page supports search and a trigger-type filter. Each row offers inline Pause/Resume, Duplicate, and a one-checkbox failure alert toggle. A bulk-disable stale scenarios action sweeps scenarios that haven't fired recently.

Reporters (the email/notification delivery side of a scenario) gain structured editors for their channel_config and a live template preview in the editor, so the rendered subject/body can be checked before saving. Reporter channels include email, Slack, and Microsoft Teams (scenario_reporters.kind='teams', delivered as a Teams card); all external-URL reporters are SSRF-guarded (private/loopback/metadata targets rejected).

Triggers

A scenario can carry multiple triggers (honeyframe.scenario_triggers), any of which fires it:

Trigger	Fires when
cron	A cron expression matches (with a per-scenario timezone).
dataset_modified	A watched dataset is rebuilt or synced.
python	A custom Python predicate returns true.
sql_query	A read-only SQL check returns at least one row (8s timeout).
after_scenario	Another scenario finishes — optionally after a grace_delay_sec delay.

after_scenario chains scenarios without a shared cron. A concurrency guard stops a scenario from overlapping itself: while a run is in flight, a new trigger returns 409 unless the caller passes force. Run history is paginated (GET .../runs).

Scenario settings

Each scenario has settings beyond its steps and triggers:

• marked_as_test — flags a scenario as a test so it's excluded from production alerting and health roll-ups.
• run_as_user — execute the scenario as a specific service user, so its steps see exactly that user's data access (see the send_email_report gotcha below).
• New Scenario modal — start from a Sequence of steps or a Custom Python scenario.

Every step runs through one shared executor, so every step type gets the same knobs: run_condition (run only when a condition holds), max_retries, and ignore_failure (continue the scenario even if this step fails).

Jobs

A job is the durable record of an execution — a build, a sync, an agent run. Build activities are recorded as job_type='build_activity'. The Jobs browser lists jobs across the project (GET /api/pipeline/jobs?scope=all); the Job detail page is a three-pane view (steps → activities → logs) for one job.

Run detail, timeline & logs

• Run-detail timeline — a scenario run renders as a timeline of its steps and activities with per-step timing and status, so a slow or failing step is obvious at a glance.
• Smart log tail — a running job streams the tail of its logs live; per-step and per-activity logs are captured durably, so they survive past the run and are readable afterward.
• Step editor — edit a scenario's steps in place from the run/detail view.
• Admin logs page — a superadmin-only system log browser (org admins do not see cross-tenant logs).

Monitoring fleet overview

A unified Monitoring overview rolls up the whole automation fleet in one board — system health, scenarios, jobs, and agents — so an operator sees what's red across all of them without opening each surface.

Running heavy jobs safely

The scheduler worker runs under a capped memory limit, so a heavy job (a large nightly build, a big extract) run as a native step can OOM-kill the worker. The safe pattern is to launch the heavy job in its own cgroup: a lightweight custom_script step calls systemctl start --no-block <unit>.service, which starts a dedicated systemd unit (with its own MemoryMax) and returns immediately. The heavy work runs isolated from the scheduler's memory budget; the scenario just kicks it off. See Custom systemd Units for the unit pattern.

Heartbeat and observability

The scheduler runs as its own service (hub-scheduler.service). It checks schedules every 30 seconds and writes a heartbeat row on each tick.

GET /api/pipeline/scheduler/health (no authentication — meant for probes) returns:

Field	Meaning
scheduler_present	false when no heartbeat has ever been written — the scheduler is not installed or its migration never ran, so cron and dataset triggers, SLA checks, retries and retention are not running.
is_stale	true when the last tick is more than 120 seconds old (the scheduler is wedged or stopped).
seconds_since_tick, last_tick_at	Age and time of the last heartbeat.
version, error_count, last_error_at	Scheduler build and recent error count. The error text itself is returned only to a platform superadmin.

At startup the platform also runs a schema check and logs a warning when the database is missing tables or columns the release expects — the usual cause of a scheduler that starts but never fires. Apply pending migrations (the update command runs them) and restart.

API reference

All routes are under /api/pipeline and require org.admin in the active project.

Endpoint	Description
GET /api/pipeline/scenarios	List the active project's scenarios with their last run.
POST /api/pipeline/scenarios	Create a scenario.
GET /api/pipeline/scenarios/{id}	Scenario detail plus recent runs.
PUT /api/pipeline/scenarios/{id}	Update a scenario (steps, settings).
DELETE /api/pipeline/scenarios/{id}	Delete a scenario.
GET / POST /api/pipeline/scenarios/{id}/triggers	List or add triggers.
PUT / DELETE /api/pipeline/scenarios/{id}/triggers/{trigger_id}	Update or remove one trigger.
POST /api/pipeline/scenarios/{id}/run	Run the scenario now, regardless of its triggers.
GET /api/pipeline/scenarios/{id}/runs	Run history (limit, offset, outcome).
GET /api/pipeline/jobs	Recent jobs. scope=all lists every top-level run (builds, recipes, scenarios) for the Jobs browser; filters status, job_type, triggered_by, search.
GET /api/pipeline/jobs/{run_id}	One job with its steps and activities.
GET /api/pipeline/jobs/{run_id}/log	A job's log.
POST /api/pipeline/jobs/{run_id}/abort	Abort a running job.
GET /api/pipeline/scheduler/health	Scheduler heartbeat (see above).

Gotchas

• Cron reads the scheduler's server time zone unless the trigger sets one. The server zone comes from TZ in the scheduler service's environment (the shipped unit sets it explicitly — change it with a systemd drop-in). A trigger's timezone overrides it with any IANA id; SERVER or empty means the server zone, and an unknown zone falls back to the server zone rather than skipping the run.
• send_email_report renders as the scenario, not as each recipient. Recipients see every card the scenario's run-as identity can see. Set run_as_user to a service user whose access matches the audience.
• A step type the scheduler doesn't know fails the run. Unknown or missing type values are refused rather than skipped.