Skip to main content
Version: Next

Guardrails

A guardrail screens what goes into and comes out of an agent. Every agent can carry a guardrail pipeline — an ordered list of checks that run on the user's query (input) and on the model's response (output). A guardrail either warns (surfaces a banner, lets the answer through) or blocks (replaces the answer with a refusal).

Guardrails are authored in Agent Builder → Guardrails (the "Safe Guard" tab) and persist on the agent's config.guardrails / config.pipeline. They run on both the blocking (/api/chat) and streaming (/api/chat/stream) paths; on the streaming path a guardrail SSE event carries the block/warn banner. Every verdict is written to agent_guardrail_events, so you can audit what fired and how often.

The pipeline​

config.pipeline is an ordered list of checks (v0.1.67). Order matters: an input check that blocks stops the turn before the model runs; output checks run against the drafted answer in sequence. Each entry names a check type and its parameters.

Guardrail types​

TypeRuns onWhat it does
PII detectioninput + outputDetect person-data in the text and redact, warn, or block.
Prompt injectioninputFlag attempts to override the system prompt or exfiltrate instructions.
Allowed topicsinput + outputConstrain the agent to an allow-list of subjects.
Forbidden termsoutputBlock a configured list of terms from appearing in the answer.
Custom regexinput + outputAuthor your own match rules (redact / warn / block).
numeric_groundingoutputFlag a reply that reports numeric / money / percent figures (or bracketed placeholders like [metric %]) when no data tool ran that turn — anti-fabrication. Warn or block.
numeric_grounding_strictoutputStricter grounding that also parses Indonesian number formatting (1.234.567,89).
period_completenessoutputAppend a "data belum lengkap" (data incomplete) caveat when an answer covers a still-running period; grain-aware (day/week/month).
table_completenessoutputDisclose when the answer draws on a table that is only partially loaded.
response_formatoutputConstrain the answer shape: json, markdown_table, or no_code.

The grounding, period-completeness, and table-completeness checks are the honesty enforcement family — they exist to stop an LLM from confidently reporting figures it did not actually query. The output judge that scores these runs on a date-aware system prompt so "latest period" reasoning is anchored to real time.

Date anchoring​

config.dateAnchor ({enabled, table?, column?}) rewrites a bare CURRENT_DATE in agent-generated SQL to the maximum business date of a named table/column, so "this month" means the latest data month rather than the wall-clock month. Pair it with period_completeness for period questions on a lagging feed.

Cost Guard​

Per-agent spend limits sit alongside the content guardrails:

  • config.costGuard ({monthly_usd, action}) — cap an agent's monthly LLM spend; action is warn or block.
  • maxCallsPerTool — cap how many times a single tool may fire within one tool-calling loop (blocking and streaming).

Per-agent spend is visible at GET /api/agents/{id}/cost; the org-wide roll-up ("AI Health") lists per-agent spend and guardrail-flag counts under Administration → Monitoring.

Eval gate​

config.evalGate gates promotion: before a new agent version is promoted, an LLM-as-judge golden-suite run scores it, and a regression warns or blocks the promote. Read the current verdict with GET /publish/assets/{id}/gate-status. The auto-run can be disabled install-wide with HONEYFRAME_EVAL_GATE_AUTORUN=0.

See also​