Human in the loop

Human-in-the-Loop Design

We design AI systems where a human stays in control by construction — approval gates the model cannot route around, tunable autonomy per workflow, and a full record of who decided what.

Human-in-the-loop AI only works if the human control is real — not a checkbox the model can quietly skip. We design agentic systems where every consequential action stops at an approval gate before it touches the outside world, where the autonomy level is a per-workflow setting you tune as trust is earned, and where every decision leaves a record. This is responsible AI as engineering: AI guardrails wired into the data model, AI approval workflows that are the only path to action, and AI security boundaries that hold under load.

Approval gates the model cannot route around

The strongest guardrail is one the system cannot violate even if the model misbehaves. We enforce human sign-off in the schema itself, not just in UI or prompt instructions. A drafted action carries a status enum whose only values require a human transition — pending, approved, sent, rejected, failed — with no auto-send state to fall into. Because there is literally no enum value an agent could set to bypass review, the model cannot act without a person, and that property holds no matter what the model generates. The pending action parks in a queue; a separate, explicit execute step performs the real side effect only after a human approves. The boundary is structural, so it does not depend on the model behaving well.

Tunable autonomy per workflow, not all-or-nothing

Trust is earned, so autonomy should be a dial, not a switch you flip once for the whole system. We model an approval mode per workflow — review-everything, where the agent drafts and a human sends each item; review-before-send as the safe default; and autopilot for the cases that have proven themselves. The same job machinery backs all three: an action either parks in a pending-approval state or runs immediately, depending on that one setting. Teams start conservative and graduate specific workflows to higher autonomy as the evidence accrues, without re-architecting. The operator owns intent and structure; the model owns expression and execution within the latitude you grant it.

Every decision leaves an audit trail

Responsible AI means you can always answer who approved this, when, and what exactly they approved. We write an audit row for every approval, rejection, and edit — generic enough that one row shape covers sending outreach, sending a proposal, or changing a record's state, capturing the approver, the timestamp, the decision, and the before-and-after of any edit. Approval records and the agent runs that produced the drafts are first-class tables with their own lifecycle, not JSON buried inside an activity log. The result is a defensible trail: for any AI-assisted action that reached the outside world, you can reconstruct the human who let it through and the reasoning attached to that draft.

Capturing human edits as a quality signal

The moment a human edits an AI draft before approving is a gift — it tells you precisely where the model fell short. We capture the edit distance between the model's draft and the version the human actually sent, plus a flag for whether the draft went out untouched. Over time those signals become a measurable quality metric per agent: how often drafts ship unchanged, how heavily they get rewritten, and on which workflows. That feedback closes the loop. The human-in-the-loop step is not just a gate that slows things down; it is the training signal that makes the next draft better, so review burden falls as the system learns the operator's voice and judgment.

AI security and guardrails as a system property

Guardrails belong in several layers, not one. We constrain what an agent is even allowed to attempt — the claims it can and cannot make, the actions it can and cannot take — as structured, first-class configuration rather than buried prompt text. Read operations can run freely; any action that mutates state or reaches an external system returns a proposed action that the UI renders as a confirm-or-reject card, and the backend mutates only on explicit approval. Suppression and do-not-contact rules are enforced globally so a single opt-out is honored everywhere. Together these make AI security a property of the architecture: the blast radius of a bad model output is bounded by what the system structurally permits.

What this includes

Approval status enums with no auto-act value, so human sign-off is enforced in the schema
Per-workflow autonomy modes — review-everything, review-before-send, autopilot — on shared job machinery
A generic approval record covering sends, proposals, and state changes, with approver, decision, and timestamp
Proposed-action pattern: read tools run live, write tools require an explicit confirm-or-reject step
Edit-distance and acceptance capture on every draft, feeding a per-agent quality metric
Structured guardrails for allowed claims and actions, plus global suppression and do-not-contact enforcement

What you get

AI that cannot take a consequential action without a human, by construction rather than by convention
A defensible audit trail showing who approved what, when, and the exact change they signed off on
Review burden that falls over time as captured edits teach the system the operator's voice and judgment

Where it fits

Use cases

Outbound that never sends itself

An agent drafts personalized outreach, but the send job parks in a pending state until a human approves. The status model has no auto-send value, so the message physically cannot leave without sign-off — and every approval is audited.

Graduating a workflow to autopilot

A team runs a workflow in review-before-send for weeks, watching the draft-acceptance rate climb. When edits drop near zero, they flip that one workflow to autopilot — no re-architecture, just a per-workflow setting.

Reconstructing a decision under scrutiny

When someone asks why an AI-assisted message went out, the team pulls the approval row: the approver, the time, the before-and-after edit, and the agent run that produced the draft — a complete, defensible record.

FAQ

Common questions

An approve button is UI; it can be bypassed by a bug or a clever model output. We make the human step structural — the data model has no state an agent can set to act on its own. The only path from draft to action runs through a human, so the guarantee holds regardless of how the model behaves, which is the difference between responsible AI and a hopeful checkbox.

It does not have to. Autonomy is tunable per workflow, so low-risk steps can run on autopilot while consequential ones stay gated. And because we capture how much humans edit each draft, the system improves — review burden falls as draft quality rises. The AI approval workflow starts conservative and earns its way to speed, rather than trading safety for it up front.

Several layers. Allowed and disallowed claims and actions are structured configuration, not prompt text. Read operations run freely; any state-changing or outbound action becomes a proposed action a human must confirm. Suppression and do-not-contact lists are enforced globally. Together these bound the blast radius of a bad output, which is the practical core of AI security in an agentic system.

Every approval, rejection, and edit writes an audit row capturing the approver, the timestamp, the decision, and the before-and-after of any change. Approval records and the agent runs that produced each draft are first-class, queryable tables. For any AI-assisted action that reached the outside world, you can reconstruct exactly who let it through and what they signed off on.

Explore more capabilities

08 — Capability

Building something that needs this?

Tell us what you're working on. The first call is always free.

Start a project→All capabilities