The Agent Asks You: Putting the Visitor in the Loop

By the time this work started, the EKS LangGraph agent had six independent ways of saying "I am not confident": a relevance floor, an LLM grounding judge, a filtered-selection validator, a deterministic coverage check, a guard for numbers that arrived via the conversation rather than retrieval, and injection detection. Every one of them decided silently. The floor swapped in a refusal, the validator dropped lines, the coverage check appended, the judge attached a note. That is the right default, most decisions a guard makes are not worth a human's time. But hiding inside those six were two decisions that were never really the system's to make.

The first was interpretation. Ask "what all projects has he done?" and the enumerate path has to pick a reading: the whole project set, or a slice of it. The classifiers here fail open by design, an unrecognised phrasing degrades to the broadest honest answer rather than a wrong narrow one, which is the correct engineering default and still a guess. The second was money. The loop is capped at three retrieval turns, and when the model spends all three still asking for more evidence, the graph force-answers from whatever it holds. Sometimes that is the honest move. Sometimes the evidence is thin and two more turns would have found the answer, and whether that is worth the extra spend is not a question a graph edge should settle.

The textbook fix is an approval queue, and this system already has one: EKS demo access itself is granted by a signed link in an email to me. Which is exactly why that pattern was wrong here. An email approval works when the decision can wait, provisioning a cluster can wait ten minutes for a human. An ask cannot. The person who typed the question is sitting in front of the Ask tab right now, and they are also the only person who actually knows which reading they meant and whether the answer is worth two more turns of spend. So the human in this loop is the visitor. The admin stays asleep.

Mechanically these are real LangGraph interrupts, not UI theatre. The graph compiles against a checkpointer, interrupt() parks the run mid-node, /v1/ask returns a paused status with the question being put to the human, and a resume endpoint feeds their decision back through Command to continue from exactly that point. I want to be precise about this because there is a cheap version of the same demo, a confirm dialog in the frontend while the backend runs to completion regardless, and the difference matters: in this build nothing is running and nothing is being spent while the card is on screen. LangGraph re-executes a node from its top on resume, so checkpoint placement had a discipline to it, the clarify interrupt sits after only cheap idempotent work, before the fan-out, before routing, before any model call. The re-execution costs a regex pass and one Qdrant scroll, not a Bedrock invoice.

Clarify fires only when a projects enumeration has no topic in its own wording. If the visitor already said "AI projects," the typed domains field scopes the set deterministically and no one is interrupted. If they did not, the card offers the readings, the whole set or the top domains by count, with the whole set as the highlighted default. The fail-open guess became a question that takes one click to answer.

Extend fires at most once per ask, and only when three conditions line up: the model was still asking for tools at the cap, the best evidence score is under a threshold, and the caller opted into interactivity at all. The card shows what the visitor is actually deciding: turns used, the top evidence score, and the marginal cost of saying yes, priced from that run's own token rate rather than a hand-waved estimate. Decline and finalize does what it always did, answer honestly from what exists. Accept and the loop gets two more turns, bounded, never more, and never again this ask. The iteration cap, the piece of the denial-of-wallet story that keeps a runaway loop impossible, never moves without a human explicitly saying so.

The checkpointer is in-memory, and that is a decision, not a shortcut. The pod runs one replica inside a cluster that lives for thirty minutes and is then destroyed on purpose. A durable checkpointer would need a store that outlives the cluster and a wider IAM role on a pod that is deliberately allowed to call Bedrock and nothing else. The failure mode of the cheap option is that a pod restart forgets a paused ask, and the UI's answer to that is "please ask again," which costs the visitor five seconds. I judged five seconds cheaper than a database. The same honesty runs through the trace: both checkpoints land there as a visible human step recording what was asked and what the visitor chose, because a pause the trace does not show would be a decision the system hid.

Two placements were considered and rejected, and the rejections say as much about the design as the checkpoints do. Interrupting before every retrieval is the textbook example, and it is ceremony here: retrieval is read-only and costs milliseconds, and checkpoints that fire on safe operations teach people to click through checkpoints. Holding low-confidence answers for admin review was the other, and it fails on latency: an approval the human cannot see arrive is theatre. Human-in-the-loop earns its place where a human genuinely has information or authority the system lacks, the reading of an ambiguous question, the go-ahead for marginal spend, and it erodes trust everywhere else. Every existing caller, the paid provision smoke test, the evals, anything that never sends the interactive flag, keeps the old never-pausing behaviour to the byte, which is what let this ship without touching a single workflow.