Proving the Same Agentic Loop on Real Kubernetes

The live Bedrock RAG demo on this portfolio already runs a bounded agentic loop against a hosted model API. That raised a fair question: does the reasoning behind it only work as a managed API call, or does it hold up on infrastructure I actually provision, wire together, and pay for by the minute. The ephemeral EKS LangGraph lab is the answer. It is a second backend behind the same Ops console, same questions, same corpus, but running on a real EKS cluster I bring up on demand and tear down when the session ends.

Ephemeral was not a cost-cutting afterthought, it was the starting design constraint. A cluster, a node group, and an internet-facing load balancer cost money every hour they exist, whether or not anyone is asking a question. Leaving that running so a recruiter could click a link whenever they felt like it would have meant paying around the clock for idle capacity nobody uses most of the day. So the foundation, container images, Terraform state, IAM trust, the shared vector collection, stays warm and close to free. Everything that actually costs real money, the cluster, the node group, the load balancer, the public IP, gets created on approval and destroyed at the end of the run, backed by a provider-side expiry and a reconciler so a forgotten session cannot keep billing quietly in the background.

What gets provisioned is production-shaped, not a toy. Real EKS, an agent service deployed with Helm, traffic reaching it through an ALB. It is the same kind of plumbing a real platform team runs, just time-boxed to a demo window instead of a quarter.

The harder engineering problem was not standing up Kubernetes. It was making the agent running inside that cluster genuinely agentic rather than a fixed pipeline wearing an agentic name. For a while it was not: retrieve once, generate once, done, with no model-decided branching at all. The Bedrock side had already proven a better pattern, a bounded loop where the model decides per turn whether to search again, capped so it cannot run away. Porting that into LangGraph meant the graph itself had to carry the decision, not just the code around it. The shape ended up as a seed step that runs injection detection and any deterministic evidence-gathering up front, an agent node that calls the model with a retrieve tool bound to it, a tools node that actually executes retrieval and feeds results back, and a conditional edge that keeps looping the agent back to itself while it still wants to search and has not hit the cap, three turns, the same bound as the Bedrock side. Only when the model stops asking for more evidence, or the cap is reached, does the graph fall through to a finalize step.

None of the guardrails that made the Bedrock side trustworthy got dropped in the port. A deterministic fan-out still seeds the loop for profile-shaped questions, what are Arup's strengths, weaknesses, failures, because a model left to pick its own search terms once answered a weakness question with a confident "no weaknesses found," and that is not a bug I am willing to reintroduce on a second backend. If the loop ever finishes without a single real tool call, that answer gets thrown away and the graph forces one grounding retrieval before it will answer at all. If the loop hits its iteration cap before the model volunteers a final answer, finalize forces one last plain-text call from whatever evidence exists rather than returning a half-finished tool request. Underneath all of that sits a hard floor: whatever the model wrote, if the retrieved evidence never cleared a minimum relevance score, the answer gets replaced with an honest refusal. That check runs last and overrides everything above it, including a fluent-sounding answer the model was confident about. A humanizer pass and an LLM-as-judge groundedness check run late in finalize too, one smooths the answer's phrasing, the other checks it against the evidence, and both stay advisory rather than a source of truth on their own.

Cost and teardown are the other half of the honesty story. The cost readout in the Ops panel is not a marketing number, it reads real per-hour rates for the control plane and node group against actual elapsed time for that run. Destroy is not "trust me" either: a run only shows Destroyed once a cleanup pass confirms the expensive pieces, cluster, nodes, load balancer, public IP, are actually gone. That distinction mattered enough to earn its own decision record after an earlier run once claimed to be torn down when it was not.

I will not pretend this is identical to the Bedrock side in every respect. It is slower to start, a real wait for a cluster to come up, and it costs real money per run instead of being free to try. What it proves is different too: not that I can call a hosted model well, but that I can run the same disciplined agent design on infrastructure I own end to end, provisioning and teardown included, and stay honest in the UI about which state it is actually in.