Patterns for agentic engineering in production

Most agent demos fall over the moment they meet real users, flaky tools, and unpredictable inputs. The gap between “works in a notebook” and “works in production” is almost entirely about engineering discipline, not model quality. A few patterns have consistently helped:

Treat tool calls like any other unreliable dependency

An LLM calling a tool is functionally a network call with a non-deterministic caller. That means the same rules apply as anywhere else in a distributed system: timeouts, retries with backoff, circuit breakers for tools that are degraded, and idempotency keys for anything that mutates state. Skipping this because “it’s just an agent” is how you end up with duplicate charges or half-applied writes.

Separate planning from execution

Letting a single model call both decide and act in one unstructured loop makes failures hard to debug and hard to bound. Splitting the agent into a planning step (produces a structured, inspectable plan) and an execution step (runs plan steps against real tools, with validation at each boundary) makes it possible to log, replay, and cap the blast radius of a bad decision.

Make the state machine explicit

An agent loop is a state machine whether you draw it that way or not. Making states and transitions explicit — instead of letting them live implicitly inside a prompt — is what lets you add guardrails: max iteration counts, escalation to a human on repeated failures, and clear “done” conditions instead of open-ended looping.

Budget everything

Tokens, tool calls, wall-clock time, and cost per request all need hard ceilings before launch, not after the first runaway loop in production. Treat these the same way you’d treat rate limits on any other API you expose.

None of this is exotic — it’s the same rigor backend engineers already apply to distributed systems. The shift is remembering to apply it to agents too, instead of treating them as a special case that’s exempt from the usual reliability practices.