Inside AI Agents (3/3): Agents in Production on AWS with Bedrock AgentCore

When you put an agent into production for the first time, six problems appear almost simultaneously. First: where does the loop run? You do not want to manage servers for something that may be idle 99% of the time. Second: how does the agent call external tools in a standardized way? Each API has its own schema; without a common protocol, you write glue code forever. Third: how does the agent remember what happened? LLMs are stateless by nature — each call starts from zero. Fourth: how does the agent obtain credentials for tools without you putting secrets in the prompt? Fifth: how do you know what the agent decided and why? Without traceability, debugging an agent is like debugging a program without logs. Sixth: how does the agent access information that was not in its training data?

AgentCore solves each of these problems with a dedicated component: Runtime (where the loop runs, serverless, isolated per session), Gateway (MCP protocol for tools), Memory (managed short and long term), Identity (tool credentials without secrets in code), Observability (traces, metrics, and evaluation), and the managed tools Browser and Code Interpreter (for browsing the web and executing code safely). Each piece can be used independently — you do not need to adopt everything at once.

This is the question that most confuses people arriving at the AWS agent ecosystem. There are three layers and they compose, they do not exclude each other.

Bedrock Agents is the high-level, fully managed service with a console UI. You define a system prompt, associate a model, connect knowledge bases (RAG) and action groups (tools). It is the fastest path to a working agent — think of it as the 'Amplify of agents': productive for common cases, less flexible for advanced ones.

AgentCore is the infrastructure layer. It does not orchestrate the agent — it provides the blocks any orchestrator needs: isolated runtime, managed memory, tool gateway, identity, and observability. You use AgentCore when you need fine control over the reasoning loop or when you are building a multi-agent system.

Strands Agents is an AWS open-source Python framework that uses AgentCore as its backend. It is the code abstraction over the infrastructure — you write @tool in Python and Strands handles the ReAct loop, the Bedrock call, and the AgentCore integration. LangGraph is a more flexible and more complex alternative, ideal when the agent state graph is non-linear or when you need explicit control over each state transition.

The practical rule: start with Bedrock Agents for proofs of concept. Migrate to Strands + AgentCore when you need session isolation, persistent memory, or custom tools. Use LangGraph when the reasoning flow is a complex graph that Strands cannot express.

Every language model has a training cutoff date — a point in time after which it does not know what happened. It is like hiring a brilliant consultant who spent the last 18 months on an internet-free retreat: they know everything they learned before, but not what came out yesterday. For a production agent, this is a serious problem: prices, regulations, news, API documentation — everything changes.

The solution is search-based grounding: instead of relying solely on the model's knowledge, the agent uses a search tool to retrieve current information before responding. AgentCore Web Search exposes this capability as an MCP tool — the agent decides when to search, formulates the query, receives the results, and incorporates them into the context before generating the final response. It is exactly the same mechanism a human uses when opening Google in the middle of a conversation.

This site — the architecture portfolio you are reading right now — uses this exact mechanism to stay current. When an agent generates or revises an architecture document, it uses Web Search via MCP to check for new services, price changes, or documentation updates that would invalidate the content. It is a real meta-example: the mechanism we are describing is the mechanism that keeps this document relevant.

From a security perspective, Web Search via AgentCore has an advantage over calling search APIs directly: the search API credentials live in AgentCore Identity, not in the agent's code. The runtime injects the token at call time — the model never sees the key.

Model choice is the highest-impact decision on an agent's TCO. In Bedrock, you have a spectrum ranging from light, cheap models (Amazon Nova Micro, ~$0.035/1M input tokens) to heavy reasoning models (Claude 3.7 Sonnet with extended thinking, ~$3/1M input tokens). The cost difference is two orders of magnitude — and in an agent with multiple ReAct steps, you pay for each model call.

The rule I use in practice: use the cheapest model that solves the problem. For classification, structured data extraction, and simple responses, Nova Micro or Haiku are sufficient. For multi-step reasoning with tools, Claude 3.5 Sonnet is the optimal cost/quality point. For problems requiring deep reasoning (complex financial planning, legal analysis), Claude 3.7 with extended thinking is worth the extra cost.

In practice, a customer support agent with 1,000 sessions/month, an average of 5 ReAct steps per session, and 2,000 tokens of context per step stays below $5/month using Haiku — and below $50/month using Sonnet. Idle cost is zero because AgentCore Runtime is serverless: you pay only for actual execution time. This is radically different from keeping an agent server running 24/7.

The second cost vector is tool calls: each Lambda invocation, each DynamoDB query (for memory), each Web Search call has a cost. AgentCore Observability gives you the visibility to identify which tools are called most frequently and optimize — for example, putting a cache in front of repeated searches.

Security in an agent system has three layers that need to work together. The first is session isolation: each user runs in a completely separate context in the AgentCore Runtime. There is no memory leakage between sessions — what user A said does not appear in user B's context. This is analogous to process isolation in an OS: each process has its own address space.

The second layer is Bedrock Guardrails: configurable filters that inspect both user input and model output. You configure forbidden topics (e.g., 'do not discuss competitors'), PII filters (tax IDs, credit cards are automatically masked), and offensive content thresholds. Guardrails run outside the model — they do not consume tokens and cannot be bypassed by prompt injection in the message content.

The third layer is least privilege on tools: the IAM Role of the AgentCore Runtime should have access only to the tools that specific agent needs. If the customer support agent does not need to write to the database, the role has no write permission. AgentCore Identity manages credentials for external tools (third-party APIs, OAuth tokens) — the model never sees these credentials, they are injected by the runtime at call time.

A pattern I recommend: use session tags in IAM so that each tool call carries the user ID. This enables granular auditing in CloudTrail — you can reconstruct exactly which tools were called by which user in which session.

This series traveled the architecture elevator from top to bottom. Part 1 established the business floor: what an AI agent is, why it is different from a chatbot, and when it makes sense to use one. Part 2 descended to the design floor: orchestration patterns (ReAct, Chain-of-Thought, multi-agent), how reasoning loops work, and the most common design pitfalls. This Part 3 reached the technical floor: how to put all of this into production on AWS with isolation, security, observability, and controlled cost.

What the three parts build together is a complete mental model: you know why agents exist (Part 1), how they think (Part 2), and how you operate them (Part 3). This mental model is what differentiates an architect who 'uses LLMs' from an architect who 'designs agent systems'.

The next studies in this architecture series go deeper on two themes we only touched here: multi-agent orchestration (how to coordinate multiple specialized agents in a coherent system) and AgentCore in production (a detailed case study with real cost and latency numbers). If you made it this far, you have the foundation to read those studies with depth.