Playbook: Which AWS AI Service to Use — The Decision Tree

Think of the five services as a single axis: on one end, maximum abstraction and delivery speed; on the other, maximum control and cost efficiency at scale. No point on the axis is better than another — each is optimal for a maturity stage and a set of constraints.

Amazon Q sits at the abstraction extreme: you don't write AI code, don't manage models, don't think about tokens. You connect data sources, configure permissions, and deliver productivity. It's the right service when the problem is access to corporate knowledge, not building an AI system.

Amazon Bedrock is the next step down: you consume foundation models via API (Anthropic Claude, Meta Llama, Mistral, Amazon Titan, Cohere, Stability AI, and others) without managing inference infrastructure. The contract is simple — you send tokens, receive tokens, pay per token. It's the correct starting point for most LLM production use cases.

Bedrock AgentCore is the runtime layer for when plain Bedrock isn't enough: when your agent needs persistent memory across sessions, tool orchestration with retry and timeout, a security gateway for model calls, and structured observability (traces, spans, latency metrics per step). Without AgentCore, you build all of this by hand — and you invariably build it wrong the first time.

SageMaker AI is where you go when the model you need doesn't exist in the Bedrock catalog, or when you need fine-tuning with your proprietary data, or when managed inference latency doesn't meet your SLA. It's a complete ML platform — experiments, pipelines, feature store, model registry, inference endpoints. Powerful, but expensive in engineering time.

Self-hosted EKS + GPU is the control extreme: you operate the cluster, manage CUDA drivers, configure GPU node autoscaling, monitor VRAM utilization. Cost per token can be a fraction of Bedrock at high volumes, but operational cost and availability risk are entirely yours. It makes sense when you have a specific model not in Bedrock, regulatory constraints preventing data from leaving your VPC, or volume so high that pay-per-token becomes prohibitive.

Bedrock is excellent — but it became the 'React' of the AWS AI world: everyone uses it regardless of the problem. I see three recurring error patterns in real projects:

Error 1 — Bedrock for internal productivity that Q already solves. A team spends weeks building a custom RAG over internal documentation, integrating with Kendra or OpenSearch, managing chunking, embeddings, and retrieval. Amazon Q Business does exactly this with native connectors for S3, SharePoint, Confluence, Salesforce, and 40+ other sources — with permission control based on IAM and Active Directory groups. The build cost isn't justified when the product already exists.

Error 2 — Plain Bedrock for complex agents. You start with a simple agent on Bedrock — one tool, one prompt, it works. Then you add session memory (you implement in DynamoDB), then retry on failing tools (you implement in the application), then observability (you instrument with X-Ray manually), then rate limiting on the gateway (you implement in API Gateway). In six months you've hand-built AgentCore, with bugs the managed service already fixed. AgentCore exists to prevent exactly this plumbing accumulation.

Error 3 — Self-hosted before validating volume. Teams that estimate Bedrock cost at scale and conclude 'it'll get expensive' jump straight to EKS + GPU without having validated the product. The operational cost of a production GPU cluster — with high availability, model updates, VRAM monitoring, spot interruption management — is real and ongoing. The rule: validate the product on Bedrock, migrate to self-hosted when the monthly Bedrock bill justifies the operational investment.

SageMaker is frequently underestimated in GenAI discussions because recent focus is on foundation LLMs. But there are four scenarios where it's the only correct answer:

1. Fine-tuning with sensitive proprietary data. If you need to adapt a model with data that cannot leave your AWS account — medical, financial, legal data — SageMaker offers training within your VPC, with encryption at rest and in transit, without data flowing to third-party infrastructure. Bedrock offers fine-tuning for some models, but with less control over the execution environment.

2. Models not in the Bedrock catalog. If your use case requires a specific model — a specialized domain LLM, a custom computer vision model, a time-series forecasting model — SageMaker is where you train and serve. JumpStart accelerates the starting point with pre-trained models from HuggingFace and other repositories.

3. Traditional ML pipelines with structured features. For XGBoost models, neural networks on tabular data, recommendation models — SageMaker Pipelines, Feature Store, and Model Monitor form a cohesive platform that Bedrock simply doesn't cover.

4. Inference latency with aggressive SLA. A dedicated SageMaker endpoint (non-serverless) offers predictable P99 latency because you control the instance. Bedrock on-demand has latency variability that may be unacceptable for cases like real-time scoring of financial transactions. Bedrock Provisioned Throughput mitigates this, but at a fixed cost that often makes SageMaker competitive.

The SageMaker trap is operational overhead: you manage endpoints, monitor drift, update models, configure auto-scaling. For teams without mature MLOps, this operational cost is systematically underestimated.