Design Doc: EKS Multi-Tenant at Scale — Karpenter, Isolation, and Cost

Internal Kubernetes platforms grow organically. One team creates a cluster, another asks for access, and eighteen months later you have forty teams sharing infrastructure designed for five. The typical result is a cluster where nobody knows exactly what costs what, critical workloads compete with batch jobs for nodes, and the blast radius of a misconfigured namespace can affect neighbors.

The scenario this document addresses is an EKS platform managed by a central platform team, serving between 30 and 60 product teams. Each team is a tenant: they have their own namespaces, repositories, pipelines, and ideally their own cost account. The cluster needs to scale reactively and economically, maintain isolation strong enough to satisfy security audits, and produce cost data granular enough for showback or chargeback.

The three most common failure vectors in this model are: (1) insufficient isolation — one tenant consumes another's resources due to missing quotas or network policies, or worse, achieves privilege escalation via misconfigured RBAC; (2) invisible cost — without consistent labels and allocation tooling, the AWS bill becomes an aggregate number nobody can contest or optimize; (3) slow or expensive autoscaling — Cluster Autoscaler with fixed node groups can't keep up with heterogeneous workload spikes and tends toward overprovisioning for safety. Karpenter solves the third vector, but needs guardrails to avoid becoming an uncontrolled cost vector in the hands of unrestricted tenants.

The isolation model adopted is namespace-as-tenant-boundary, reinforced across four independent layers. No single layer is sufficient; defense in depth is the governing principle.

Layer 1 — Namespace and identity: Each tenant receives a dedicated namespace (or a set of them, by environment: team-x-prod, team-x-staging). Provisioning is automated via GitOps (ArgoCD or Flux): a PR to the platform repository creates the namespace, applies default ResourceQuota and LimitRange, and creates the RoleBinding granting the team's ServiceAccount edit permissions within the namespace, never cluster-admin. The RBAC policy follows least-privilege: teams cannot create ClusterRoles, access secrets from other namespaces, or modify NetworkPolicies — those operations are reserved for the platform team.

Layer 2 — Network: A default-deny-all NetworkPolicy is applied to each namespace at creation time. Teams must explicitly declare the ingress and egress policies their workloads require. Cross-namespace traffic is blocked by default; exceptions are granted via policy reviewed by the platform team. This eliminates the risk of one tenant inadvertently accessing another tenant's internal APIs. Cluster-internal DNS (kube-dns) remains accessible to all namespaces, but pod access to the Kubernetes API is restricted via NetworkPolicy to the control plane CIDR.

Layer 3 — Compute and quotas: ResourceQuota defines limits on CPU, memory, pod count, and object counts (Services, ConfigMaps, Secrets) per namespace. LimitRange defines default requests and limits for containers without explicit specification — this is critical for the Kubernetes scheduler and Karpenter to make correct bin-packing decisions. Workloads without defined requests are treated as BestEffort and may be evicted first under memory pressure. The platform team defines three quota tiers (small, medium, large) and tenants request the appropriate tier via PR.

Layer 4 — Workload identity: No pod should carry static AWS credentials. The preferred model is EKS Pod Identity, which associates a Kubernetes ServiceAccount with an IAM Role via the eks-pod-identity-agent DaemonSet. The role is created with minimal scope and reviewed by the security team. IRSA remains supported for legacy workloads, but new onboardings use Pod Identity. This eliminates the vulnerability class where credentials are exposed in environment variables or ConfigMaps.

Karpenter replaces Cluster Autoscaler with a fundamentally different approach: instead of scaling pre-defined node groups, it observes Unschedulable pods and provisions the most suitable node directly via the EC2 Fleet API, respecting constraints declared in the pod (nodeSelector, affinity, tolerations, topology spread). This enables more efficient bin-packing and significantly lower provisioning latency.

NodePool structure: I define three NodePools with distinct responsibilities:

1. system — On-Demand nodes dedicated to platform workloads (Karpenter, CoreDNS, Kubecost, admission webhooks). These nodes carry a CriticalAddonsOnly taint and are excluded from aggressive consolidation. I use fixed m6i.large or m6i.xlarge instances for predictability.

1. production — nodes for product workloads with SLAs. Strategy: On-Demand as the base, with Spot capacity for up to 30% of nodes when available. The EC2NodeClass defines a diverse list of instance types (m6i, m7i, c6i, c7i) to maximize Spot availability. karpenter.sh/capacity-type: [on-demand, spot] with spot-to-on-demand-ratio controlled via weight. PodDisruptionBudgets are mandatory for workloads in this pool — without a PDB, Karpenter can consolidate nodes and bring down all pods of a Deployment at once.

1. batch — Spot-first nodes for ML, ETL, and CI jobs. Here I accept interruption with declared tolerance (karpenter.sh/interruption-queue integrated with SQS to receive EC2 interruption notifications and gracefully drain nodes before termination). Instance type diversity is maximized in this pool — r6i, m6i, c6i, r7i, m7i in multiple sizes — to ensure high Spot Placement Scores.

Consolidation: I enable consolidationPolicy: WhenUnderutilized for the batch pool and WhenEmpty for production. The difference matters: WhenUnderutilized allows Karpenter to move pods between nodes to free underutilized nodes (active bin-packing), while WhenEmpty only removes completely empty nodes. For production, aggressive consolidation can cause excessive disruption; I prefer WhenEmpty with well-configured PDBs.

Cost guardrails: Without constraints, a tenant can create a Job requesting 1000 pods of 4 vCPU each, and Karpenter will provision 250 c6i.xlarge nodes in seconds. To prevent this, ResourceQuota in the namespace limits total requestable CPU/memory. Additionally, the NodePool defines global limits.cpu and limits.memory — Karpenter won't provision beyond these limits even if there are pending pods. This creates a cost ceiling per pool.