ADR: Aurora Sharding — App-Level vs Aurora Limitless vs Citus

Before any conversation about sharding, it's mandatory to exhaust vertical and read-scaling alternatives. In this scenario, we've already done that: the writer is on db.r7g.16xlarge, read replicas absorb 100% of read queries via PgBouncer with session pooling, and indexes have been reviewed by a senior DBA. The problem is structural — write volume doesn't fit in a single node, and Aurora doesn't offer native horizontal write scaling in the classic model.

The diagnosis matters here. The bottleneck is not storage I/O — Aurora decouples storage from compute and the distributed storage layer scales automatically. The bottleneck is the WAL writer process and the lock manager on the compute node. Every committed transaction must pass through the writer, serialize the WAL, and wait for the storage node quorum ACK. With 50k TPS peak and average transactions of 5-8 statements, we're talking about hundreds of thousands of lock operations per second. This is a compute limit, not a storage limit.

The second dimension of the problem is growth. With 15% monthly growth, even if a vertical upgrade bought 6 months, we'd be back at the same point in under a year — with an even larger, more expensive instance and fewer escape options. Sharding isn't just about the present; it's about creating an architecture that scales with the business without requiring a new platform decision every growth cycle.

The third dimension is the team. Six data engineers with a 99.99% SLA means every unplanned oncall hour is expensive. The sharding choice cannot create an operational surface the team can't sustain. This immediately eliminates any solution that requires the team to implement their own rebalancing, shard metadata management, or cross-shard query routing from scratch.

Any sharding decision involves tensions that can't be resolved — only managed. I'll be explicit about each one:

Shard key and data distribution. We have tenant_id as a natural key. This is a huge advantage: most queries are scoped by tenant, meaning cross-shard queries are the exception, not the rule. But tenants aren't uniform — some have 100x more data than others. Any sharding strategy needs to handle hot shards from large tenants, either via sub-sharding or special routing.

Cross-shard queries. Even with 80% of queries scoped by tenant, the remaining 20% exist: aggregate reports, admin queries, cross-tenant joins in specific features. The question isn't whether cross-shard queries will happen, but what the cost of executing them is and who pays that cost — the application, the database, or the engineer who needs to rewrite the query.

Rebalancing. Shards aren't static. Tenants grow, new tenants arrive, tenants leave. The ability to move data between shards without downtime is critical. Solutions that require the team to implement this manually are a serious operational risk for a 6-person team.

PostgreSQL compatibility. The system uses PostgreSQL-specific features: JSONB, pg_trgm for partial full-text search, and some legacy stored procedures. Any solution needs to preserve this compatibility or the application migration cost explodes.

Cost. Aurora Limitless has a different pricing model than classic Aurora — you pay for Distributed Processing Units (DPUs) in addition to storage. For workloads that could be served by 2-3 independent writers with app-level sharding, Limitless cost may be 30-50% higher (estimate based on public AWS benchmarks, not direct measurement in this scenario). That delta needs to be justified by reduced operational complexity.

Vendor lock-in. Aurora Limitless is AWS-specific. Citus is open-source but has a managed version on Azure (Cosmos DB for PostgreSQL). App-level sharding with standard PostgreSQL is the most portable. For a system with 99.99% SLA and critical business dependency, portability has real value — but it's not free.

Figma and Notion are well-documented cases of application-layer sharding. Both worked — but both have data engineering teams of dozens of people, years of runway to stabilize the implementation, and the capacity to absorb the costs of a complex migration. What these cases teach is not that application-level sharding is a good idea for everyone — it's that it's a good idea for those who have the resources to do it correctly.

For a 6-engineer team with a 99.99% SLA, the real cost of application-level sharding is not the initial routing code. It's what comes after: the first hot shard that appears at 2am and needs to be manually rebalanced; the schema migration that needs to run across 8 shards in a coordinated fashion with tested rollback; the reporting query someone wrote without a shard key that's now doing a full scan across all shards; the bug in the shard registry that sends 0.1% of writes to the wrong shard and is only discovered 3 days later.

These are not hypothetical problems. They are the problems that invariably appear in manual sharding implementations, and the question is whether the team has the capacity to solve them without compromising the SLA. The honest answer, for a 6-person team without prior experience in distributed sharding, is no.

This doesn't mean application-level sharding is wrong. It means it has an organizational maturity cost that needs to be acknowledged. If the team grows to 15-20 engineers with specific distributed systems experience, the equation changes — and the portability of manual sharding becomes a real asset. Today, it's not.