SYSTEM DESIGN:Lesson 2: Scaling the Data Layer
Mastering lesson 2: scaling the data layer concepts and implementation.
Practice this chapter
Read the theory, then wire up the architecture yourself. These labs match what you just learned.
Your app servers are fine. Your database is dying.
"We added six API servers behind a load balancer. Traffic was smooth. Then Postgres hit 98% CPU and every query started taking 3 seconds. The load balancer did its job. The database did not."
This is the most common scaling story in Indian tech. You horizontally scale the app tier during a sale or an IPL fantasy league launch — and discover the real bottleneck was hiding in the data layer all along.
App servers are stateless and cheap to replicate. Databases are stateful, expensive, and full of trade-offs. This chapter covers the four tools that actually fix data-layer bottlenecks: read replicas, sharding, caching, and CDN — plus the five-stage evolution diagram that shows how real systems grow.
If you have not read the scaling basics chapter yet, start there: Scaling Basics.
App servers happy, database not happy
The load balancer did its job. Postgres did not get the memo.
Read replicas: split reads from writes
Most applications are read-heavy. A social feed might have a 100:1 read-to-write ratio. A product catalog might be 1000:1. Yet by default, every SELECT and every INSERT hits the same primary database.
A read replica is a copy of your primary database that handles read queries only. Writes still go to the primary. Reads get distributed across replicas.
Read replicas distributing SELECT queries
Write path: App → Primary DB (INSERT, UPDATE, DELETE)
Read path: App → Replica 1, Replica 2, Replica 3 (SELECT)
Sync: Primary → async replication → Replicas
| Aspect | Primary | Read Replica |
|---|---|---|
| Handles writes | Yes | No (read-only) |
| Handles reads | Yes | Yes |
| Data freshness | Always current | Slightly stale (replication lag) |
| Failover target | — | Can be promoted to primary |
| Cost | Higher (write-optimized) | Lower (read-optimized) |
Replication lag is the catch. A user posts a tweet, refreshes the page, and does not see it for 200ms because the replica has not caught up yet. For most products — Swiggy restaurant listings, Flipkart product pages, cricket scoreboards — eventual consistency on reads is perfectly acceptable.
For payments and account balances? You read from the primary. Know which queries need strong consistency and which can tolerate a few hundred milliseconds of lag.
Practical numbers:
- One primary + 3 read replicas → roughly 4x read capacity
- PostgreSQL async replication lag: typically 10–500ms under normal load
- AWS RDS read replicas: add one in ~10 minutes, no code change if you use a read/write connection pooler
Sharding: when one database is not enough
Read replicas help when you have too many reads. Sharding helps when you have too much data or too many writes for a single machine to hold or process.
Sharding means splitting your data across multiple independent databases, each holding a subset of the total data. Each shard is a full database — schema, indexes, everything — just with fewer rows.
Without sharding:
users table → 500 million rows → one Postgres instance
With sharding (by user_id % 4):
Shard 0: user_id ending in 0,4,8... → 125M rows
Shard 1: user_id ending in 1,5,9... → 125M rows
Shard 2: user_id ending in 2,6... → 125M rows
Shard 3: user_id ending in 3,7... → 125M rows
Choosing a shard key is the hardest part of sharding.
| Shard key strategy | Example | Pros | Cons |
|---|---|---|---|
| Hash of user ID | `user_id % num_shards` | Even distribution | Cross-shard queries are painful |
| Geographic region | `country_code` | Data locality, compliance | Hot regions overload one shard |
| Tenant ID | `org_id` | Natural isolation for B2B | Large tenants create hot shards |
| Time-based | `created_at` month | Good for logs/events | Recent shard gets all writes |
PhonePe does not shard UPI transactions by user name — they shard by user ID hash so each shard gets roughly equal write volume. A celebrity sending 10,000 payments in a minute would create a hot shard if you sharded by something correlated with activity.
When to shard vs when to wait:
- Under 100 GB of data and moderate write volume → read replicas + better indexes first
- Over 500 GB or write throughput exceeding single-node IOPS → start planning sharding
- Cross-shard JOINs do not exist in production — design your queries around one shard key
Sharding is a last resort, not a first move. Every company that shards wishes they had designed for it earlier — but every company that shards too early wishes they had waited.
Redis cache-aside: stop hitting the database for hot data
Even with read replicas, some queries are so frequent that hitting any database at all is wasteful. "What is the current IPL points table?" does not need a Postgres query 50,000 times per second.
Cache-aside is the most common caching pattern in production:
Cache-aside pattern with Redis
READ:
1. Check Redis for key "ipl:points_table"
2. Cache HIT → return cached data (1ms)
3. Cache MISS → query Postgres → store in Redis with TTL → return data
WRITE:
1. Write to Postgres (source of truth)
2. Delete cache key "ipl:points_table" (invalidate)
3. Next read will repopulate the cache
| Pattern | When to use | Risk |
|---|---|---|
| Cache-aside | General purpose, you control cache logic | Cache miss stampede on cold start |
| Write-through | Cache and DB updated together | Write latency increases |
| Write-behind | Write to cache, async flush to DB | Data loss if cache crashes before flush |
| Read-through | Cache layer handles DB fetch automatically | Less control over query logic |
What to cache (and what not to):
- Cache: user profiles, product catalogs, leaderboards, config flags, session data
- Do not cache: financial balances, real-time inventory counts, anything where stale data costs money
TTL (time-to-live) is your safety net.
Set a TTL even when you invalidate on write. If your invalidation logic has a bug, TTL ensures stale data expires eventually. A 5-minute TTL on a restaurant menu cache is fine. A 5-minute TTL on a wallet balance is not.
Redis numbers that matter in interviews:
- Single Redis instance: ~100,000 ops/sec for simple GET/SET
- Latency: sub-millisecond for in-memory reads
- Memory: 1 GB Redis holds roughly 1 million small key-value pairs
- Redis Cluster: shards data across nodes for horizontal cache scaling
CDN: serve static content from the edge
Your API servers should not be serving images, JavaScript bundles, and CSS files. That is what a CDN (Content Delivery Network) does.
A CDN caches static assets at edge locations close to users. A user in Chennai loading a Zomato restaurant photo hits a Chennai edge server — not your origin server in Mumbai.
Without CDN:
User (Kolkata) → API server (Mumbai) → 80ms latency per image
1M users × 10 images = 10M origin requests
With CDN (CloudFront / Cloudflare):
User (Kolkata) → Edge (Kolkata) → 5ms latency per image
Cache hit rate 95% → only 500K origin requests
| Content type | CDN? | Why |
|---|---|---|
| Product images | Yes | Large, rarely change, same for all users |
| JS/CSS bundles | Yes | Versioned by hash, cache forever |
| User-uploaded photos | Yes | Immutable once uploaded |
| API responses | Sometimes | Only if responses are identical across users |
| Personalized feed | No | Different per user, low cache hit rate |
Cloudflare has PoPs in Delhi, Mumbai, Chennai, and Bangalore. For an Indian user base, CDN is not optional — it is the difference between a 200ms page load and a 2-second one.
The five-stage evolution: how real systems grow
Every scaled system follows a similar path. This is not theory — it is the pattern behind every Indian unicorn from Flipkart to Zerodha.
Five-stage system evolution diagram
Stage 1 — Single server:
One machine. App + DB + file storage. Works until ~1,000 DAU. Your college project lives here.
Stage 2 — App server + database server:
Split app and DB onto separate machines. DB gets dedicated CPU and RAM. Works until ~50,000 DAU.
Stage 3 — Load balancer + multiple app servers:
Horizontal scaling of the stateless app tier. Sessions move to Redis. Works until the database chokes. This is where most startups get stuck.
Stage 4 — Database scaling (read replicas + cache + CDN):
Read replicas for query load. Redis for hot data. CDN for static assets. This chapter. Works until ~10 million DAU for most products.
Stage 5 — Sharding + message queues + microservices:
Shard the database. Add Kafka for async processing. Split monolith into services. This is where engineering gets genuinely hard — and genuinely interesting.
| Stage | DAU range | Key addition | Complexity |
|---|---|---|---|
| 1 | 0 – 1K | Single server | Trivial |
| 2 | 1K – 50K | Separate DB server | Low |
| 3 | 50K – 500K | Load balancer + app pool | Medium |
| 4 | 500K – 10M | Replicas + cache + CDN | Medium-high |
| 5 | 10M+ | Sharding + async + services | High |
Do not skip stages. A team at Stage 2 that deploys Kubernetes and sharding because a senior engineer read a Netflix blog post will spend six months debugging distributed systems instead of shipping features.
Combining the tools: a worked example
Imagine you are designing the data layer for a cricket fantasy app during IPL season.
Traffic profile:
- 50 lakh DAU during IPL
- Peak: match days, 7 PM – 11 PM
- 80% reads (leaderboards, player stats, team rosters)
- 20% writes (team selection, point updates)
Architecture:
CDN → static assets (player photos, team logos)
Redis → leaderboards (sorted sets), live scores (TTL 10s), user sessions
Read replicas (×3) → player stats, historical match data, team rosters
Primary DB → team selections, point calculations, payments
Live scores get a 10-second Redis TTL because 10 seconds of stale score data is acceptable for a fantasy app. Wallet balances read from the primary because stale balance data is a lawsuit.
This is not over-engineering — it is matching each piece of data to the right storage tier based on access pattern and consistency requirements.
Common mistakes in data layer scaling
Mistake 1 — caching everything.
A 30% cache hit rate on personalized data means 70% of requests still hit the DB — plus you added Redis complexity. Cache what is actually hot and shared.
Mistake 2 — ignoring cache invalidation.
Phil Karlton was right: there are only two hard things in computer science — cache invalidation and naming things. Plan your invalidation strategy before you add Redis, not after users see stale data.
Mistake 3 — sharding too early.
A 10 GB database with 500 writes/sec does not need sharding. It needs better indexes and maybe a read replica. Sharding adds cross-shard query pain that lasts forever.
Mistake 4 — no monitoring on replication lag.
If your read replicas lag 30 seconds behind the primary during peak traffic, users see data from half a minute ago. Alert on replication lag > 1 second.
Mistake 5 — serving API responses from CDN without cache headers.
CDN only works if you set proper Cache-Control headers. Without them, every request is a cache miss and you are paying for a CDN that does nothing.
Practice with hands-on labs
These labs let you see data layer trade-offs in action:
- Sharding Lab — pick a shard key, watch what happens with hot tenants
- Consistency Lab — feel the difference between strong and eventual consistency
In the sharding lab, try sharding by user ID vs by region. The hot-shard problem becomes obvious in about 30 seconds.
What comes next: back-of-envelope estimation
You now know how to scale the data layer. The next skill is knowing how much to scale — how many servers, how much storage, how much bandwidth.
The next chapter teaches back-of-envelope estimation: powers of 2, latency numbers every engineer should know, and a worked example for designing a social feed.
Continue here: Back-of-Envelope Estimation.