SYSTEM DESIGN:Lesson 8: Scaling Short Links to Millions

IPL final, one link, 2 million clicks

A cricket fan shortens a Hotstar match link and posts it on X during an IPL final. Within 90 minutes, 2 million people click it. Your URL shortener — which handled 500 redirects per second yesterday — is now facing 6,000 per second and climbing.

Create (POST /urls) traffic barely moved. Redirect (GET /x/code) is 100× heavier. That ratio is the entire design.

This chapter takes the core shortener from the previous lesson and scales it: cache, CDN, database choices, napkin math, and the pushback interviewers throw at your diagram.

One IPL link, two million clicks, zero chill

POST /urls barely moved. GET /x/code became a national sport.

Read-heavy architecture

URL shortener at scale architecture

Layered read path (fastest to slowest):

Browser → CDN edge (cached 301) → Redis cache → Read replica → Primary DB

Write path stays simple:

Client → App server → Primary DB (+ invalidate cache entry)

Every layer absorbs traffic so the layer below sees less. Goal: 95%+ of redirects never touch your database.

Redis cache — cache-aside pattern

Cache-aside on redirect: Redis first, DB on miss

On redirect:

def redirect(short_code: str):
    # 1. Check Redis
    long_url = redis.get(f"url:{short_code}")
    if long_url:
        increment_click_async(short_code)  # do not block redirect
        return redirect_301(long_url)

    # 2. Cache miss — hit database
    row = db.query("SELECT long_url FROM urls WHERE short_code = ?", short_code)
    if not row:
        return 404

    # 3. Populate cache
    redis.setex(f"url:{short_code}", 86400, row.long_url)  # 24h TTL
    return redirect_301(row.long_url)

TTL choice: 24 hours is common. Viral links stay hot; stale links fall out of cache naturally. Adjust based on memory budget.

Cache invalidation on update/delete:

def delete_url(short_code: str):
    db.delete(short_code)
    redis.delete(f"url:{short_code}")

At 6,000 req/s with 95% cache hit rate, Redis sees ~300 misses/sec — trivial for a single Redis instance (100k+ ops/sec). The cache is doing its job.

CDN for redirect responses

A 301 redirect is tiny — a few hundred bytes. But 2 million of them from Mumbai hitting your US-East server adds latency and bandwidth cost.

Put CloudFront, Cloudflare, or Fastly in front:

• Edge caches the 301 response including Location header
• User in Delhi gets redirected from an edge node 10 ms away, not Virginia 250 ms away
• Origin load drops by another 80–90% on top of Redis

301 vs 302 revisited at scale:

• 301 + CDN — redirect cached at edge for days. Origin barely touched. Click analytics undercount (CDN serves cached redirect, your server never sees the hit).
• 302 + CDN — shorter cache TTL at edge, more origin hits, accurate analytics.

bit.ly uses 301 for speed. If the interviewer asks about analytics accuracy, say: "Track clicks via a separate beacon pixel or use 302 for links where count matters."

During a viral event, CDN cache hit ratio above 99% is normal. Your origin might see 60 req/s while the world sees 6,000.

Database schema at scale

CREATE TABLE urls (
  id          BIGINT PRIMARY KEY,        -- Snowflake or auto-increment
  short_code  CHAR(7) NOT NULL,
  long_url    VARCHAR(2048) NOT NULL,
  user_id     BIGINT NULL,
  created_at  TIMESTAMP NOT NULL,
  expires_at  TIMESTAMP NULL
);

CREATE UNIQUE INDEX idx_short_code ON urls(short_code);

Why CHAR(7) not VARCHAR? Fixed length, slightly faster index lookups. Minor optimization.

Separate analytics table — do not UPDATE click_count on every redirect (write amplification on read path):

CREATE TABLE click_events (
  short_code  CHAR(7) NOT NULL,
  clicked_at  TIMESTAMP NOT NULL,
  country     VARCHAR(2),
  referrer    TEXT
);

Buffer clicks in Redis (INCR url:clicks:kF3xP), flush to analytics DB every 60 seconds via a background worker. Redirect stays fast.

Database choice:

• Postgres/MySQL — fine up to ~10k writes/sec with sharding. Familiar, ACID.
• Cassandra/DynamoDB — if you need horizontal write scaling across regions. Partition key = short_code.

For most interviews, Postgres + read replicas + Redis cache is enough. Mention NoSQL as the "if we outgrow Postgres" path.

Capacity estimate (napkin math)

Assumptions (state these out loud in interviews):

• 100 million new URLs per month
• 100:1 read-to-write ratio
• Average long URL: 500 bytes

Writes:

100M URLs / month ÷ 30 days ÷ 86,400 sec ≈ 39 writes/sec average
Peak (3× average): ~120 writes/sec

Any database handles 120 writes/sec. Writes are not your problem.

Reads:

39 writes/sec × 100 = 3,900 reads/sec average
Peak (3×): ~12,000 reads/sec
Viral event (10× peak): ~120,000 reads/sec

Storage (5 years):

100M/month × 12 months × 5 years = 6 billion URLs
6B × (7 bytes code + 500 bytes URL + 20 bytes metadata) ≈ 3 TB

Redis memory (hot cache):

Top 20% of links get 80% of traffic (Pareto)
Cache 200M hottest URLs × ~600 bytes ≈ 120 GB Redis cluster

These numbers do not need to be exact. Interviewers want to see you decompose the problem, not calculate to the byte.

ID generation at scale

Single auto-increment counter breaks across database shards. Options:

• Snowflake IDs — 64-bit, timestamp + machine ID + sequence. Twitter's approach. Encode to base62 for short code.
• Range allocation — Server A gets IDs 1–1M, Server B gets 1M–2M. Simple, works for moderate scale.
• UUID → truncate — risky for collisions; avoid unless combined with uniqueness check.

For the interview: "Counter per shard, or Snowflake if we need global uniqueness without coordination." Move on — do not spend 10 minutes on ID generation unless asked.

What interviewers push back on

"Your cache and DB will be inconsistent"

Yes, briefly. If you delete a URL in DB and Redis still has it, users get redirected for up to TTL seconds. Acceptable for URL shorteners — not a bank ledger. Mitigate with explicit cache delete on write path.

"How do you handle a hot key in Redis?"

One viral link = one Redis key getting 50k reads/sec. Redis handles single-key reads well, but at extreme scale, replicate the hot key across Redis nodes or let CDN absorb it (301 cached at edge — Redis never sees those requests).

"Why not store everything in Redis?"

Memory cost. 6 billion URLs × 600 bytes = 3.6 TB of Redis. At ~$5/GB/month for managed Redis, that is ₹1.5 crore/month. Database + cache for hot data is cheaper.

"Custom domains (branded short links)?"

Add domain column to schema. Route links.company.com/x/abc via DNS to your service. Lookup by (domain, short_code) composite key. CDN configured per custom domain.

"Multi-region?"

Redirects are latency-sensitive. Deploy read replicas + Redis in each region. Writes go to primary region (or use CRDT/conflict-free replication for URL creation). CDN already handles most multi-region read load.

"Spam and abuse?"

Rate limit URL creation (previous chapters). Scan long URLs against Google Safe Browsing API. Block suspicious TLDs. Require auth for custom aliases.

Load balancer and app tier

Even with CDN and Redis, some requests reach origin. Size your app tier for cache misses and write traffic:

12 app servers × ~1,000 req/s each = 12,000 req/s origin capacity
With 95% CDN+Redis hit rate, 120k viral req/s → ~6,000 origin req/s → fits comfortably

Stateless app servers behind an ALB. Scale horizontally on CPU or request count. No sticky sessions needed — all state lives in Redis and DB.

Use connection pooling to Postgres (PgBouncer). Redirect path should hold a DB connection for milliseconds on cache miss only.

Read replicas

Writes to primary, cache-miss reads from replicas

Point cache-miss lookups at a read replica, not the primary. Writes (new URLs) still go to primary. Replication lag of 100–500 ms is fine — worst case a brand-new link misses cache and replica, user gets 404 once, retries, succeeds.

For URL shorteners, eventual consistency on the read path is acceptable. Mention it proactively — interviewers like when you name the consistency model without being asked.

Monitoring at scale

• Redirect latency p50/p99 — p99 > 200 ms means cache or DB trouble
• Cache hit ratio — target > 95%; drop means new viral link or TTL too short
• CDN hit ratio — target > 90% on redirect responses
• Create error rate — spikes mean DB or counter issues
• 404 rate on redirects — typos vs broken links vs attack

The 5-minute interview answer

1. Clarify: shorten + redirect, 100:1 read/write, 100M URLs/month
2. API: POST /urls, GET /x/{code} with 301
3. ID: Snowflake or counter + base62, 7-char codes
4. DB: Postgres, index on short_code, analytics in separate async pipeline
5. Cache: Redis cache-aside, 24h TTL, 95%+ hit rate
6. CDN: cache 301 at edge for global latency and origin protection
7. Napkin math: ~120 writes/sec peak, ~12k reads/sec peak, CDN absorbs viral spikes

Draw the diagram top-down: Client → CDN → Load Balancer → App → Redis → DB. Label read and write paths differently. That alone puts you ahead of candidates who jump straight to "use MongoDB."

Wrap-up checklist (if the interviewer says "anything else?")

Use this when you have two minutes left. Tie back to other Schoolabe chapters:

Scaling checklist (verbal):

• [ ] Redirect path: CDN → Redis → read replica → primary (only on miss)
• [ ] Create path: rate limited → primary DB → invalidate cache if updating
• [ ] Hot key plan: viral link cached at CDN; Redis optional second line of defense
• [ ] Monitoring: redirect p99, cache hit ratio, 404 spike, create error rate

You do not need to build all of this on day one. Say what you ship at 1M links/day vs 100M — interviewers want evolution, not a day-one Kubernetes cluster.

What comes next: distributed primitives

URL shorteners, KV stores, and CDNs all need a way to spread keys across machines without reshuffling the world every time you add a server. That is consistent hashing.

Continue here: When Hash Modulo Breaks.

Practice under pressure

Run a timed mock: Mock Interview — design a URL shortener in 45 minutes with feedback.

Browse system design questions for follow-up drills: Interview Prep.