System Design Layers

NFR answers tell you what the system needs. Layers are what you draw. Start with the core 4, then add layers based on what the interviewer tells you.

The Core 4 — Always Draw These

Every system design diagram starts here. Add more only when NFRs demand it.

Copy[Client]
   ↓
[Load Balancer]
   ↓
[App Server(s)]
   ↓
[Database]

Optional Layers — Added by NFRs

Add only when an NFR demands it. Each one solves a specific problem but adds latency, cost, and operational complexity. Use the trigger column to decide during requirements gathering.

NFR → Layers to Add

Translate the interviewer's requirements directly into layers. Use this to build your diagram from the answers you've gathered.

Latency Reference — Cost Per Layer

Single source of truth for per-layer latency costs. Use the Typical column for estimates — Best Case shows the theoretical floor with everything going right.

Network is included: Each component figure (Redis 1ms, DB 5ms, etc.) covers the full round-trip including the internal network hop — App→component→App. The only network cost listed separately is the user-facing leg (Browser↔LB) because that varies by geography and is outside your control.

Scenario breakdowns below use the Typical column. Update this table to recalibrate all scenarios.

Copy                                              ┌─(1ms)──► Redis
Browser ─(20ms)─► CDN ─(1ms)─► LB ─(5ms)─► App ─┼─(5ms)──► DB
                                              ├─(3ms)──► Kafka
                                              └─(50ms)─► S3

Scenario 1: Read-Heavy

Design Twitter feed, Reddit homepage, YouTube video page.

NFRs that triggered this: high scale, read-heavy, low latency

Copy[Client]
   ↓
[CDN] ──────────────────→ (cache hit: return static assets / cached response)
   ↓ (miss)
[Load Balancer]
   ↓
[App Servers]
   ↓
[Cache (Redis)] ─────────→ (cache hit: return feed)
   ↓ (miss)
[DB Primary] + [DB Read Replicas]

Latency breakdown:

Key decisions:

• Feed data is precomputed and stored in Redis. App servers read from cache, not DB.
• Writes go to DB primary only. Read replicas absorb the bulk of DB read traffic.
• CDN handles profile images, thumbnails, static JS/CSS.

Scenario 2: Write-Heavy

Design an analytics pipeline, logging system, IoT sensor ingestion.

NFRs that triggered this: high scale, write-heavy, fault tolerance

Copy[Client / Producers]
   ↓
[Load Balancer]
   ↓
[App Servers]  ← accept writes, validate, publish to queue
   ↓
[Message Queue (Kafka / SQS)]  ← absorbs bursts, durable buffer
   ↓
[Workers / Consumers]  ← process at own pace, retry on failure
   ↓
[Database / Data Warehouse]

Latency breakdown:

Key decisions:

• App servers never write directly to the DB — they publish to the queue and return 200 immediately.
• Queue decouples ingestion speed from processing speed. If workers are slow, queue grows but nothing drops.
• Workers can be scaled independently. Failed jobs stay in queue for retry.

Scenario 3: Strong Consistency Required

Design a payment system, hotel booking, airline seat reservation.

NFRs that triggered this: strong consistency, durability, security

Copy[Client]
   ↓
[Load Balancer]
   ↓
[API Gateway]  ← auth, rate limiting, fraud checks
   ↓
[App Servers]
   ↓
[DB Primary]  ← all reads AND writes go here (no cache on write path)
   ↓ (synchronous)
[DB Standby]  ← hot standby, promoted on failure
   ↓
[Audit Log]  ← append-only record of every transaction

Latency breakdown:

App servers do more work here (auth checks, business rule validation) but the dominant cost is sync replication — that's the price of RPO = 0.

Key decisions:

• No cache on the write path — stale reads could cause double-booking or double-charging.
• DB standby is synchronous (primary waits for standby to confirm before acking write).
• API Gateway handles rate limiting to prevent abuse at the payment endpoint.
• Audit log is append-only and separate — required for compliance and debugging. Written asynchronously after the DB confirms, so it is not on the user's critical path and does not add to the ~91ms above.

Scenario 4: Low Latency

Design autocomplete/typeahead, live leaderboard, stock price feed.

NFRs that triggered this: latency < 100ms, read-heavy, high scale

Copy[Client]
   ↓
[CDN / Edge Cache] ──────→ (return if result cached at edge)
   ↓ (miss)
[Load Balancer]
   ↓
[App Servers]
   ↓
[Cache (Redis)] ──────────→ (precomputed results — return immediately)
   ↓ (cold start / miss only)
[Database]

Latency breakdown:

At < 100ms target, even a DB miss is within budget — but only if the query is indexed. The goal is 99%+ Redis hit rate so DB is never on the hot path.

Key decisions:

• DB is off the hot path entirely. Cache must have near-100% hit rate for common queries.
• Results are precomputed and pushed into Redis (e.g., top 10 autocomplete results per prefix).
• CDN caches at the edge for global users — reduces round-trip time before even hitting your servers.
• Any update to results is pushed into Redis asynchronously, not on the request path.

Scenario 5: Balanced / General Purpose

Design Uber, Airbnb, a general marketplace.

NFRs that triggered this: moderate scale, availability, latency, some consistency

Copy[Client (Web + Mobile)]
   ↓
[CDN]  ← static assets only
   ↓
[Load Balancer]
   ↓
[API Gateway]  ← auth, rate limiting
   ↓
[App Servers]
   ├──→ [Cache (Redis)]   ← hot reads (search results, listings)
   ├──→ [Message Queue]   ← async tasks (email, notifications, billing)
   │         ↓
   │     [Workers]
   ↓
[DB Primary] + [DB Read Replicas]
   ↓
[Object Storage (S3)]  ← user photos, listing images

Latency breakdown:

Key decisions:

• Most features read from cache, write to DB. A few critical paths (booking, payment) skip cache and write directly to primary.
• Message queue handles notifications, confirmation emails, and analytics events — nothing that should block the user's request.
• Object storage for all media. App servers only store the URL reference in the DB.

Scenario 6: Real-Time / Fan-Out

Design a chat system, live notification feed, live scoreboard, collaborative editor.

NFRs that triggered this: real-time push, low latency, high availability

Copy[Client A]  [Client B]  [Client C]
    ↕            ↕           ↕       ← persistent WebSocket connections
[Load Balancer]  ← sticky routing (session affinity by connection ID)
    ↓
[WebSocket Servers]  ← stateful — each holds thousands of open connections
    ↑   (all servers subscribe to relevant channels)
[Message Broker (Redis Pub/Sub or Kafka)]
    ↑
[App Servers]  ← receive events, validate, publish to broker
    ↓
[DB Primary]   ← persist messages / events

The fan-out problem: An event (new message, score update) must reach all connected clients, who may be connected to different WebSocket servers.

CopyClient A sends message
   → App Server processes + writes to DB
   → publishes event to broker channel
   → all WebSocket Servers subscribed to that channel receive it
   → each pushes to its connected clients (B, C, etc.)

Latency breakdown:

Key decisions:

• WebSocket servers are stateful — a client's connection lives on one server. The LB must use sticky routing (session affinity) so reconnects land on the same server, or all servers must subscribe to all channels.
• Redis Pub/Sub for fan-out when durability isn't needed (notifications, live scores). Kafka when you need replay or durability (chat history, audit trail).
• Each WebSocket server can hold ~10K–100K concurrent connections. At 1M concurrent users you need 10–100 servers — plan capacity around connection count, not QPS.
• Heartbeat / ping-pong on each connection to detect stale clients and free resources.
• DB stores message history. WebSocket layer is for delivery only — clients fetch history via a normal REST API on connect.