Redis PubSub vs Kafka — When to Use Which for Broadcast

The Doubt

In the Google Drive write pipeline, operations go through Document Service → Kafka → Broadcast Consumer → WebSocket. Does that Kafka hop delay what collaborators see?

Short answer: 10–50ms — acceptable for humans (real-time feels instant under ~100ms). But there’s a better design: don’t use Kafka for broadcast at all.

Two Paths, Not One Chain

Option A — Broadcast through Kafka (simpler on paper):
Document Service → Kafka → Broadcast Consumer → WebSocket → Priya
→ Extra 5–15ms from Kafka
→ Decoupled, replayable, but slower

Option B — Fast path + slow path (what production uses):
Document Service → Redis PubSub → WebSocket → Priya     (fast, ~2–5ms)
Document Service → Kafka → Cassandra                     (async persistence)

Kafka is for durability. Redis PubSub is for real-time signaling. Mixing them on one path forces durability overhead onto the hot path.

Why Redis PubSub — Same Pattern as Lesson 11

From the Notification System design:

Notification for user u456:
→ Which of 2,000 SSE servers holds u456's connection?
→ Publish to Redis channel "user:u456"
→ Only that SSE server is subscribed → pushes to user

Google Drive is the same problem, different granularity:

Lesson 11:  channel per USER     → "user:u456"
Google Drive: channel per DOCUMENT → "doc:doc123"

1,000 WebSocket servers
Raj → Server 347, Priya → Server 892, Kumar → Server 156

Document Service doesn't know which server holds whom ❌

Fix:
→ Each WebSocket server subscribes to "doc:doc123" when a user opens that doc
→ Raj types → Document Service publishes to "doc:doc123"
→ All subscribed servers receive → push to their connected users
→ One publish, all collaborators notified ✅

Redis PubSub vs Kafka

For collaborator broadcast:

• Need replay? No — gap detection via sequence numbers catches missed ops
• Need sub-ms? Yes — minimize Priya’s latency
• Message loss OK? Yes — client requests missing ops if gap detected

For persistence:

• Need replay? Yes — crash recovery, snapshot workers
• Latency critical? No — Raj already has optimistic update
• Message loss OK? No — every edit must be saved

Corrected Write Pipeline

Raj types "H"
→ Raj sees "H" instantly (optimistic update)
→ WebSocket → Document Service
      ↓
Document Service (three parallel paths):

Path 1 — Redis PubSub "doc:doc123"     → Priya in ~2–5ms
Path 2 — Kafka "document.operations"     → Cassandra in ~100ms (batched)
Path 3 — ACK + seq number to Raj         → ~2ms

Lesson Connections

Rule of thumb: Redis PubSub whenever you need a real-time signal routed across many servers and persistence doesn’t matter. Kafka whenever the work must survive crashes and be replayable.

Which Architecture Is Better?

For Google Drive specifically: Redis PubSub for broadcast + Kafka for persistence is the better architecture. Not Kafka for everything.

Two separate paths. Each optimal for its job.

When pure Kafka for broadcast is acceptable:

• Collaborators seeing updates in 50–100ms is fine
• Team wants simpler ops (one system, not two)
• Strict ordering within partition matters more than latency

Production reality: Google Docs and Figma use direct WebSocket broadcast for collaboration — latency is prioritized over architectural simplicity.

Redis Does Not Run Inside Document Service

Common misconception:

Wrong:
┌─────────────────────────┐
│ EC2 (Document Service)  │
│  ┌───────────────────┐  │
│  │ Redis (embedded)  │  │
│  └───────────────────┘  │
└─────────────────────────┘
→ EC2 handles ~50k msg/sec, Redis claims 1M ops/sec — contradiction ❌

Correct:
Document Service fleet (200 EC2 instances)  →  Redis Cluster (20 dedicated servers)
         ↕ network calls                              ↕
Kafka cluster, Cassandra, WebSocket gateways

Redis is its own fleet of memory-optimized servers. Document Service calls Redis over the internal network — same as it calls Kafka or Cassandra.

But Document Service is still a bottleneck if not scaled:

10M ops/sec → must pass through Document Service first
Single EC2: ~50,000 WebSocket messages/sec
→ Need 200 Document Service instances

Load balancer routes by document:
hash(documentId) % 200 → always same instance for same doc
→ Natural ordering, no sequence conflicts
→ Each instance: 50,000 ops/sec ✅

Redis load after scaling:

Each op = 1 INCR + 1 PUBLISH = 2 Redis calls
10M ops/sec = 20M Redis calls/sec
Single Redis node: ~1M ops/sec → need 20-node Redis Cluster
hash(documentId) % 20 → each node ~1M ops/sec ✅

Full scaling breakdown is in the Google Drive post (Step 8).

The Principle

When you hit a component’s limit, don’t look for a faster component — distribute load across more instances. Route so correctness is preserved (same document → same Document Service instance).