Design a Chat System — WebSockets, Delivery, Presence

Why this session exists

Chat combines durable history with ephemeral connections. The hard question is not just how to push bytes: it is what the sender may believe after an ambiguous failure, how each device catches up, and how the service survives many clients reconnecting together. This lesson designs those boundaries, rather than claiming to reproduce WhatsApp, Slack or Discord.

You will be able to: distinguish new-connection balancing from message routing; trace gateway, conversation authority, route directory, durable log and delivery workers; implement idempotent acceptance and receipt cursors; compare group push/pull costs; and explain presence, offline notification and reconnect recovery. Prerequisites are queue semantics, partitioning, caches and a requirements-first design method.

Prerequisites

Know the difference between TCP receipt and application commit, a primary key and a timestamp, and a database transaction and two independent writes. The related queue/caching/sharding lessons provide background, but no external service is required for the local exercise here.

Optional preparation

The original 30-minute preparation suggestion is a selection budget. Historical video labels below are discovery aids, not verified recording/creator identities, runtime promises or endorsements. Unseen resources and external availability remain unverified; no socket service needs to run for this lesson.

• System Design · Chat Application — optional component/tradeoff overview pointer.
• Design WhatsApp / Facebook Messenger — optional presence and group-chat design pointer, not evidence of either product's implementation.
• Build Realtime Chat with WebSockets — search — a search, not a verified tutorial; evaluate authentication, replay and backpressure before adopting a scaffold.
• Design a Chat System — optional alternative design discussion, not a universal reviewer syllabus.
• How Discord Stores Billions of Messages — original 2017 resource: channel/time-bucket history and message-ID access patterns. The separate 2023 report below covers the later ScyllaDB migration; the 2017 pointer is not evidence for that later event.
• The WebSocket Protocol · RFC 6455 — original handshake/framing reading destination. Its HTTP/1.1 opening handshake and framed TCP transport do not themselves implement durable acceptance or replay.

(a) Intuition · 5 min

The phone-line analogy captures full duplex, but HTTP is not a fresh connection for every letter: persistent HTTP, long polling and SSE all support useful real-time patterns. WebSocket provides framed bidirectional communication over a connection. RFC 6455 defines the HTTP/1.1 upgrade path; modern protocol mappings can differ. A new connection may be round-robin balanced. After selection, its frames stay with that connection's backend. An unrelated message addressed to Alice must find her current user/device/session route; balancer stickiness alone does not locate her from another service.

Connection capacity depends on memory per socket, TLS, heartbeat rate, payload/fanout bandwidth, runtime and failover headroom. There is no defensible universal 100,000–1,000,000 sockets-per-server promise. Measure a disposable workload before selecting capacity. Route-directory saturation, reconnect bursts and fanout are important failure candidates—not a ranked list of all historical chat outages.

Transport decision: polling, SSE or WebSocket

A fallback is a product/network decision, not a law. If supporting two transports, share identity, authorization, receipts and replay semantics so switching transport does not duplicate messages.

The five-layer architecture

This is one database-first design. The transaction stores message identity, conversation sequence and outbox event before durable acceptance. A relay publishes the outbox to a broker; failures can duplicate publication, so consumers deduplicate. A log-first alternative may acknowledge a replicated Kafka append and materialize the database afterward, but then reads must account for materialization lag. Do not draw log-first and describe database-first while leaving two independent writes unexplained.

The route directory is an index of live sessions, not the message source of truth. Use (tenant,user,device,session_generation) and a lease. An old disconnect removes only its own generation, not a newer session's route. A failed directory can delay live delivery while durable history remains available.

Alice sends "hi" to Bob — the full path

1. Alice authenticates; the server derives her tenant/user identity rather than trusting a URL user ID. She sends a stable client operation ID, conversation ID and body.
2. The conversation authority checks current membership, body/attachment limits and dedup identity. In one transaction it assigns a per-conversation order, stores the message and outbox row, and commits. A conflicting reuse of the operation ID is rejected.
3. It returns durably accepted, not “Bob read it.” If this acknowledgement is lost, retrying the same identity returns the same stored message.
4. A delivery worker publishes/retries the outbox and locates Bob's devices. The gateway puts the event in a bounded output buffer; buffer overflow becomes a catch-up signal, not unbounded memory growth.
5. Bob persists the received message locally and acknowledges the highest contiguous persisted sequence for that conversation/device. A separate read receipt is sent only under the product's read-state definition.
6. Receipt state advances monotonically. Alice may see accepted, device-delivered and read as different states. Expired membership, deleted history and permanently absent devices constrain delivery.

The latency budget must include ingress, auth, durable commit, queueing, route lookup, gateway and device acknowledgement. The old 80–150 ms range was not measured; cross-region/mobile tails can dominate. Define p50/p99 and acknowledgement boundary before benchmarking.

The offline path — never lose a message

“No loss” is an objective bounded by retention, durability and recovery policy, not a promise from a push service. Persist first; push notifications are wake-up hints and may be delayed, collapsed or absent. On reconnect, authenticate again, load the persisted-delivery cursor (not the read cursor), fetch authorized messages after it, and deduplicate by stable identity. A read cursor represents human interaction and must not be used to discard received-but-unread history.

A reconnect racing live delivery can see the same message twice. Capture a high-water mark for the initial page/replay, merge subsequent live events, and advance only the contiguous received prefix. Tombstones or explicit unavailable ranges explain gaps caused by deletion/access policy; otherwise dense sequence gaps trigger a history request. Multi-device receipt state is separate so an active laptop cannot falsely acknowledge a sleeping phone.

Group chat — inbox pull is the win

The heading preserves the original bookmark; the actual decision is conditional. One shared conversation log avoids N durable body copies for N members, while per-device receipt metadata and N recipient bytes remain. A 500-member group at 20 messages/minute would create 10,000 inbox-pointer writes/minute under full push, versus 20 shared message appends plus receipts/fanout work. The message bodies need not be duplicated even in a pointer-push design.

Small active groups can push immediately. Large mostly idle groups can retain one log and pull pages on open, with compact notification hints. Gateways can subscribe per group and fan out locally, reducing inter-gateway routing messages but not total output bytes. Choose from activity distribution, acceptable freshness and catch-up cost; do not claim every platform uses one policy or that a fixed group size makes push impossible.

Message ID choice — Snowflake

For an illustrative Snowflake-like design, allocate 41 timestamp, 10 worker and 12 sequence bits plus the unused/sign bit in a signed 64-bit representation. This is a chosen layout for reasoning, not a claim that every named chat product uses those exact bit widths. Timestamp ordering is approximate physical-time ordering, not a total causal order across workers. Worker IDs must be uniquely allocated/fenced; overflow in a millisecond must wait or fail; clock rollback must be handled. “Generated locally” does not remove those coordination obligations.

Use a stable client operation ID for acceptance deduplication and a conversation sequence for authoritative order. A sortable global message ID can still support pagination, but the storage partition and clustering/index order must support that range. In JavaScript, a full 64-bit identifier should be transported as a string rather than an imprecise Number.

Sharding the message store

Discord's 2017 article describes (channel_id,bucket) with a ten-day bucket in its implementation, bounding historical partition size. It does not spread simultaneous writes within the current bucket: a hot channel can still be a hot partition. Discord's 2023 account describes the move from Cassandra to ScyllaDB, with Rust data services—not replacement of ScyllaDB by a custom Rust database.

Choose bucket duration from row size, activity and read ranges. Very hot conversations may need a dedicated authority, admission limits, subpartitioning with explicit merge order, or coalesced reads. A history request spanning buckets must bound fan-in and preserve the cursor. Cassandra clustered ranges are well suited to selected access patterns; PostgreSQL is also plausible at a suitable scale. Product names alone prove neither latency nor durability.

Misconception: an open socket proves delivery

A successful WebSocket library send may mean buffered/enqueued bytes, depending on the API; it does not prove application receipt. A TCP ACK proves transport receipt at the peer endpoint, not a database commit, local device persistence or human reading. The protocol must define its own acknowledgements.

client operation -> authorized durable append -> accepted ACK
                 -> outbox relay -> device buffer -> device persistence ACK
                                                   -> explicit read receipt
Lost ACK: retry same identity; replay may repeat transport, not logical effect.

First principles: order and failover

Physical clocks alone cannot establish a conversation's canonical order. A single logical authority can assign sequence numbers in commit order, but distributed failover needs a lease/consensus epoch and fencing so two leaders cannot both assign the same next value. “One assigner” means coordination is localized, not absent. Clients sort by that order; a per-sender sequence detects sender retries/reordering but cannot order two different senders by itself.

Separate ordering from causality: an explicit reply-to identity can preserve the relationship even if a reply arrives before its parent in a paginated client. Sequence allocation may have holes depending on the database; only claim gap-detection semantics if the append protocol makes committed sequences dense or publishes skipped ranges.

Mental model: durable log plus fallible delivery

Treat every device as potentially offline. Route leases are expendable; durable message identity and history are not. Store only bounded output queues and disconnect slow readers with a resume path. Presence is an approximate hint, not an authorization fact. Subscribe only to visible contacts/groups or query lazily; do not broadcast every heartbeat to every contact. For 500 contacts, one status transition can create 500 deliveries under naive push, versus a bounded number of active-view subscriptions. Heartbeat interval and lease expiry trade freshness against battery, bandwidth and false-offline reports.

(c) Hands-on · 25 min

The original socket scaffold lost messages on process restart, popped offline queues before acknowledgement and called send_text “delivered.” The complete CPU exercise below instead executes the authoritative acceptance/cursor logic in in-memory SQLite. It opens no listener and does not pretend SQLite memory survives restart. The network-facing reference procedure follows it.

import sqlite3
 
class ChatLog:
    def __init__(self):
        self.db = sqlite3.connect(":memory:")
        self.db.executescript("""
        CREATE TABLE members(conv TEXT, user TEXT, PRIMARY KEY(conv,user));
        CREATE TABLE messages(conv TEXT, seq INTEGER, sender TEXT, op TEXT, body TEXT,
          PRIMARY KEY(conv,seq), UNIQUE(conv,sender,op));
        CREATE TABLE outbox(conv TEXT, seq INTEGER, PRIMARY KEY(conv,seq));
        CREATE TABLE receipts(conv TEXT, user TEXT, device TEXT, seq INTEGER,
          PRIMARY KEY(conv,user,device));
        """)
        with self.db:
            self.db.executemany("INSERT INTO members VALUES (?,?)", [("c","alice"),("c","bob")])
 
    def authorize(self, conv, user):
        if not self.db.execute("SELECT 1 FROM members WHERE conv=? AND user=?", (conv,user)).fetchone():
            raise PermissionError("not a current member")
 
    def accept(self, conv, sender, op, body):
        self.authorize(conv, sender)
        if not op or not isinstance(body, str) or not 1 <= len(body) <= 1000:
            raise ValueError("invalid operation or body")
        with self.db:
            prior = self.db.execute("SELECT seq,body FROM messages WHERE conv=? AND sender=? AND op=?",
                                    (conv,sender,op)).fetchone()
            if prior:
                if prior[1] != body:
                    raise ValueError("conflicting identity reuse")
                return prior[0]
            seq = self.db.execute("SELECT COALESCE(MAX(seq),0)+1 FROM messages WHERE conv=?", (conv,)).fetchone()[0]
            self.db.execute("INSERT INTO messages VALUES (?,?,?,?,?)", (conv,seq,sender,op,body))
            self.db.execute("INSERT INTO outbox VALUES (?,?)", (conv,seq))
            return seq
 
    def replay(self, conv, user, after=0):
        self.authorize(conv, user)
        return self.db.execute("SELECT seq,sender,body FROM messages WHERE conv=? AND seq>? ORDER BY seq",
                               (conv,after)).fetchall()
 
    def receipt(self, conv, user, device, seq):
        self.authorize(conv, user)
        last = self.db.execute("SELECT COALESCE(MAX(seq),0) FROM messages WHERE conv=?", (conv,)).fetchone()[0]
        if type(seq) is not int or not 0 <= seq <= last:
            raise ValueError("invalid cursor")
        # Client contract: seq is highest contiguous locally persisted sequence.
        with self.db:
            self.db.execute("""INSERT INTO receipts VALUES (?,?,?,?)
              ON CONFLICT(conv,user,device) DO UPDATE SET seq=MAX(receipts.seq,excluded.seq)""",
              (conv,user,device,seq))
 
log = ChatLog()
assert log.accept("c","alice","op1","hi") == 1
assert log.accept("c","alice","op1","hi") == 1
assert log.accept("c","alice","op2","again") == 2
assert log.replay("c","bob",1) == [(2,"alice","again")]
log.receipt("c","bob","phone",2)
log.receipt("c","bob","phone",1)
assert log.db.execute("SELECT seq FROM receipts").fetchone()[0] == 2
assert log.db.execute("SELECT COUNT(*) FROM outbox").fetchone()[0] == 2
print("chat acceptance, replay, atomic outbox and monotone receipt PASS")

What each block is doing

members is the authorization fixture. UNIQUE(conv,sender,op) records business identity with a payload-conflict check. Message and outbox insertion share one transaction. MAX(seq)+1 is safe only under this single-connection sequential fixture; a real concurrent store needs row locking/serialized allocation or a log authority and leader fencing. replay checks membership again, and receipts cannot go backward. A malicious client could lie about persistence; this model verifies cursor bounds, not the phone's storage hardware.

Security and backpressure are end-to-end contracts

Use TLS (wss) for transport confidentiality and integrity; that is not end-to-end encryption against the service. If end-to-end encryption is a requirement, separately design device keys, membership changes, recovery, attachment encryption and what metadata the server can still see. The server-readable body fixture does not implement that product.

RFC 6455 distinguishes browser Origin checks from authentication: a non-browser client can forge Origin. Derive tenant, user and device from authenticated state, scope operation IDs and every storage/cache/route key to that authority, and authorize send, replay, receipts, presence subscriptions and attachment access. Recheck long-lived sessions when credentials or membership change. Define the policy-version or transaction boundary for a revocation racing acceptance/delivery; an earlier permission check is not permanent permission. Never put bearer credentials or message bodies into routine route/connection logs.

Bound frame size and reassembled message size, input rate, queued output bytes, replay page size, per-tenant work and retry attempts. A message-count limit alone allows a few huge messages to exhaust memory. When a slow recipient exceeds its budget, preserve authoritative history and disconnect with a catch-up path; drop/coalesce expendable typing/presence updates rather than silently discarding accepted durable messages. Apply admission limits before expensive replay or fanout so reconnects cannot monopolize the system. A group membership change must affect both history authorization and live fanout; route existence alone grants neither.

The in-memory fixture has one tenant and a sequential caller, accepts trusted user/device arguments, and returns an unbounded replay list. Its membership lookup and append are not a production concurrent revocation protocol. It never deletes messages: MAX(seq)+1 must not be copied into a delete-capable service, where deleting the highest row could reuse an old sequence. Keep a durable per-conversation high-water mark, tombstones/skipped-range rules, bounded pagination and tested concurrency in the real authority. These are explicit extension obligations, not properties established by this fixture.

Complete WebSocket service procedure — explicitly unexecuted

1. Implement an authenticated upgrade endpoint that derives tenant/user/device and validates Origin where browser credentials make it relevant. Do not use an unauthenticated path ID as identity. Set message-size, connection and idle limits before accepting work.
2. Register a generation-scoped route lease; renew by heartbeat. Permit multiple devices explicitly. A compare-and-delete on disconnect prevents an old socket from deleting a new route.
3. Decode a typed send envelope with operation ID and conversation membership; run the durable transaction; return accepted identity/sequence. Use a file-backed replicated store according to the durability requirement, not the in-memory fixture.
4. Relay outbox events idempotently. Buffer per-device delivery up to a bounded limit; on overflow close with a resumable reason. Never label library send completion device-delivered.
5. The client persists an outbox before sending; receives/deduplicates ordered messages into local storage; sends the highest contiguous persisted cursor; and separately reports read state. Keep unacknowledged sends across reconnects.
6. Reconnect with capped full jitter, for example uniform delay from zero to min(30,0.5×2^attempt) seconds with exponent capped before computation. Reset after a stable connection period, not every successful handshake. On resume, reconcile history and resend the same operation IDs; supervise reader/writer tasks and cancel siblings when either closes.
7. In a disposable service test, drop a connection after commit but before accepted ACK; kill the gateway; restart the store under its persistence policy; race two devices; revoke membership during replay; fill a slow-reader buffer; and drain gateways gradually. Assert identities/cursors/authorization, then measure p99 commit/delivery/reconnect rates. No such server, dependency install or network test is run here.

Group fanout exercise and full worked answer

Compare N=10, 100 and 1,000 recipients at one message each. A sequential per-socket loop performs N sends and its completion includes all awaited sends; that does not prove p99 is exactly linear because batching, concurrency and network tails differ. A shared log performs one append, while actual delivery still sends N payloads when everyone reads. Measure append acknowledgement separately from last-recipient latency, cache/storage pointers and backlog. For a mostly idle group, pull avoids many unused inbox writes; for an always-active small group, immediate push may be cheaper than repeated polling. This answer replaces the invitation to benchmark against a live service without a bounded setup.

(d) Production reality · 15 min

Source-backed incident lessons

Slack's January 4, 2021 report describes infrastructure/network capacity and connection demand during recovery. It is not evidence for the original unsupported maintenance-blip story, millions of retries per second, or a universal six-month industry rollout. Read it as a dependency/capacity recovery case. RFC 6455 section 7.2.3 independently recommends a random delay and increasing backoff after abnormal closure; jitter is a sound mitigation without inventing incident-specific numbers.

Discord's 2017 and 2023 engineering reports support bucketed history, hot-partition problems, a Cassandra-to-ScyllaDB migration and Rust data services. Time bucketing bounds history; concurrent hot reads still need coalescing/traffic control. A synthetic missing-message scenario—commit succeeds, live delivery fails, ACK is lost—shows why a persisted log, outbox and stable retry identity are needed. It is not an attributed 0.01% field incident.

Common failure modes

Reconnect overload needs client backoff and server admission/load shedding and staggered drains. Route updates need generations. Ordering needs an authority, not only client timestamps. Presence needs lease expiry and demand-limited subscriptions. Time buckets need hot-current-bucket handling. Push provider quotas/errors must be interpreted from the actual account/API response; there is no source-backed universal APNs 1,000/s cap here, and silently dropping durable chat content is not a retry policy.

Where this shows up in the rest of the plan

The framework lesson supplies requirements and acknowledgement SLOs; URL shorteners contrast stateless requests with live connection state; newsfeed reuses push/pull tradeoffs; AI chat reuses stream lifecycle and cancellation but may use SSE rather than WebSocket. Queue semantics explain relay replay; observability needs connection counts, route lookup latency, unacknowledged age, delivery latency and reconnect admission—not just request success.

(e) Recall + stretch · 10 min

Explain-out-loud test

Explain the connection/message-routing distinction, the durable transaction-to-receipt path, and how client jitter plus server admission contain a reconnect storm. The worked trace, procedure and ten answers above supply the complete solution; jitter alone is not an outage-proofing switch.

Study route and pacing

Revisit system-design requirements, message queues, caching, load balancers and sharding/partitioning. These topic-matched links avoid the original mismatched S066/S070 labels. URL shortening contrasts request-bound mapping with live connection routing. The original S142 observability extension remains a topic cue: track connection count, reconnect admission, route lookup, unacknowledged age/queue bytes and delivery latency, rather than assuming that label is a current route.

The original 130-session series used a suggested 90-minute block: intuition → visual walkthrough → hands-on → production/failure reasoning → recall/stretch. The timed headings are planning cues, not guaranteed completion times or evidence that external exercises were run; historical “evergreen” wording is not a freshness guarantee.

What comes next

Newsfeed design transfers the fanout reasoning; AI chat design adds expensive cancellable generation. Learning-plan hub connects these lessons to the wider curriculum.

Sources and review boundary

Checked 8 October 2026: RFC 6455, WHATWG SSE, Redis Pub/Sub, Discord 2017, Discord 2023, Slack 2021, OWASP authorization. SQLite and independent local state checks are executed; WebSocket clients/servers, brokers, provider notifications, production failover and load measurements are not.