System design

Wavelink, a real chat app under system-design/chat-demo

This one isn't a story about a made-up company. It's a real Elixir/Phoenix backend and React frontend, living at demo/chat-demo in this site's own repo, built specifically to make the design decisions below checkable in code instead of taking my word for them.

Topic

System design, working code

Source

demo/chat-demo in this repo — backend/ (Elixir, Phoenix), ui/ (React, Vite)

Updated

August 2026

What it is

Wavelink is a small WhatsApp-shaped chat app: register a username, see who else is online, send messages, watch the tick go sent → delivered → read. No password, no server-side auth beyond a unique username — that's a deliberate cut, not an oversight, so the project stays about message delivery rather than identity.

The backend is Phoenix, chosen specifically for Channels and Presence, the two pieces that make "push a message to whichever open connection this user currently has" a library call instead of a project. The frontend is a plain React SPA that talks to it over the official phoenix JS client, no state library, one hook.

Functional requirements

Register with a username and immediately be reachable, with no email or password step in the way.
See which other registered users are online right now.
Send a 1:1 message and have it show up on every open tab/device the recipient is signed into.
Know, as the sender, whether a message was accepted by the server, reached the recipient's device, and was actually read.
Receive messages sent while offline, replayed in order once back online.
Get a browser notification for a message that isn't the conversation currently in view.
Create a group with a name and a set of members, send and receive messages in it, and add or remove members after the fact.
See, per message sent to a group, roughly how many members have received and read it, without that costing more the bigger the group gets.

Non-functional requirements

A message to an online recipient should arrive over the open socket, not on a poll — effectively immediate.
No message should be lost because a channel process crashed or the recipient happened to be offline at send time.
The persistence layer should be swappable without touching call sites, since a demo running in dev shouldn't need a real database, but the design should still hold up against one.
Two clients racing to register the same username should never both win.
Online/offline state doesn't need to survive a restart — it's allowed to be wrong for a moment and rebuild itself from whoever's actually connected.
Sending a message to a group shouldn't cost more database writes the bigger that group gets — a 200-member group and a 2-person DM should both be one write to the message log per message sent.

Features actually built

Username registration, race-safe. POST /api/register does an atomic conditional write (DynamoDB's attribute_not_exists(username) in prod, ETS insert_new in dev) instead of check-then-write, so two clients can't both claim the same name. Re-registering an existing name isn't an error, it's a resume.
Live directory. A new registration broadcasts on a shared directory:lobby topic so every connected client's contact list updates without a refresh.
Messaging with offline delivery — DMs and groups on one path. Every message is written to a durable, conversation-keyed store before any push is attempted. A DM and a group are the same kind of conversation underneath (see Group chat below) — joining a conversation's channel replays anything written while you were away, there's no separate "offline" code path, just a store read that happens to return nothing when you were online the whole time.
Sent / delivered / read, via a cursor, not a flag per message. Each member of a conversation has one last_delivered_id / last_read_id pair; acking a whole backlog after being offline moves that cursor once, not once per message. A DM's tick and a group's "Read 3/4" are both read straight off those cursors.
Multi-tab / multi-device, for free. Every open tab joins the same conversation topic directly, so a second tab of the same account sees an outgoing message the instant it's sent — no special-casing the sender's own topic the way a per-recipient design would need.
Presence. Phoenix Presence tracks who's connected right now, separately from the durable directory of who has ever registered. An online/offline dot follows from that in the inbox, the new-chat panel, and the chat window.
Browser notifications. Permission is requested from the register button's click (a real user gesture, not on page load), notifications are tagged by sender so a burst collapses into one, and suppressed entirely when that conversation is already the focused window.
Group chat. Create a named group, add or remove members, send and receive messages in it — all on the same conversation/message/ channel shape a DM uses, not a parallel system. See the dedicated section below for why that unification is the whole design story here.
Health check. GET /healthz, written for an ALB target group that doesn't exist yet — see below.

Architecture

One Phoenix node, one socket per client carrying three channels: a per-user InboxChannel (the rooms list), a ConversationChannel joined only for whichever conversation is actually open, and a shared DirectoryChannel for the contact list. A DM and a group both live behind the same ConversationChannel — see Group chat below for why that unification, not a separate group system, is the actual design decision here. Storage, the user directory, and conversation membership all sit behind a small behaviour so the same code path runs against ETS in dev/test or DynamoDB in prod — picked by one runtime config value, not a different code branch.

This shows the shape before group chat: two channels on one socket. Group chat adds a third, ConversationChannel, joined per open conversation rather than per user — see the diagram in Group chat below. The behaviour boundary here is what lets dev run on nothing but ETS while prod points the same code at DynamoDB.

A message, end to end

The write-before-push ordering is the one decision the rest of the messaging feature hangs off: a message exists durably before the server ever tries to hand it to a live connection, so "recipient is offline" isn't a special case, it's just a push that doesn't happen yet.

Every ack — sent, delivered, read — travels the same channel/broadcast path as the message itself. No separate acknowledgment system.

Group chat

The design decision here isn’t “add a groups feature”, it’s that a DM stopped being its own special case. Wavelink.Conversations resolves both a DM (an id deterministic from the two usernames, sorted, created lazily the first time either side opens it) and a group (an explicit id, handed out at creation) down to one shape: a conversation with members. Everything downstream — the message store, the channel, the read state — only ever sees that one shape.

That unification is what made fan-out the real problem to solve, not membership management. The old per-recipient store wrote one row per recipient per message — fine at two, not at two hundred. The messages table is now keyed by conversation_id instead: one row per message, regardless of member count. ConversationChannel, topic conversation:<id>, is what every member’s every open tab joins directly, so Phoenix PubSub fans the broadcast out to N sockets on the way out instead of the write side fanning out to N rows on the way in.

The message itself is written once and pushed once, to whichever members are actually connected. The inbox update is the cheap part — a small event, not a write — sent to every member regardless of whether they have this conversation open.

Read state follows the same shape either way: a (conversation, user) cursor (last_delivered_id / last_read_id), not a flag per message per recipient. A DM’s tick and a group’s “Read 3/4” are the same comparison — message id against the other members’ cursors — run over one member instead of several. The tradeoff, named rather than hidden: a group’s tick only ever reflects the slowest member, the same convention WhatsApp uses, not a per-member breakdown.

What this still doesn’t solve: Phoenix.PubSub and Presence are still the local, single-node adapter (see below), so a broadcast to a very large group is still N pushes from one BEAM node’s one channel process. Group chat made that ceiling more visible, it didn’t raise it — horizontal scale is still the infra/modules/scale work named but not built, see below.

Complexity worth naming

Message ids without a global clock. "<ms-timestamp>-<per-node monotonic integer>" is sortable and effectively unique with no shared sequence, no lock, and no coordination between requests — it doubles as the DynamoDB sort key.
Registration races solved with a write, not a check. Both the DynamoDB and ETS implementations of Directory.register/1 use an atomic conditional insert rather than "look it up, then write if free," which is the gap where two simultaneous registrations for the same name would otherwise both succeed.
A behaviour boundary instead of an environment flag. Wavelink.Store and Wavelink.Directory are each an Elixir @behaviour with a Memory and a Dynamo implementation. Swapping one for the other is a config value in config/runtime.exs, not an if env == :prod scattered through the business logic.
Deliberately unscaled, and it says so. The Memory store is a single GenServer-owned ETS table, and its own moduledoc calls that out as fine for a demo's traffic and not a claim about production concurrency. Presence and PubSub both run on Phoenix's local, single-node adapter — real horizontal scale would need a distributed PubSub adapter and clustered Erlang nodes, neither of which is configured.
A clustering hook that's wired up but inert. A DNSCluster child process starts on boot, reading a DNS_CLUSTER_QUERY env var that nothing in the repo ever sets — the multi-node path exists in code but has nowhere real to point yet.
One narrow, defended footgun. Message status uses String.to_atom/1 rather than the safer to_existing_atom/1, normally a way to let a client grow the atom table unbounded. It's safe here only because a pattern-match guard upstream already closes the input to exactly two literal strings, "delivered" and "read" — and the code says so, rather than leaving that safety argument implicit.

Tech stack

Backend: Elixir ~> 1.17, Phoenix 1.8, served by Bandit rather than Cowboy. ex_aws / ex_aws_dynamo for the DynamoDB-backed store. No Ecto, no Postgres, no LiveView — it's a JSON API plus Channels, with the UI as a fully separate SPA.
Frontend: React 19 on Vite, TypeScript, the official phoenix npm client for the socket. No router, no Redux/Zustand — view state lives in one hand-rolled hook lifted into App.tsx. Linted with oxlint instead of ESLint. Plain CSS, no Tailwind.
Data: DynamoDB in prod (messages partitioned by conversation id and sorted by message id; users partitioned by username; conversations by id; memberships partitioned by conversation id, sorted by user id, with a by-user GSI so “what conversations is this user in” is a query, not a scan), ETS in dev/test behind the same interface.
Realtime: Phoenix Channels over one WebSocket per client, Phoenix PubSub (local adapter) for fanout, Phoenix Presence for who's online.
Infra: infra/ has the shape of a Terraform layout — modules/core, groups, media, scale — but no .tf files exist yet; the conversations and memberships tables group chat needs are named in config (DYNAMO_CONVERSATIONS_TABLE, DYNAMO_MEMBERSHIPS_TABLE) the same way the messages table always was, still waiting on modules/groups. See below.

Other things worth knowing

Infra is scaffolded, not built. The module directory names and config-driven env vars (DYNAMO_MESSAGES_TABLE, ALLOWED_ORIGINS, an example origin of chat-demo.rishavraj.info, AWS_REGION defaulting to ap-south-1) sketch an ALB-plus-DynamoDB deployment that hasn't been written in Terraform yet, and there's no Dockerfile or CI pipeline either. The /healthz endpoint exists specifically for a target group that doesn't exist.
No auth is a scope decision. A unique username is the entire identity system, on purpose, so the project stays about delivery, presence, and ordering rather than becoming an auth project wearing a chat UI.
Horizontal scale is still named but not built; group chat and media are not anymore. The groups infra module now matches real code — see Group chat above. Attachments are handled too, not by an upload endpoint inside this app but by a separate standalone service, media-service, that Wavelink’s backend calls — see its own write-up for why that’s a different service rather than a module in here. scale is still just a directory name: no clustered PubSub, no second BEAM node.
The code comments are the real design doc. Nearly every non-trivial function in backend/lib carries a moduledoc explaining the "why," and several point back at this write-up by name — the two were built together, not one after the other.

All system design notes · Home