Figma: Scaling Postgres 100× and Real-Time Multiplayer
This page is one interview loop in three rounds, built on a real company's public engineering history. All three rounds design the same product: Figma, a design tool where many people edit one file at the same time in a browser. Behind it sit two very different systems. One keeps each open file in memory and merges everyone's edits in real time. The other is a relational database that remembers who owns what: files, teams, permissions, comments. Each round is an era of that second system, and each era ended because the one before it ran out of room.
| Round 1: Mid-level | Round 2: Senior | Round 3: Architect | |
|---|---|---|---|
| Era | About 2016–2020: multiplayer servers and one big Postgres | 2020–2022: vertical partitioning under pressure | Late 2022–2024: horizontal sharding without stopping |
| Level (Amazon) | SDE II (L5) | Senior SDE (L6) | Principal (L7) |
| Database (published) | One RDS Postgres instance, upgraded in 2020 from r5.12xlarge to r5.24xlarge, "the largest instance available" | About a dozen vertically partitioned databases by the end of 2022; the last move took 50 tables | First horizontally sharded table in September 2023 |
| Growth (published) | Database traffic "grows approximately 3x annually" | Same | The database stack "has grown almost 100x since 2020" |
| Traffic we plan for | 60,000 queries/s and ~10,000 row writes/s at the peak (writes: the order of magnitude Figma published for 2021; queries: assumption) | ~30,000 row writes/s across all databases (one more year of 3× growth, derived) | One hot table: 6 TB, 30,000 row writes/s, 120,000 IOPS (assumptions); 540,000 queries/s fleet-wide (derived) |
| Target | Edits feel instant; the database keeps up | Headroom without long outages | Near-zero-downtime shard splits that can be rolled back |
| Reading time | ~35 min | ~40 min | ~45 min |
You can start at any round. Rounds 2 and 3 open with a "Where we left off" summary that catches you up.
How to read a case study. Every claim about what Figma actually did comes from Figma's engineering blog, and each round ends with a Sources list. We mark those claims Figma published or (published). Where Figma hasn't published the details, we say so and show a design that fits, labeled as ours. Numbers marked assumption are ours, chosen to make the arithmetic concrete. They are not Figma's internal figures.
Cloud note. Figma runs on AWS: its 2019 infrastructure post describes "a single database instance running on one of the beefiest machine in AWS" in its "datacenter in Oregon", the database posts describe Amazon RDS for PostgreSQL, and the 2022 multiplayer post describes checkpoints in S3 and a journal in DynamoDB. Where this page names other AWS pieces (instance classes for new shards, CloudWatch alarms, quotas), those are our choices. Prices are us-west-2 (Oregon) on-demand list prices, checked September 2026, and a month is 30 days (720 hours) everywhere on this page. Runways, which grow at a yearly rate, convert years to months at 12 months per year.
Loop Opener: Why Figma?
A Live Whiteboard Backed by a Filing Cabinet
Picture a whiteboard that ten people draw on at once, from different countries, and a filing cabinet in the corner that remembers who owns each whiteboard, who may see it, and every comment pinned to it.
- The whiteboard is live. Every drag and every keystroke must show up for everyone, right away. Figma's clients "send updates every 33ms (30 FPS)" while someone edits (2022).
- The filing cabinet is relational. Files belong to projects, projects to teams, teams to organizations; permissions cut across all of them. That's what SQL, joins and transactions are good at.
- The filing cabinet had to grow about a hundredfold. Figma wrote in 2024 that its database stack "has grown almost 100x since 2020", and it did that without closing the office: no maintenance windows.
Figma kept these two jobs in two different engines, and never let the second one sit on the first one's hot path.
Synthesizing vector architecture diagram...
Read it as two tracks. The multiplayer engine changed its implementation (TypeScript, then Rust) and its durability (checkpoints, then a journal). The metadata database changed its shape three times: one instance, then a dozen by table group, then tables split across many.
A few words we'll use all page:
| Word | What it means on this page |
|---|---|
| Multiplayer server | The process that holds one open Figma file in memory, receives every editor's changes over WebSockets, orders them and broadcasts them. |
| Checkpoint | A full copy of a file's current state, encoded, compressed and written to S3. |
| Journal | A durable log of the small changes made since the last checkpoint (Figma described one in 2022). |
| OT / CRDT | Two families of algorithms for merging concurrent edits. Operational transformation (OT) rewrites each edit against the edits it raced with; a conflict-free replicated data type (CRDT) is a data structure whose copies always converge, whatever order updates arrive in. |
| PgBouncer | A lightweight connection pooler: thousands of client connections share a few hundred real Postgres connections. |
| Read replica | A copy of the database that follows the primary's changes, a little behind, and serves reads. |
| Vertical partitioning | Moving whole tables, grouped by what they're used for, onto their own database. |
| Horizontal sharding | Splitting the rows of one table (or group of tables) across many databases by a key. |
| Colo | Figma's word for a group of tables that share one shard key, so related rows live together. |
| Logical vs physical shard | A logical shard is a slice of the key space the application routes to; a physical shard is the database server that holds it. Many logical shards can sit on one physical shard. |
| DBProxy | Figma's Go service between the application and PgBouncer that parses SQL and routes it to the right shard. |
| LSN | Log sequence number: a position in Postgres's write-ahead log (WAL). |
| XID wraparound | Postgres transaction IDs are 32-bit; if old rows aren't "frozen" by vacuum in time, the database must stop accepting writes. |
What Makes It Hard
- Two workloads with nothing in common. Editing is a stream of tiny updates per file, latency-critical and fine in memory. Metadata is relational, needs transactions, and must be durable before the user sees "saved".
- Migrations under live traffic. Every scaling step moves live data between databases while thousands of application servers keep querying it.
- Keeping the product's queries working. Once data is split, joins, foreign keys and transactions across the split stop working, and the application was written assuming they do.
The Question the Whole Loop Answers
How do we keep real-time editing instant and scale a single relational database far beyond one machine, step by step, without downtime?
The answer grows every round:
- Round 1: keep the database off the keystroke path. A multiplayer server per file holds it in memory and orders edits; Postgres holds metadata, helped by a bigger machine, read replicas and PgBouncer.
- Round 2: when one machine isn't enough, move groups of tables to their own databases, using logical replication, a pause of seconds and a way back.
- Round 3: when single tables outgrow a database, shard them: colos, logical sharding through views before any data moves, a query-routing proxy, and physical splits that fence the old copy for good.
Round 1 · Mid-level · "Era 1: Multiplayer Servers and One Big Postgres"
~35 min · SDE II (L5) · about 2016–2020 · one RDS Postgres instance (published) · 60,000 queries/s and ~10,000 row writes/s at the peak (planning figures) · edits feel instant and the database keeps up
R1.1 Establish Design Scope
The interviewer sets the scene: "We're building a design tool that runs in the browser. Several people can have the same file open and edit it together, live. We also have the usual product data: users, teams, projects, files, permissions, comments. Design the backend."
| We ask | Interviewer answers | What it changes in the design |
|---|---|---|
| What does editing involve? | Several people change the same file at once: drag shapes, change colors, type. Figma (2022): clients "send updates every 33ms (30 FPS)" while editing. | The edit path is a continuous stream per file, not occasional requests. It can't be a database transaction per update (step 1.1). |
| What lives in the database? | Figma (2019): "comments, users, teams, projects, etc." are "stored in Postgres, not our multiplayer system". Figma (2023): "metadata—like permissions, file information, and comments". | Two data sets with different rules. The file's contents live with the multiplayer server; metadata lives in Postgres. |
| How fast must edits be? | Your own edit shows immediately; others see it about one network round trip later. | Clients apply their own changes locally before the server answers (step 1.2). The server must never block a file's stream on disk or database I/O. |
| What's the database? | One PostgreSQL instance on Amazon RDS, on one of the largest machines AWS offers. | One primary takes every write; its size sets our ceiling (step 1.3). |
| Where are the users? | Figma (2019): "More than 80% of our weekly active users are outside the US", and the servers sit in "our datacenter in Oregon". | A round trip from Europe or Asia to Oregon is often over 100 ms. That's why local-first application of edits matters. |
| What about offline? | Figma (2019): you can "go offline for an arbitrary amount of time and continue editing". On return, "the client downloads a fresh copy of the document, reapplies any offline edits on top of this latest state". | Reconnecting is simple: fetch a fresh copy, replay local changes. No complex resync protocol. |
Out of scope for this round: splitting the database, cross-region failover, plugins, rendering.
R1.2 Functional Requirements, Derived Step by Step
| Phrase from the problem | Operation |
|---|---|
| "Open a file" | Check access, download the file, then open a WebSocket to the file's multiplayer server |
| "Edit together" | Send property changes, object creates and deletes; receive everyone else's, in one order |
| "Don't lose my work" | The server saves the file's state durably, on a schedule |
| "Manage files and teams" | POST /v1/files, rename, move, share: ordinary requests against Postgres |
| "Comment" | Create and list comments on a file (Postgres) |
Not yet: partitioning the database, sharding, a journal of every change (Figma described one in 2022; we meet it in R1.9).
R1.3 Non-Functional Requirements: the Questions
We name each quality first; the numbers come in R1.7.
- Edit latency. Your own change: immediate (applied locally). Everyone else's: one round trip plus server processing. A slow operation on one file must not stall other files.
- Convergence. When edits stop, every client must show exactly the same file. Figma (2019): "we cannot allow two clients editing the same Figma document to diverge and never converge again."
- Edit durability. How much work can a server crash lose? In this era: up to one checkpoint interval.
- Metadata consistency. Permissions, ownership and billing need transactions. Some reads must see the user's own write immediately (read-your-writes).
- Database headroom. The one primary must stay well below saturation. Figma (2023): "If our database became completely saturated, Figma would stop working."
R1.4 The API
Figma hasn't published its multiplayer wire format. The messages below are illustrative: their fields come from mechanisms Figma did publish (property-level changes, client-generated object IDs, one parent-and-position property, server ordering and rejection), and the names are ours.
Client → server: changes (sent at most every 33 ms while the user edits)
| Field | Type | Meaning |
|---|---|---|
file_key | string | Which file |
client_id | integer | This session's ID, given by the server at connect |
batch_id | integer | Increments per batch from this client, so the server's acknowledgement can name it |
changes[] | list | Each is {object_id, property, value} |
creates[] | list | New objects: {object_id, properties}; object_id embeds client_id, so no two clients can pick the same ID |
deletes[] | list | Object IDs to remove |
Server → client: ack, broadcast, reject
| Message | Fields | Meaning |
|---|---|---|
ack | batch_id | Your batch is applied, in the server's order |
broadcast | from_client, changes[], creates[], deletes[] | Someone else's changes, in the server's order |
reject | batch_id, object_id, reason | A change the server refused, for example a parent change that would create a cycle |
The parent property. An object's place in the tree is one property, parent, whose value is {parent_id, position}. The position is a fractional index: a number between 0 and 1 that sorts siblings. Figma (2019): "the parent link and this position must both be stored as a single property so they update atomically."
Metadata: create a file (an illustrative internal endpoint; Figma's public REST API is not this)
httpPOST /v1/files HTTP/1.1 Authorization: Bearer <session token> Idempotency-Key: 5d2f8c1e-9a4b-4e7f-b1c3-2a6d9e0f7b18 Content-Type: application/json { "team_id": "8812", "project_id": "40251", "name": "Checkout redesign" }
httpHTTP/1.1 201 Created Content-Type: application/json { "file_key": "fk_7Qm2xR", "name": "Checkout redesign", "project_id": "40251", "created_at": "2020-03-02T17:04:11Z" }
Retries are safe because the guard comes first. Creating a file also writes an empty first checkpoint to S3, an external call. The server claims the idempotency key before that call, in the same transaction that creates the file row, with an insert that does nothing if the key already exists:
sqlBEGIN; INSERT INTO idempotency_keys (key, user_id, request_hash, file_key) VALUES ($1, $2, $6, $3) ON CONFLICT (key) DO NOTHING; -- 0 rows inserted means: a retry -- only if 1 row was inserted: INSERT INTO files (file_key, project_id, name, created_by) VALUES ($3, $4, $5, $2); COMMIT; -- then write the empty checkpoint to S3 under a key derived from file_key
On a retry, the insert does nothing, the server reads the stored row, checks that request_hash matches this request's body (a mismatch is a 409), and returns the same file. If the first attempt died after the commit but before S3, the retry finds the file without a checkpoint and writes it; the S3 key is derived from file_key, so writing it twice is harmless.
Status codes
| Code | Meaning |
|---|---|
201 Created | File created (or the same file returned for a retried key) |
400 Bad Request | Missing name or project |
403 Forbidden | No edit access to the project |
409 Conflict | The same idempotency key was reused with a different body |
503 Service Unavailable | The database is failing over; retry with backoff and the same key |
Recap
- Editing is a stream of small property changes over a WebSocket, ordered by the server.
- Metadata is ordinary HTTP against Postgres; retries are safe because the key is claimed before any side effect.
R1.5 Design Evolution: From Rows per Keystroke to Two Engines
Each step is a problem, your turn to think, the answer, and what it costs us.
Step 1.0: The Baseline
Every edit is a database write. A drag updates a row in a nodes table; other clients poll or get notified and re-read.
Synthesizing vector architecture diagram...
Every keystroke becomes a transaction on the one machine that also serves permissions, comments and billing. The database sits on the keystroke path.
Step 1.1: Edits Must Be Instant for Everyone
The problem: a busy file has several people dragging and typing, each sending a change every 33 ms. Everyone must see everyone else's changes within about one round trip, and a slow file must not slow down other files. What would you do?
Primitive: WebSocket, SSE and Long Polling · Drill: The Gateway Restart That DDOSed the Chat Fleet (answered here: the deploy reconnect storm and its jittered backoff in "What it costs us", and WebSockets vs SSE in the wrong answers) · Primitive: Event Sourcing and CQRS (a snapshot plus later changes, the shape the journal takes in R1.9)
Step 1.2: Two People Changed the Same Property
The problem: Alice sets a rectangle's fill to red while Bob, a continent away, sets it to green. Both apply their change locally first. The two messages arrive at the server in some order. Everyone must end with the same color, and nobody's screen should flicker between the two. What would you do?
Step 1.3: The Metadata Database Keeps Growing
The problem: the multiplayer tier scales by adding servers, since each file lives on one of them. The metadata database doesn't: it's one Postgres primary. In 2020 it peaked at over 65% CPU (Figma, 2023), traffic triples every year, and connections number in the thousands. What would you do?
Loop primitive: Replication, Quorums & Read-Your-Writes (Part 3: replication lag and stale reads; Part 4: read-your-writes)
Round 1 Step Summary
| Step | Problem | Component | What it costs us |
|---|---|---|---|
| 1.0 | (baseline) | Every edit a Postgres transaction | The database on the keystroke path |
| 1.1 | Edits must be instant | A multiplayer process per file, in memory, WebSockets, checkpoints to S3 | Stateful servers, routing, deploy storms, up to 60 s at risk |
| 1.2 | Same property, two editors | Server-ordered last-writer-wins per property | Concurrent text edits: one wins |
| 1.3 | Metadata keeps growing | The largest instance, read replicas, new databases, PgBouncer | Buys about a year, not a solution |
R1.6 Architecture v1
Synthesizing vector architecture diagram...
Two paths leave the browser. Edits go to the multiplayer server that owns the file and never touch Postgres; only checkpoints leave it, to S3. Everything else goes through the web tier and PgBouncer to one primary, with replicas taking lag-tolerant reads.
The web backend is Ruby; Figma (2023): "We use Ruby for the application backend, which services the majority of our web requests." How multiplayer checks a user's access to a file isn't published; in our design it asks the web tier once, when the WebSocket opens.
Trace: open a file and drag a shape
Synthesizing vector architecture diagram...
Postgres is touched once, at open, for a permission check. The drag itself goes client, server, other clients.
Trace: rename the file, then reload
PATCH /v1/files/fk_7Qm2xRgoes to the web tier, then through PgBouncer to the primary:UPDATE files SET name = ..., commit.- The web tier notes "this user wrote at time T" in the session (our design).
- The user reloads within a few seconds: the file list is read from the primary, because this session wrote recently. A read replica lagging by 500 ms would still show the old name.
Sources for this round
- Wallace, Rust in production at Figma, Figma blog, May 2, 2018: the TypeScript server launched two years earlier; each document on exactly one worker, slow operations locking a worker, the heavy-worker pool; a Rust child process per document; serialization over 10× faster; network handling kept in Node.js.
- Wallace, How Figma's multiplayer technology works, Figma blog, October 16, 2019: client/server over WebSockets, a process per document, why not OT, "inspired by" CRDTs but not true CRDTs, property-level last-writer-wins ordered by the server, flicker avoidance, object creation and IDs, the parent property and cycle rejection, fractional indexing, offline editing, metadata in Postgres.
- Goel, Under the hood of Figma's infrastructure, Figma blog, November 21, 2019: a single database instance on one of the largest machines in AWS; more than 80% of weekly active users outside the US; the Oregon datacenter; one Multiplayer instance per file.
- Liang, The growing pains of database architecture, Figma blog, April 4, 2023: the 2020 state (one RDS database, over 65% CPU at peak, traffic about 3× a year), the four tactical fixes, replication-lag sensitivity, Ruby and ActiveRecord.
- Tsung, Making multiplayer more reliable, Figma blog, October 20, 2022: multiplayer is authoritative and in memory; checkpoints every 30 to 60 seconds to S3; clients send updates every 33 ms; up to 60 seconds at risk.
Everything else in this round (the wire format, the routing table, the idempotency table, the replica routing rule, the trace details) is our design.
R1.7 Numbers
Figures marked published come from the sources above; the rest are assumptions or derived from them.
One busy file (assumptions: 10 people have it open, 3 edit at the same moment, 150 bytes per batch; the 33 ms cadence is published)
| Quantity | Arithmetic | Result |
|---|---|---|
| Batches per active editor | 1,000 ms ÷ 33 ms | ~30/s |
| Batches into the server | 3 editors × 30 | 90/s |
| Broadcasts out | 90 × 9 other clients | 810 messages/s |
| Bytes out | 810 × 150 B | ~122 KB/s, about 1 Mbit/s |
Tiny for one process. The point is the shape: a steady stream per file, which a process that owns the file absorbs easily and a shared database would pay for as 90 commits a second, per file.
Edits vs the whole database (published figures, two different years)
| Quantity | Arithmetic | Result |
|---|---|---|
| Changes the journal received (2022) | 2.2 billion ÷ 86,400 s | ~25,500/s on average |
| The whole database's write stream (2021) | Published: "on the order of 10,000 writes/second" | ~10,000/s |
| The same, a year later at 3× | 10,000 × 3 (derived) | ~30,000/s |
Even at the daily average, edits are 85% to 255% of everything the metadata database writes (25,500 ÷ 30,000 and 25,500 ÷ 10,000). Putting them in Postgres would roughly double its write load, or more.
Checkpoints: they scale with file size, not with edits (assumptions: a 5 MB compressed file; 45 s between checkpoints, the middle of the published 30–60 s)
| Quantity | Arithmetic | Result |
|---|---|---|
| Checkpoints per hour of editing | 3,600 s ÷ 45 s | 80 |
| Checkpoint bytes per hour | 80 × 5 MB | 400 MB |
| Edit bytes per hour (the busy file above) | 90 × 150 B = 13.5 KB/s × 3,600 s | ~49 MB |
| The same checkpoints for a 50 MB file | 80 × 50 MB | 4,000 MB, about 80× the edits |
| S3 requests per hour | 80 PUTs × $0.005 per 1,000 | $0.0004, negligible |
This is the imbalance Figma described in 2022: "The contents of an entire file are often orders of magnitude larger than the incremental updates to the file."
A deploy (assumptions: 50,000 files open fleet-wide, 5 MB average, a 10-minute rolling deploy). Each file must checkpoint before it closes: 50,000 × 5 MB = 250 GB in 600 s, about 417 MB/s to S3, set by how much is open rather than by how much is being edited. Figma (2022): when multiplayer is re-deployed, "all files that are held in memory are closed, which causes a surge in checkpoints and subsequently increased load on the database."
One slow file on a shared worker (assumptions: 100 open files per worker in the TypeScript era; a 3-second encode of one huge file). Every change that arrives for any of those 100 files during the encode waits for it to finish. A change arriving at a random moment waits anywhere from 0 to 3 s, uniformly: 1.5 s on average, up to 3 s. With encoding 10× faster (published), the pause is 0.3 s; with a process per document, the other 99 files don't wait at all.
Database CPU runway (published: over 65% CPU at peak in 2020; 48 → 96 vCPUs; ~3× a year. Assumption: CPU load scales linearly with vCPUs)
| Quantity | Arithmetic | Result |
|---|---|---|
| Load after the upgrade | 65% × 48 ÷ 96 | 32.5% |
| Months until 65% again | years | ~7.6 months |
| Months until 80% (our ceiling) | years | ~9.8 months |
The upgrade alone buys under ten months. With replicas taking some reads and new features going to new databases, Figma's "additional year of runway" is consistent.
Synthesizing vector architecture diagram...
From 32.5%, load passes 65% around month 8 and approaches 100% by month 12. (Month 3: 32.5 × 3^0.25 = 42.8; month 6: × 3^0.5 = 56.3; month 9: × 3^0.75 = 74.1; month 12: × 3 = 97.5.)
PgBouncer (published: client connections "in the thousands". Assumptions: 5,000 client connections; 60,000 queries/s at the peak; 1.5 ms average time in the database; ~10 MB per Postgres backend process)
| Quantity | Arithmetic | Result |
|---|---|---|
| Queries running at any instant (Little's law) | 60,000/s × 0.0015 s | 90 |
| Server connections we give PgBouncer | 90 × ~2 headroom | ~200 |
| Backend memory without a pooler | 5,000 × 10 MB | ~50 GB |
| Backend memory with it | 200 × 10 MB | ~2 GB |
Postgres runs one operating-system process per connection. Thousands of mostly idle connections cost memory and scheduling for work that 90 busy ones could do.
A database failover (assumptions: Multi-AZ, a 99.99% monthly availability target). AWS documents Multi-AZ failover times as "typically 60–120 seconds". The monthly budget is 30 × 86,400 s × 0.0001 = 259.2 s, so one failover spends 23% to 46% of it. Figma hasn't published its RDS configuration; the point is that one database is one blast radius.
R1.8 Trade-Offs
How to merge concurrent edits
| Operational transformation | A true CRDT | Server-ordered last-writer-wins per property (Figma) | |
|---|---|---|---|
| Needs a central server | Usually | No | Yes |
| Handles concurrent text in one string | Well | Well (sequence CRDTs) | One edit wins |
| Complexity | High: every pair of operation types needs a transform | Medium: extra metadata per value and per deleted item | Low |
| Offline edits | Transformed against what they missed | Merge naturally | Re-sent on reconnect; the server orders them |
| Fits a design tree | Awkward | Yes, with care for moves | Yes: parent is a property; the server rejects cycles |
Figma's reason (2019): "Our primary goal when designing our multiplayer system was for it to be no more complex than necessary to get the job done." With a central server available anyway, the CRDT's decentralization is overhead.
Where a file's live state lives
| In the multiplayer process (chosen) | In a database | |
|---|---|---|
| Latency per change | Memory speed | A transaction per change |
| Who orders changes | One owner | Row locks, or an arbiter anyway |
| Crash | Lose back to the last checkpoint (Round 1), or to the journal (2022) | Nothing lost |
| Scaling | Add servers; files spread out | One machine takes every file's writes |
| Deploys | Every open file closes and checkpoints | Nothing special |
Checkpoint interval. 30 s risks at most 30 s of work and costs twice the checkpoint bytes of 60 s. Figma's published range is 30 to 60 seconds; the journal (R1.9) later made the interval a cost knob instead of a durability knob.
R1.9 Failure Modes
| Trigger | What you'd see | How the design responds |
|---|---|---|
| A multiplayer server crashes | Every file it owned closes; clients disconnect | Clients reconnect to a new owner, which loads the last checkpoint. Changes the server had acknowledged after that checkpoint are gone: Figma (2022): "we could lose up to 60 seconds of work on the server-side if multiplayer crashes." A client's own unacknowledged changes survive: it re-sends them on top of the fresh copy, as it does after going offline. |
| A huge file hogs a worker | Other files on the same worker stall (TypeScript era) | Published: first a separate pool of "heavy" workers, filled by hand; then a Rust process per document. |
| A deploy | Every open file checkpoints at once; every client reconnects at once | Roll the deploy over minutes, jitter reconnects, bound the reconnect rate per server. |
| The database primary fails | Metadata requests fail for 60–120 s (Multi-AZ, AWS's typical range); open files keep editing | The multiplayer path doesn't need Postgres while a file is open, so editing continues. New file opens (the permission check) wait for the failover. |
| A replica falls behind | Stale lists | Lag-sensitive reads already go to the primary; alarm on lag and move more reads back if it grows. |
| Two people reparent at once | One puts A under B while the other puts B under A: together, a cycle | The server applies the first and rejects the second. The client whose change will be rejected may briefly show both objects removed from the tree; they reappear where they belong when the rejection arrives (published). |
Go deeper: the journal Figma added (published 2022), and how it fences a stale owner.
The 60-second loss window is the weak spot of Round 1. Figma's fix: "a write-ahead log", which it calls the journal. Each change gets a per-file sequence number, "an incrementing integer associated with the file"; every checkpoint records the sequence number it includes. After a crash, the new owner loads the checkpoint and then "queries for all entries that are newer (i.e. have a higher sequence number) than the checkpoint". Results Figma reported: the journal "handles >2.2B received changes per day, persists 95% of changes within ~600ms"; the goal was "<1s of data loss". Figma describes the journal as "asynchronously written to as multiplayer accepts incoming changes", so (our reading) an edit can be broadcast before its journal entry is durable; that's the sub-second window.
- Store: DynamoDB. Figma considered Postgres but "the volume of writes would require us to consider a horizontally-scalable database—and we’re not there quite yet with Postgres."
- Batching: entries cover a range,
start_sequence_numbertoend_sequence_number. Loading checkpoint 7 and replaying an entry that covers 5–9 still ends at the right state for 9. Changes 5 and 6 may briefly set a property back to an older value, but 7, 8 and 9 are applied after them, in order, and under last-writer-wins the last value applied is the one that stays. Figma (2022): "applying that entry will still result in the correct file at sequence number 9 due to the last-writer wins conflict resolution strategy we use." That's why replay must keep the original sequence numbers and order: renumbering, or applying entries out of order, would let an old value win. - One writer per file. Two servers that both think they own a file would write conflicting histories. Figma (2022): "Multiplayer writes a (lock UUID, file key) entry to take ownership of that file. When new entries are being written to the journal, the update is conditional on the lock UUID matching in the other table." And journal entries "are not read until after ownership is acquired, and the reads are strongly consistent."
Synthesizing vector architecture diagram...
The check that stops the stale owner happens in the store, on every write, and needs no clock. Ownership changes hands by a conditional write, so two contenders can't both win.
Figma doesn't say which DynamoDB call makes a journal write conditional on an item in another table; a transaction (TransactWriteItems) with a ConditionCheck on the lock item plus the Put of the entry is one way (our reading). Takeover conditioned on the previous owner's UUID is also our design. What the design deliberately avoids, and why it matters in DynamoDB:
- No time in the condition. DynamoDB conditions compare stored values; the service offers no server time to compare against. Ownership is an identity (the UUID), never "expires at".
- No TTL for freeing locks. DynamoDB TTL deletes expired items only eventually, typically within a few days, so it can't be what releases a lock.
- No global tables for the lock. Global tables replicate with last-writer-wins and check conditions only against the local copy; two Regions could each grant ownership. Figma also rejected global tables for the journal on cost ("increase the cost of the feature by 6x"). For recovery from a lost Region it relies on checkpoints: S3 is "already set up to be replicated cross-region", and all journal changes are checkpointed within 30 minutes.
Why not optimistic concurrency on the entries alone, say "append sequence 41 only if 41 doesn't exist"? That stops two writers from writing the same number, but a stale owner that's behind could still write numbers the new owner hasn't reached yet, and the history would interleave two different in-memory states. Checking ownership on every write rejects the stale owner's whole history in one comparison. A lock with a TTL and no check in the store fails the other way: an owner paused past the TTL wakes up and writes anyway.
Loop primitive: Write-Ahead Log, fsync & Group Commit (Part 1: what "acknowledged" has to mean; Part 5: group commit, the batching idea) · Loop primitive: Leases, Fencing Tokens & Distributed Locks (Part 4: the pause, the zombie and the fencing token; Part 5: conditional writes are the check) · Drill: The GC Pause That Corrupted Shared Storage (answered here: why a TTL lock doesn't stop a paused owner, and per-entry version checks vs an ownership check in the store)
R1.10 Pillar Check
| Pillar | What Round 1 covers |
|---|---|
| Reliability | A crash loses one file's recent work, not the fleet's; editing keeps running through a database failover; ownership is fenced in the store (the 2022 journal) REL 11; checkpoints in S3 are each file's backup, replicated across Regions (published, 2022) REL 9 |
| Performance Efficiency | Each workload in the store that fits it: live file state in memory, metadata in Postgres; a process per document so one slow file can't stall others PERF 1 · PERF 3 |
| Security | Access to a file is checked against Postgres before the WebSocket opens; edits travel over TLS (wss://) SEC 3 · SEC 9 |
| Cost Optimization | A pooler instead of thousands of backend processes; checkpoint requests to S3 cost fractions of a cent per file-hour COST 6 |
| Operational Excellence | Skipped this round: migrations and runbooks are Rounds 2 and 3. |
| Sustainability | Skipped this round: one database and a modest fleet; the footprint isn't yet the question. |
R1.11 Round 1 Rubric and Follow-Ups
What a strong mid-level (L5) answer shows
- Separates the two workloads early and keeps the database off the keystroke path, with numbers.
- Chooses WebSockets for a two-way stream, and a single owner per file that orders changes.
- Describes last-writer-wins per property with server order, and names what it can't do (concurrent text).
- Explains the durability trade-off of periodic checkpoints and how a crash loses up to one interval.
- Buys database time with the obvious levers (a bigger machine, replicas, a pooler) and quantifies how long they last.
Follow-up questions
-
"Why not make every client talk to every other client directly, peer to peer?" Answer: then there's no single authority to order conflicts, validate changes or reject a cycle, which is exactly what a true CRDT has to pay for. Figma chose the server as the authority because it simplifies everything else, and every file needs a server copy for durability anyway.
-
"A file becomes enormous. What happens?" Answer: checkpoints get slower and bigger (they scale with file size), and in the single-threaded era one huge file stalled its neighbours. The fixes: a process per document so it stalls only itself; faster serialization; later, a journal so durability no longer depends on writing the whole file often. Figma (2022) reported that "for the worst 5% of files, creating a checkpoint can take ~4 seconds or more".
-
"Can a user lose work?" Answer: their own unacknowledged changes, no: the client keeps them and re-sends after reconnecting. Changes the server acknowledged but hadn't checkpointed, yes, up to the checkpoint interval in this era. The journal cut that to under a second for 95% of changes.
Interview gotchas from this round's wrong answers
| Gotcha | Why it's wrong |
|---|---|
| "Store each edit in the database" | Edits alone would roughly double the busiest machine's write load, or more. |
| "Last writer wins by client time" | Laptop clocks disagree; the skewed clock always wins. |
| "Lock objects while editing" | A round trip before every drag; no offline editing. |
| "Send all reads to replicas" | Lag breaks read-your-writes right after a rename or a permission change. |
| "A bigger machine fixes it" | Doubling at 3× growth a year lasts about seven months. |
Round 2 · Senior · "Era 2: Vertical Partitioning Under Pressure"
~40 min · Senior SDE (L6) · 2020–2022 · one database → about a dozen (published) · traffic ~3× a year (published) · ~30,000 row writes/s across all databases (planning figure) · headroom without long outages
R2.0 Where We Left Off
This is what the candidate says aloud in the first 60 seconds of Round 2. If you're starting here, it's everything you need from Round 1.
Round 1 in 60 seconds. "Figma has two engines. Live editing runs on multiplayer servers: each open file is owned by one process that holds it in memory, receives every editor's changes over WebSockets, orders them, resolves conflicts per property with last-writer-wins in the server's order, and broadcasts. It checkpoints the whole file to S3 every 30 to 60 seconds, and since 2022 also writes a journal to DynamoDB, fenced by an ownership UUID. Postgres is never on the keystroke path. Everything else (users, teams, files, permissions, comments) lives in one RDS Postgres primary. We bought time with the largest instance, r5.24xlarge, read replicas for lag-tolerant reads, new databases for new features, and PgBouncer in front of thousands of connections. At 3× growth a year that buys about a year. Open costs: one primary takes every write, and everything shares its fate."
Architecture v1, compact
Synthesizing vector architecture diagram...
Round 1 in one picture: the edit path never touches Postgres; everything else shares one primary.
Round 1 step summary
| Step | Problem | Component | What it costs us |
|---|---|---|---|
| 1.1 | Edits must be instant | Multiplayer process per file, in memory | Stateful servers, routing |
| 1.2 | Same property, two editors | Server-ordered last-writer-wins per property | Concurrent text: one wins |
| 1.3 | Metadata keeps growing | Largest instance, replicas, PgBouncer | About a year of runway |
Open costs: a single write primary with no bigger machine to move to; writes are a large share of its load; every feature's queries compete for the same CPU.
R2.1 The Scope Raise
Interviewer: "It's 2020. The database is on the biggest instance RDS sells, peaking above 65% CPU, and traffic triples every year. We're also preparing to launch a second product. We can't take maintenance windows, and we can't ask every product team to rewrite their queries. Give us years of headroom, not months."
| We ask | Interviewer answers | What it changes in the design |
|---|---|---|
| Is there a bigger instance? | No. r5.24xlarge was "the largest instance available". | We have to use more than one database. |
| Can we shard horizontally now? | Figma evaluated it (2023): NoSQL or Vitess (MySQL) "would require a complex double read and write migration"; for Postgres-compatible NewSQL, "we would’ve had one of the largest single-cluster footprints for cloud-managed distributed Postgres"; self-hosting meant building skills the team didn't have. | We need something smaller and faster to deliver than sharding (step 2.1). |
| What's the load made of? | Many queries, and writes are "a significant portion of database utilization". Some reads can't go to replicas because of lag. | Replicas are exhausted as a lever; we must move write load off the primary. |
| How much downtime can a migration take? | Figma's goal (2023): "Limit potential availability impact to <1 minute". It also had to be automated and undoable. | Offline dump-and-restore is out; we need a live copy and a pause of seconds (step 2.3). |
| How is the application written? | Ruby with ActiveRecord; thousands of application backend instances query the database. | We can't statically find every query's tables, and we can't coordinate thousands of clients at a cut-over moment (steps 2.2, 2.3). |
| What else reads the database? | LiveGraph, Figma's real-time data layer, tails the Postgres replication stream to push updates to clients (published 2021). | Anything that consumes the WAL must keep working when tables move (R2.8). |
Scope change
| Round 1 | Round 2 | |
|---|---|---|
| Databases | 1 primary | About a dozen, one per table group (end of 2022, published) |
| Writes | ~10,000 row writes/s | ~30,000 row writes/s (one more year at 3×, derived) |
| Biggest single lever | A bigger machine | Moving tables |
| Migration downtime | Not our concern | Under a minute per move, automated, reversible |
| Cross-table features | Any join, any transaction | Only within a table group |
The "Not yet" list from R1.2 comes back: partitioning is in scope now; horizontal sharding is still not.
R2.2 What Breaks in the Round 1 Design
| Round 1 piece | What breaks at the new scale |
|---|---|
| One primary | No larger instance exists; at 3× a year, the 2020 peak of over 65% CPU becomes saturation within months (R1.7). |
| Read replicas | They only help reads that tolerate lag; writes all land on the primary. |
| Huge tables on one machine | Vacuum, index builds and backups on multi-terabyte tables take longer every month, and one table's maintenance slows every other feature. |
| Every feature in one blast radius | A runaway query from one feature raises latency for all of them; a failover takes everything down together. |
| Clients that assume one database | Joins and transactions across any tables are everywhere in the code, and nobody has a list of them. |
R2.3 New Requirements and API Additions
A table-group map. Every application instance must know which database holds each table. Figma (2024): "With vertical partitioning, we relied on a simple, hard-coded configuration file that mapped tables to their partition." An illustrative version (the table names are ours):
yaml# table-group map, version 14 (illustrative) version: 14 partitions: main: pgbouncer: pgbouncer-main.internal tables: [users, organizations, teams, projects, permissions] files: pgbouncer: pgbouncer-files.internal tables: [files, file_versions, file_thumbnails] comments: pgbouncer: pgbouncer-comments.internal tables: [comments, comment_reactions]
Later Figma replaced per-client knowledge with a service: "We’ve since introduced a new query routing service which will centralize and simplify routing logic as we scale to more partitions" (2023). That service is the ancestor of Round 3's DBProxy.
A migration runbook, as published (2023). "At a high level, we implemented the following operation (steps 3–6 complete within seconds for minimal downtime)":
| # | Published step | What it means in practice |
|---|---|---|
| 1 | "Prepare client applications to query from multiple database partitions" | Every query for the moving tables goes through that group's own connection (step 2.3) |
| 2 | "Replicate tables from original database to a new database until replication lag is near 0" | Logical replication of just those tables |
| 3 | "Pause activity on original database" | Pause at the PgBouncer layer; revoke privileges; cancel stragglers |
| 4 | "Wait for databases to synchronize" | Compare LSNs |
| 5 | "Reroute query traffic to the new database" | Point that group's PgBouncer at the new database |
| 6 | "Resume activity" | Release the pause |
Error contract during a move (our design). A query that reaches the old database for a moved table fails with Postgres's "permission denied" (SQLSTATE 42501), which the data layer turns into a retryable error that reloads the table-group map. A query that waits in a paused pooler longer than its timeout fails as a normal timeout, and is retried with jittered backoff.
R2.4 Design Evolution: One Database Becomes a Dozen
Step 2.1: One Database Is Near Its Ceiling
The problem: in 2020 the largest instance peaked at over 65% CPU (published) and traffic triples every year. Horizontal sharding would take a year or more to build and would touch most of the codebase. We need relief this year. What would you do?
Primitive: Database Sharding and Partition Keys (vertical vs horizontal partitioning)
Step 2.2: Which Tables Can Move Together?
The problem: there are hundreds of tables. Moving the wrong group either frees no CPU or breaks transactions all over the product. The application is Ruby with ActiveRecord, so you can't reliably tell from the source code which tables a request touches. What would you do?
Step 2.3: Move the Tables Without Downtime
The problem: a table group holds terabytes and takes writes every millisecond. Thousands of application instances hold connections to the old database. We must end with the tables on the new database, no write lost, no write applied twice, under a minute of impact, and a way back if something goes wrong. What would you do?
Primitive: Change Data Capture and the Outbox Pattern · Loop primitive: Change Streams & the Transactional Outbox (Part 1: why writing twice fails; Part 5: reading the database's own log, including slot limits; Part 7: republish when a slot is lost) · Loop primitive: Replication, Quorums & Read-Your-Writes (Part 8: catching up by log or by snapshot) · Loop primitive: Write-Ahead Log, fsync & Group Commit (Part 9: the WAL in B-tree engines, where LSNs come from)
The move as a state machine (our design around Figma's steps)
Synthesizing vector architecture diagram...
Until Serving, aborting is just un-pausing on the old database, since no application write ever reached the new one. After Serving, the only way back is a full move in the other direction, which fences the new copy first.
Step 2.4: Huge Tables and Vacuum
The problem: even after moves, some tables are several terabytes and take thousands of write transactions a second. Postgres must vacuum them: clean up old row versions and freeze old ones so their 32-bit transaction IDs can be reused. If freezing falls behind, Postgres eventually refuses writes. What would you do?
Primitive: Database Isolation Levels, ACID and Concurrency Anomalies (MVCC row versions, which vacuum cleans up)
Round 2 Step Summary
| Step | Problem | Component | What it costs us |
|---|---|---|---|
| 2.1 | One database near its ceiling | Vertical partitioning by table group | No joins or transactions across groups |
| 2.2 | Which tables move together | AAS sampling; runtime validators into Snowflake | Instrumentation; rewrites of crossing call sites |
| 2.3 | Move without downtime | Split PgBouncers first; logical replication without indexes; pause, revoke, drain, LSN, promote, reverse | Seconds of paused writes; WAL pinned during copy |
| 2.4 | Huge tables and vacuum | Per-table autovacuum, XID-age alarms | Operational care; a ceiling only row-splitting removes |
R2.5 Architecture v2
Synthesizing vector architecture diagram...
Each table group has its own pooler and primary. The application picks the pooler by table; only the poolers can reach the databases. LiveGraph now has to read several replication streams, not one.
The multiplayer tier is unchanged from Round 1.
Trace: moving the comments group (our timings; the steps are Figma's)
- Weeks before: the application sends every query for
commentsandcomment_reactionstopgbouncer-comments, which still points at the original database. Validators show no transactions that mixcommentswith other groups. - Create the new instance; create a publication for the two tables on the original, and a subscription on the new database with its secondary indexes dropped. Initial copy: hours. Rebuild the indexes. Lag falls to near zero.
PAUSEon the comments PgBouncers. New comment queries wait.REVOKEthe application role's access to the two tables on the original; after about 2 s, terminate the few sessions still inside a transaction; take and release the table lock; sample LSN3F/9A0012C8.- The new database reports it has replayed past
3F/9A0012C8within a second. - Stop the subscription, set each sequence above its copied maximum, start reverse replication from new to original without an initial copy.
- Point
pgbouncer-commentsat the new database andRESUME. The waiting queries run there.
Sources for this round
- Liang, The growing pains of database architecture, Figma blog, April 4, 2023: the 2020 limits and tactical fixes; options evaluated; vertical partitioning; AAS from
pg_stat_activityevery 10 ms; runtime validators and Snowflake; goals (under a minute, automated, undoable); the six steps; PgBouncer split first and security groups; logical replication's three advantages; dropping indexes during the copy; pause, revoke, cancel, LSN, promote, reverse replication; outcomes (2 tables first, 50 last in October 2022, ~30 s and ~2% per move, largest partition ~10% CPU); the new query routing service. - Steele, How Figma’s databases team lived to tell the scale, Figma blog, March 14, 2024: a dozen vertically partitioned databases by the end of 2022; the hard-coded table-to-partition configuration; the vacuum risk; avoiding double writes.
- Bandi, Keeping it 100(x) with real-time data at scale, Figma blog, May 17, 2024: LiveGraph's global-ordering assumption and the stopgap of combining streams.
- Chen and Kim, GraphQL, meet LiveGraph, Figma blog, October 14, 2021: LiveGraph tails the replication stream, about 10,000 writes/s.
- PostgreSQL documentation: Routine vacuuming, Vacuuming settings, Logical replication restrictions, Replication settings; PgBouncer usage (
PAUSE,RESUME), checked September 2026.
Figma didn't publish its table groups, its lock step, how it handled sequences, its slot limits, its autovacuum settings or its partition instance sizes; everything on this page about those is our design.
R2.6 Numbers and Cost
Figures marked published come from the sources above; everything else is an assumption or derived from one.
Moves and their impact (published)
| Value | |
|---|---|
| First move | "two high-traffic tables" |
| Last move | 50 tables, October 2022 |
| Impact per move | "a ~30 second period of partial availability impact (~2% of requests dropped)" |
| Databases by end of 2022 | "a dozen vertically partitioned databases" |
| Largest partition afterwards | "CPU utilization hovering ~10%" |
Copy time (assumptions: a 3 TB table group; 150 MB/s bulk copy without secondary indexes; index maintenance row by row makes the copy 10× slower; 3 h to rebuild indexes)
| Arithmetic | Time | |
|---|---|---|
| Copy without indexes | 3,000,000 MB ÷ 150 MB/s = 20,000 s | 5.6 h |
| Rebuild indexes | Assumption | 3 h |
| Total | 5.6 + 3 | ~8.6 h |
| Copy with indexes kept | 20,000 s × 10 = 200,000 s | ~2.3 days |
Consistent with Figma's published before-and-after: "days, if not weeks" became "a matter of hours".
WAL pinned by the replication slot (assumptions: the original database writes 20,000 rows/s at about 1 KB of WAL each)
A replication slot keeps every WAL segment the subscriber hasn't confirmed. For the whole copy, that's all of it:
| Arithmetic | Result | |
|---|---|---|
| WAL rate | 20,000 × 1 KB | 20 MB/s = 72 GB/h |
| WAL held during an 8.6 h copy | 72 × 8.6 | ~620 GB |
So the original database needs about 620 GB of free storage just for the move, and more if the copy stalls. Set max_slot_wal_keep_size (default −1, unlimited) to a cap the disk can afford, say 800 GB. Past the cap the slot is lost and the subscriber "may no longer be able to continue replication due to removal of required WAL files"; the plan for that case is to drop the subscription, empty the target tables and start the copy again (a republish), instead of letting a full disk stop the original database's writes.
The pause, step by step (our design; Figma: steps 3–6 "complete within seconds")
| # | Step | Time (assumption) |
|---|---|---|
| 1 | PAUSE the group's PgBouncers | ~0.1 s |
| 2 | REVOKE on the moved tables | ~0.05 s |
| 3 | Grace period, then terminate leftover transactions | 2 s |
| 4 | LOCK granted and released; sample LSN | ~0.05 s |
| 5 | New database replays past the LSN (lag near zero) | up to 1 s |
| 6 | Stop the subscription, advance sequences, start reverse replication | ~1 s |
| 7 | Repoint PgBouncer, RESUME | ~0.5 s |
| Total (every step waits for the one before) | ~4.7 s |
Each step depends on the previous one, so the times add. One nuance: PgBouncer's PAUSE returns only once in-flight transactions have released their server connections, so in practice it runs alongside the revoke, grace and terminate steps rather than before them (Figma: "While PgBouncer pauses new connections, we revoke clients’ query privileges"); the 0.1 s in the table is the time to stop new queries, and the ~4.7 s total still holds. Figma's observed impact was about 30 s; the rest plausibly comes from connections re-establishing, cold caches on the new database and clients backing off before retrying. That split is our guess; the post doesn't break the 30 s down.
What a request feels during a 4.7 s pause (assumption: a 3 s client timeout). A query arriving at a random moment in the pause waits for the rest of it: anywhere from 0 to 4.7 s, uniformly, 2.35 s on average. It times out if it waits more than 3 s, that is, if it arrives in the first 4.7 − 3 = 1.7 s: 36% of the queries that arrive during the pause (1.7 ÷ 4.7). The shorter the grace period, the fewer time out.
Error budget (assumption: a 99.99% monthly target). A 30-day month is 2,592,000 s; the budget is 259.2 s of full outage. One move costs about 30 s × 2% = 0.6 s of equivalent full outage. 259.2 ÷ 0.6 = 432 moves a month would fit. Figma wrote that it had "successfully performed the partitioning operation many times in production", over roughly two years (our inference from the post's dates): the moves are cheap; the risk is in getting one wrong, which is why they're automated and reversible.
Headroom (published: ~10% CPU on the largest partition; ~3× a year)
| Target | Arithmetic | Runway |
|---|---|---|
| Back to 65% | years | ~20 months |
| To 80% | years | ~23 months |
About two years, if every partition grows at the overall rate. Some tables grew faster (Round 3).
Cost: capacity bought at the same unit price (illustrative sizes; us-west-2 on-demand, Multi-AZ, 720 hours; Figma didn't publish its instance mix or whether it uses Multi-AZ)
| Instances | Hourly | Monthly | vCPUs | |
|---|---|---|---|---|
| Before | 1 × db.r5.24xlarge ($24.00/h Multi-AZ) | $24 | $17,280 | 96 |
| After, 12 partitions | 2 × db.r5.24xlarge ($24.00) + 4 × db.r5.12xlarge ($12.00) + 6 × db.r5.4xlarge ($4.00) | $48 + $48 + $24 = $120 | $86,400 | 192 + 192 + 96 = 480 |
Five times the vCPUs for five times the bill: $0.25 per vCPU-hour both ways. Vertical partitioning doesn't make capacity cheaper; it makes more of it possible, and lets small partitions run on small instances ("we’ve decreased the resources allocated to some of the lower traffic partitions", Figma 2023). Read replicas and storage come on top.
R2.7 Trade-Offs
Vertical partitioning vs horizontal sharding
| Vertical partitioning (chosen in 2020) | Horizontal sharding | |
|---|---|---|
| What moves | Whole tables | Rows of a table |
| Application changes | Only queries that cross groups | Every query must carry a shard key |
| Build time | Months | Figma: "roughly nine months to shard our first table" |
| Limit | One table can't be split | Near-unlimited, per shard key |
| Tooling reuse | The move tooling is reused for shard splits | Needs the move tooling plus a router |
Replication-based moves vs dual writes
| Logical replication, then a short pause (chosen) | Dual writes from the application | |
|---|---|---|
| Consistency | The target replays exactly what the source committed | Two independent writes can diverge |
| Application changes | None beyond routing | Every write path |
| Downtime | Seconds of paused writes | None in theory |
| Rollback | Reverse replication | Keep dual writes running; reconcile |
Logical vs streaming replication for a move
| Logical (chosen) | Streaming (physical) | |
|---|---|---|
| Copies | Chosen tables | The whole database |
| Target's major version | May differ | Must match |
| Reverse direction for rollback | Yes | No |
| Weak points | Slow initial copy with indexes; sequences and DDL not replicated; the slot pins WAL | Wastes storage on tables that stay behind |
R2.8 Failure Modes
| Trigger | What you'd see | How the design responds |
|---|---|---|
| The copy can't catch up | Replication lag stays high; slot WAL keeps growing | Don't pause until lag is near zero, since the pause lasts as long as the remaining lag. Find the cause (the target's IO, a long transaction on the target); alarm on the slot's retained WAL (OldestReplicationSlotLag, ReplicationSlotDiskUsage in CloudWatch). |
The slot passes max_slot_wal_keep_size | The subscription errors: required WAL was removed | Drop the subscription, empty the target tables, copy again. The original database kept its disk and kept serving. |
| Something is wrong after the switch | Errors or latency on the new database | Roll back with a reverse move: pause, revoke on the new database, drain, sample its LSN, wait until the original has replayed past it through reverse replication, grant back on the original, repoint, resume. The fence goes on the side being left before the other side opens. |
| A repair after a partial failure | Rows differ between the two copies | Re-copy the affected rows from the side that is currently authoritative, keeping their original values and version columns (updated_at, row versions). A repair that stamped rows with the current time would make old data look newer than real later edits. |
| An anti-wraparound vacuum is running on a moving table | The LOCK step waits; the pause grows | Postgres doesn't cancel an autovacuum that runs "to prevent wraparound" for a conflicting lock. Check pg_stat_activity before pausing; lock_timeout aborts the step, and the controller resumes on the old database and retries later. |
| Sequences left at their start values | Duplicate-key errors on the first inserts after the switch | The controller advances every sequence before resuming (step 2.3). |
| A schema change during a move | Replication stops: logical replication doesn't carry DDL | Freeze migrations on a table group while it moves. |
| A client with an old table-group map | It sends a query for a moved table to the old database | Permission denied: a retryable error that reloads the map. Safe however old the map is, because the refusal is in the database, not in a cache timeout. |
| LiveGraph assumed one ordered stream | Updates from several databases arrive in no global order | Figma's stopgap (published 2024): "artificially combining all replication streams into one". It worked but meant "every database shard participated in every user optimistic update", so one slow database stalled all of them. LiveGraph was later redesigned around invalidations (Round 3). |
R2.9 Production Gotchas
| Gotcha | Why it hurts | Fix |
|---|---|---|
| Putting the database back on the keystroke path | A new feature stores cursor positions or live selections in Postgres "just for now"; at 30 updates a second per editor it becomes the biggest writer | Anything that changes at interaction speed goes through multiplayer or another in-memory service; Postgres gets the durable summary. |
| Cross-database joins sneaking back | A new query joins files (now on its own database) with teams; it works in a one-database dev setup and fails in production | Run tests against one database per partition; keep the runtime validators on so a new crossing shows up before a move, not after. |
| Treating replicas as free capacity | Lag grows under load exactly when reads move there | Route only lag-tolerant reads; alarm on ReplicaLag. |
| Forgetting the slot | A copy that stalls for a weekend fills the original database's disk | Cap with max_slot_wal_keep_size; plan the re-copy. |
R2.10 Pillar Check
| Pillar | What Round 2 adds |
|---|---|
| Reliability | Moves that are automated, repeatable and reversible, with a permanent fence on the side being left REL 8; table groups on separate primaries, so one group's failure or runaway query doesn't take the others down REL 10 |
| Performance Efficiency | Table groups chosen by measured load (AAS) and measured coupling, not by size or guesswork PERF 5 · PERF 3 |
| Security | Only the poolers can reach the databases (security groups); the move's fence is a privilege revoke on the application role SEC 5 · SEC 3 |
| Cost Optimization | Partition instances sized to their own load; small partitions on small instances COST 6 |
| Operational Excellence | Moves as a scripted procedure with abort points; LSN checks instead of judgment; XID-age and slot alarms OPS 6 · OPS 8 |
| Sustainability | Skipped this round: capacity grows about as fast as load; Round 3 returns to it. |
R2.11 Round 2 Rubric and Follow-Ups
What a senior (L6) answer adds over L5
- Picks the cheaper step first, and says what it can't do (split one table).
- Chooses tables by measurement: load share and coupling.
- Designs a live move: split the connection layer first, logical replication with deferred indexes, a pause, an LSN check, a way back.
- Makes the old copy refuse writes permanently, in the database, and explains why a cache interval isn't a fence.
- Knows the logical-replication traps: slots pinning WAL, sequences, DDL.
- Does the XID arithmetic for the biggest tables.
Follow-up questions
-
"Why not restore an RDS snapshot to create the new database?" Answer: it's fast to create, but it brings the whole database's storage along, and Figma wanted "a much smaller storage footprint in the destination database"; and it must line up exactly with where replication starts. Logical replication of just the moving tables, with indexes built afterwards, copied in hours.
-
"Each move cost about 30 seconds of partial availability. How would you shrink it?" Answer: shorten the grace period (most queries are short; Figma cancelled "a little under 10"), have the new database's connections and caches warm before resuming, and make clients retry quickly with jitter instead of waiting out long timeouts. The pause itself (R2.6) is about 5 s in our design; the rest is recovery.
-
"Why revoke privileges instead of just repointing PgBouncer?" Answer: repointing controls the path we know about. The revoke stops every path, including a misconfigured client, a forgotten cron job or a pooler that didn't get the new config, and it stays in force until someone deliberately grants access back.
-
"Would Postgres table partitioning have avoided all this?" Answer: declarative partitioning splits a table into smaller tables on the same server. It helps vacuum and index maintenance (each partition is smaller) but not CPU, IO or connection limits of the one machine. It's a tool for step 2.4, not for step 2.1.
Interview gotchas from this round's wrong answers
| Gotcha | Why it's wrong |
|---|---|
| "Shard now" | Months of runway vs a year-plus project. |
| "Dual writes to migrate" | Two writes without a shared transaction diverge silently. |
| "Clients reload the config every 30 s, so the old database is safe" | A cache interval bounds how long a stale client misbehaves, not whether it can. |
| "Dump and restore" | Hours of downtime per move. |
| "Autovacuum defaults are fine" | Big tables take days to vacuum; the wraparound window is days. |
Round 3 · Architect · "Era 3: Horizontal Sharding Without Stopping"
~45 min · Principal (L7) · late 2022–2024 · first horizontally sharded table in September 2023 (published) · the database stack almost 100× its 2020 size (published) · one hot table: 6 TB, 30,000 row writes/s, 120,000 IOPS (assumptions) · near-zero-downtime shard splits that can be rolled back
R3.0 Where We Left Off
This is what the candidate says aloud in the first 60 seconds of Round 3. If you're starting here, it's everything you need from Rounds 1 and 2.
Rounds 1 and 2 in 60 seconds. "Figma keeps live editing out of Postgres: one multiplayer process per open file holds it in memory, orders everyone's changes, checkpoints to S3 and journals to DynamoDB under an ownership lock. Metadata lived on one RDS Postgres primary until, in 2020, it was on the largest instance, peaking at over 65% CPU, with traffic tripling yearly. We then split it by table group into about a dozen databases. Each move split PgBouncer first, copied the tables with logical replication while their indexes were dropped, paused the group for seconds, revoked the application's access on the old database so it can never be written again, waited for the new database to replay past the old one's LSN, and switched, with reverse replication kept for rollback. Each move cost about 30 seconds of partial availability. Open costs: one table can't be split this way, and some tables are several terabytes, with vacuum and IOPS limits coming within months."
Architecture v2, compact
Synthesizing vector architecture diagram...
Round 2 in one picture: many databases, but each table still lives whole on exactly one of them.
Round 2 step summary
| Step | Problem | Component | What it costs us |
|---|---|---|---|
| 2.1 | One database near its ceiling | Vertical partitioning | No cross-group joins or transactions |
| 2.2 | Which tables move together | AAS and runtime validators | Instrumentation, rewrites |
| 2.3 | Move without downtime | Split poolers; logical replication; pause, revoke, LSN, reverse | Seconds of pause per move |
| 2.4 | Vacuum on huge tables | Per-table autovacuum, XID alarms | A ceiling only row-splitting removes |
Open costs: the biggest tables can't be split by table; their databases are running out of IOPS; vacuum on them gets riskier every month.
R3.1 The Scope Raise
Interviewer: "It's late 2022. Some single tables are several terabytes, and traffic to them is growing ~3× a year. Their databases will run out of IOPS within months, and vacuuming them is already causing incidents. Split those tables across machines. You can't rewrite the product, you can't take downtime, and every step has to be reversible."
| We ask | Interviewer answers | What it changes in the design |
|---|---|---|
| How big are the biggest tables? | Figma (2024): "some of our tables, containing several terabytes and billions of rows, were becoming too large for a single database." | Moving the table elsewhere doesn't help; its rows must split (step 3.1). |
| What limit comes first? | Figma (2024): the highest-write tables were growing so quickly "that we would soon exceed the maximum IO operations per second (IOPS) supported by Amazon’s Relational Database Service (RDS)", and vacuums were hurting reliability. | Writes must spread over many databases (R3.6 does the runway math). |
| How long do we have? | Figma (2024): "we had only months of runway remaining." | No time to migrate to a whole new database product (R3.7). |
| Could we switch to a distributed database? | Figma evaluated "CockroachDB, TiDB, Spanner, and Vitess"; switching "would have required a complex data migration" and rebuilding years of RDS Postgres expertise. NoSQL couldn't express its "very complex relational data model". | We build sharding on top of the RDS Postgres we already run. |
| What must stay true for developers? | Figma's goals (2024) include "Minimize developer impact", "Scale out transparently", "Skip expensive backfills", "Make incremental progress", "Avoid one-way migrations" and near-zero downtime. | A routing layer that keeps most SQL working (step 3.3); a rollout that's reversible at every stage (steps 3.2 and 3.5). |
| Do we need transactions across shards? | Figma (2024): "we chose not to support atomic cross-shard transactions because we could work around cross-shard transaction failures." | The product must survive partial commits (step 3.4). |
Scope change
| Round 2 | Round 3 | |
|---|---|---|
| Unit of scaling | A table group | Rows of a table, by shard key |
| Largest table | Whole on one database | Split over many physical databases |
| Routing | A table-to-database map in each client | DBProxy: parses SQL and routes by shard key |
| Transactions | Any, within a group | Within one shard key in one colo; none across shards |
| A move | 1 database → 1 | 1 → N, with rollback |
| Writes we plan for | ~30,000 row writes/s total | ~90,000 row writes/s total; 30,000 on the hottest table (derived and assumed) |
R3.2 What Breaks in the Round 2 Design
| Round 2 piece | What breaks at the new scale |
|---|---|
| A table lives on one database | The hottest table alone will exceed one database's IOPS ceiling (its volume's and its instance's EBS limit), and its vacuums take days. |
| Queries assume one database | A query without a shard key can't be sent to one place; joins across tables with different shard keys can't run in one Postgres. |
| Foreign keys and unique indexes | Figma (2024): "Foreign keys and globally unique indexes can no longer be enforced by Postgres." |
| Transactions | Figma (2024): "It is now possible that writes to some databases will succeed while others fail." |
| Auto-increment IDs | Each shard's sequence counts on its own; two shards hand out the same ID. And logical replication doesn't carry sequence values at all. |
| Schema changes | Figma (2024): they "must be coordinated across all shards to ensure the databases stay in sync." |
| A hard-coded table-to-database file | Figma (2024): "Our topology would change dynamically during shard splits", so routing needs live metadata. |
R3.3 New Requirements and API Additions
A shard key per table, grouped into colos (published concept; the table names are ours). Figma (2024) "selected a handful of sharding keys like UserID, FileID, or OrgID. Almost every table at Figma could be sharded using one of these keys."
| Colo | Shard key | Tables (illustrative) |
|---|---|---|
| user colo | user_id | users, user_favorites, user_notification_settings |
| file colo | file_key | files, file_comments, file_versions |
| org colo | org_id | organizations, org_memberships, org_billing |
What SQL still works (published rules; the table layout is ours)
| Query shape | Supported? | How DBProxy runs it |
|---|---|---|
| Point or range query with the shard key | Yes | One shard; the whole query is pushed down to Postgres |
| Point or range query without the shard key | Yes | Scatter-gather: sent to every shard, results combined |
| Join of two tables in the same colo, on the shard key | Yes | Pushed down to one shard (or scattered, if the key is missing) |
| Join across colos, or not on the shard key | No | Rejected; rewrite the call site |
| A transaction touching one shard key in one colo | Yes, fully | One Postgres transaction |
| A transaction touching several shards | Not atomic | Each shard commits separately (step 3.4) |
| Unique index | Only if it includes the shard key | Figma: "currently unique indexes are only supported on indexes including the sharding key" |
Figma's summary (2024): "all range scan and point queries are allowed, but joins are only allowed when joining two tables in the same colo and the join is on the sharding key."
The topology DBProxy routes by (our model of what Figma describes)
Synthesizing vector architecture diagram...
A table belongs to a colo; a colo's key space is cut into logical shards by hash range; each logical shard maps to exactly one physical database at a given topology version.
Figma (2024) enforced invariants on this model, for example "every shard ID should be mapped to exactly one physical database", and built it to "deliver real-time updates in under a second". Its diagram shows the topology library backed by S3 and etcd. The logical-shard count, the hash function and the version field are our choices.
The fence on every physical database (our design, the same rule as the hotel and wallet loops):
sqlCREATE TABLE shard_fence ( logical_shard integer PRIMARY KEY, state text NOT NULL CHECK (state IN ('serving', 'moved')), topology_version bigint NOT NULL, moved_to text -- physical database now serving it ); -- local to each database: never included in a replication publication
Error contract (our design). A write that finds its logical shard moved rolls back and returns a retryable "shard moved" error carrying the new topology version; DBProxy reloads the topology and retries once. A query DBProxy doesn't support returns a non-retryable error naming the rule it broke, so the call site gets fixed rather than retried.
R3.4 Design Evolution: Split a Table Without Stopping
Step 3.1: Pick Shard Keys
The problem: hundreds of tables, and most queries follow relationships: a file's comments, a user's favorites, an organization's members. Whatever key we shard by, most queries must include it, and tables that are joined or updated together should land on the same shard. What would you do?
Go deeper: the organization that's bigger than a shard. Hashing spreads keys evenly, not load: one organization with a hundred thousand members is one org_id, so all its org-colo rows sit in one logical shard. Figma hasn't published how it handles this; a design that fits:
- Carve out the logical shard. Because the unit of placement is the logical shard, we can move the one containing the big organization to its own physical database, with the same split procedure as step 3.5. Its neighbours in that hash range move with it; that's the price of fixed ranges.
- If one organization alone saturates a database, placement can't help: every one of its rows is on one key. The fix is to move its heaviest tables into a colo with a finer key (file, not organization), which is a data-model change, not an operation.
- "One query over every organization" (a platform-wide report) should not be a scatter-gather through DBProxy: each scatter-gather "contributes the same amount of load as it would if the database was unsharded" (Figma, 2024). Run it on a copy of the data in a warehouse instead, minutes behind; Figma already uses Snowflake as its data warehouse (2023).
Primitive: Database Sharding and Partition Keys · Loop primitive: Sharding, Hot Keys & Rebalancing (in preparation) · Drill: One Customer, One Shard, One Outage (answered here: the tenant too big for its shard in "Go deeper", and why a cross-tenant report goes to the warehouse, not a scatter-gather)
Step 3.2: Sharding Is Risky and Hard to Undo
The problem: the first time we split a real table, bugs will surface: queries missing the shard key, joins that cross colos, transactions that span shards. If we find them after the data is physically split, rolling back means merging databases back together under live traffic. What would you do?
Step 3.3: The Application Sends Plain SQL
The problem: thousands of call sites build SQL through an ORM. Something must look at each query, find its shard key, decide which logical shards it touches, map those to physical databases, and send it, all fast, without every product team rewriting its data access. What would you do?
Primitive: API Gateway and Reverse Proxy (a proxy that inspects and routes each request)
Step 3.4: Cross-Shard Transactions and Foreign Keys
The problem: Figma's own example: "imagine moving a team between two organizations, only to find half their data was missing!" Rows keyed by different organizations live on different shards. Postgres can't hold one transaction across them, can't check a foreign key across them, and can't enforce a unique index across them. What would you do?
Primitive: Two-Phase Commit and Saga Orchestration · Primitive: Database Isolation Levels, ACID and Concurrency Anomalies · Loop primitive: Change Streams & the Transactional Outbox (Part 1: why writing twice fails; Part 2: the outbox in one transaction; Part 5: reading the database's own log) · Loop primitive: Idempotency & Effectively-Once Processing · Drill: The Dual-Write That Broke Search Consistency (answered here and in R3.7: the dual write in the wrong answers, and why tailing the log beats polling, as LiveGraph found) · Loop: Design a Distributed Unique ID Generator (worker numbers and clocks)
Step 3.5: The Physical Split
The problem: the table is logically sharded and working through views on one database. Now its rows must end up on four databases, with every write accepted by the old database also on the right new one, a pause of seconds, the ability to go back, and no way for a stale route to write the old copy. What would you do?
Primitive: Change Data Capture and the Outbox Pattern · Loop primitive: Leases, Fencing Tokens & Distributed Locks (Part 5: conditional writes are the check; Part 7: single writers) · Loop primitive: Sharding, Hot Keys & Rebalancing (in preparation) · Loop: Design a Hotel Reservation System (Round 2, step 2.4: a bucket move whose old cluster keeps a bucket_state row read FOR SHARE in every transaction and never removed) · Loop: Design a Digital Wallet System (Round 2: carving buckets out of a hot shard, with a permanent "moved" row on the old shard)
The life of one logical shard across a split
Synthesizing vector architecture diagram...
The only way back to the old database, once version v+1 is out, is a full reverse move; no timer or cache expiry ever re-opens it.
Round 3 Step Summary
| Step | Problem | Component | What it costs us |
|---|---|---|---|
| 3.1 | Pick shard keys | Colos keyed by user, file or org; hashed routing | Range scans scatter; hot keys; some tables unsharded |
| 3.2 | Risky, hard to undo | Logical sharding through views, one pooler per logical shard, flags | Nine months to the first table; view overhead |
| 3.3 | Plain SQL from the app | DBProxy: parser, logical and physical planners; 90% subset; scatter-gather | A hop; restrictions; scatter-gather load |
| 3.4 | Cross-shard transactions and keys | No atomic cross-shard transactions; ordered writes, operation records, outbox, sweeper; global IDs | Product constraints; background machinery |
| 3.5 | Physical split | Full logical replication 1 → N; fence rows and revoke; topology version as commit point; reverse replication | Full-copy storage; a fence check per write |
R3.5 Global Architecture
Synthesizing vector architecture diagram...
Every metadata query now passes through DBProxy, which reads the topology and picks poolers; the poolers reach either the old unsharded databases or the sharded ones. The multiplayer tier didn't change. LiveGraph's invalidators follow the same database topology.
Figma (2024) on LiveGraph after this change: "The invalidator is sharded the same way as the physical databases and tails a single replication stream. It is the only service with knowledge about database topology." Its caches are "sharded by query hash" and agnostic to the database layout.
Trace 1: a single-shard query
Synthesizing vector architecture diagram...
One shard, the whole query pushed down. Three components touched on the way: DBProxy, a pooler, a database.
Trace 2: a rejected query
- The application sends
SELECT ... FROM files f JOIN user_favorites u ON u.file_key = f.file_key WHERE u.user_id = 42. - The logical planner sees
filesin the file colo anduser_favoritesin the user colo: a join across colos. - DBProxy returns an error naming the rule ("join across colos"). Nothing reaches a database.
- The fix at the call site: read the user's favorites on the user shard (one shard, by
user_id), then fetch those files byfile_key: a batch of single-shard reads, or one scatter-gather if the batch is large.
Trace 3: the physical split. Shown in step 3.5: pause, fence, LSN, wait for all four, publish version v+1, resume.
Sources for this round
- Steele, How Figma’s databases team lived to tell the scale, Figma blog, March 14, 2024: almost 100× growth since 2020; the limits (table size, vacuum, RDS IOPS); goals; options evaluated; colos and keys; hashed routing; logical vs physical sharding; views, poolers per view, feature flags, the 10% overhead and shadow reads; DBProxy in Go, its planner, scatter-gather, shadow planning, the 90% subset and join rules; the topology service; full logical replication; 1 → N failover; September 2023 and ten seconds of partial availability; the future-work list; reassessing NewSQL.
- Liang, The growing pains of database architecture, Figma blog, April 4, 2023: the move machinery the split reuses.
- Bandi, Keeping it 100(x) with real-time data at scale, Figma blog, May 17, 2024: LiveGraph's invalidators sharded like the physical databases; dbproxy as the service that knows database topology.
- PostgreSQL documentation: CREATE VIEW (
WITH CHECK OPTION), Routine vacuuming (multixacts), Logical replication restrictions, checked September 2026. - AWS documentation: Amazon RDS DB instance storage (io2 Block Express up to 256,000 IOPS per volume), Quotas for Amazon RDS, Amazon RDS pricing (data transfer), and the AWS Price List for RDS and S3 in
us-west-2, checked September 2026.
Figma didn't publish its logical-shard count, hash function, physical shard counts, instance classes, how it fenced the old copy, how it cleans up the extra rows, or how its product handles partial commits; those parts of this round are our design.
R3.6 Numbers and Cost
Figures marked published come from the sources above; everything else is an assumption or derived from one.
Why now: the IOPS runway (assumptions: the hot table's database does 30,000 row writes/s at the peak at ~4 IOPS each, heap, indexes and WAL together; published: ~3× a year)
A database's IOPS ceiling is the lower of two limits: the volume's (up to 256,000 IOPS for io2 Block Express on RDS today) and the instance's EBS limit. The instance limit bites first on most classes. AWS's EC2 tables list 80,000 IOPS for r5.24xlarge (the 2020 instance), 260,000 for r5b.24xlarge and 86,667 for r5b.8xlarge. So our hot table's database has to be a db.r5b.24xlarge to reach 120,000 IOPS at all, and its ceiling is then min(256,000, 260,000) = 256,000.
| Quantity | Arithmetic | Result |
|---|---|---|
| Peak IOPS | 30,000 × 4 | 120,000 |
| Months until the ceiling | years | ~8.3 months |
"Only months of runway", as Figma wrote. (The IOPS ceiling in Figma's own 2022 setup isn't published; we use today's documented limits.)
Logical and physical shards (our choices: 64 logical shards; the first split goes 1 → 4 physical, each a db.r5b.8xlarge with an 86,667-IOPS instance limit)
| Quantity | Arithmetic | Result |
|---|---|---|
| Logical shards per physical database | 64 ÷ 4 | 16 |
| Live data per physical database | 6 TB ÷ 4 | 1.5 TB |
| Row writes per physical database | 30,000 ÷ 4 | 7,500/s |
| IOPS per physical database | 7,500 × 4 | 30,000 |
| Runway before the next split | years | ~11.6 months |
| Headroom with 64 physical databases | 64 × 86,667 = 5.55M IOPS, 46× today: | ~3.5 years of 3× growth |
The instance limit, not the volume, sets each shard's runway; a bigger class per shard (db.r5b.24xlarge) would lift it to 256,000 at about three times the instance price. After 64 physical databases, one per logical shard, splitting further would mean re-hashing into more logical shards. Pick the logical count for a decade, not for the first split; Figma hasn't published its count.
The price of full copies (us-west-2: io2 storage $0.25/GB-month in Multi-AZ)
| Arithmetic | Monthly | |
|---|---|---|
| Storage provisioned for the split | 4 × 6,000 GB = 24,000 GB × $0.25 | $6,000 |
| Live data after cleanup | 4 × 1,500 GB = 6,000 GB × $0.25 | $1,500 |
RDS doesn't shrink an instance's allocated storage in place, so each new database keeps its 6,000 GB. Deleting the other shards' rows frees space inside Postgres, and the shard grows into it: at 3× a year, 1.5 TB becomes 6 TB in years. Treat the full copy as prepaid headroom.
One table's databases, before and after the first split (illustrative sizes; us-west-2 on-demand, Multi-AZ, 720 hours)
| Before: 1 × db.r5b.24xlarge | After: 4 × db.r5b.8xlarge | |
|---|---|---|
| Instances | $28.416/h × 720 = $20,460 | 4 × $9.472/h × 720 = $27,279 |
| io2 storage | 6,000 GB × $0.25 = $1,500 | 24,000 GB × $0.25 = $6,000 |
| io2 IOPS ($0.20 per IOPS-month, Multi-AZ) | 150,000 provisioned = $30,000 | 4 × 40,000 = $32,000 |
| Total a month | $51,960 | $65,279 |
| vCPUs | 96 | 128 |
| IOPS ceiling (lower of volume and instance) | 256,000 | 4 × 86,667 = 346,668 |
About $13,320 a month more (+26%) for a 35% higher IOPS ceiling today (346,668 ÷ 256,000 = 1.35), and, more important, the ability to keep splitting up to 64 databases without touching the application. Most of the bill is provisioned IOPS, not instances.
DBProxy fleet (assumptions: 540,000 queries/s at the peak, 60,000 × 3², two years of 3× growth after Round 1; 20,000 queries/s per DBProxy instance; three AZs, and losing one must not overload the rest)
| Arithmetic | Result | |
|---|---|---|
| Load per AZ with one AZ lost | 540,000 ÷ 2 | 270,000/s |
| Instances per AZ | 270,000 ÷ 20,000 = 13.5, round up | 14 per AZ |
| Fleet | 14 × 3 | 42 |
Round up per AZ, not for the fleet: 13.5 × 3 = 40.5 rounds to 41, which can't be spread evenly and leaves one AZ short.
Latency: count every component a request touches (assumptions: DBProxy adds 0.2 ms of parsing and planning and one 0.1 ms same-AZ hop; each component's latency is independent)
- A single-shard query now touches three components (DBProxy, a pooler, a database) instead of two: about +0.3 ms on average.
- Tails compound. If each component is above its own p99 1% of the time, a request avoids all three tails with probability 0.99³ = 0.970, so 3.0% of requests see at least one component's p99, not 1%.
- Scatter-gather is worse. The answer waits for the slowest shard, so it's the max over all of them:
| Shards a scatter-gather waits for | Chance at least one is past its p99 |
|---|---|
| 4 | 1 − 0.99⁴ = 3.9% |
| 16 | 1 − 0.99¹⁶ = 14.9% |
| 64 | 1 − 0.99⁶⁴ = 47.4% |
Synthesizing vector architecture diagram...
At 64 shards, the p99 of one database becomes roughly the median of a scatter-gather. That's the latency side of Figma's load argument against scatter-gathers.
The split's pause (our chain; published: "ten seconds of partial availability on database primaries")
| # | Step | Time (assumption) |
|---|---|---|
| 1 | Pause 64 poolers, in parallel | ~0.1 s (the slowest of them) |
| 2 | Revoke on the moving shards' views | ~0.05 s |
| 3 | Grace period, terminate leftovers, fence rows under FOR UPDATE | ~2 s |
| 4 | Sample LSN; wait for all 4 targets, in parallel | up to 1 s (the slowest target) |
| 5 | Stop forward replication, give each target a disjoint sequence range, start reverse replication, 4 in parallel | ~1 s (the slowest target) |
| 6 | Publish topology v+1; DBProxy fleet picks it up | up to 1 s (published: "under a second") |
| 7 | Repoint and resume poolers | ~0.5 s |
| Total (steps in sequence; parallel work inside a step counts once, at its slowest) | ~5.7 s |
As in Round 2, PAUSE returns only once in-flight transactions have drained, so step 1 overlaps steps 2 and 3 in practice; the total doesn't change.
What a request feels (assumption: DBProxy holds a write for a moving shard for up to 2 s, then returns a retryable error). A write arriving at a random moment in a 10 s window waits the rest of the window: 0 to 10 s, uniformly, 5 s on average. It rides through only if it arrives in the last 2 s: 20% complete after a delay, 80% get the retryable error. With our own ~5.7 s chain instead of the published 10 s, 2 ÷ 5.7 = 35% ride through and 65% get the retryable error. Reads that tolerate lag keep going to replicas: Figma reported "no availability impact on replicas".
Cross-AZ transfer on the query path (assumptions: an average of 270,000 queries/s, half the peak; 2 KB per response, requests negligible; three AZs; the primary of each database sits in one AZ. Prices: EC2 to EC2 across AZs $0.01/GB each way, so $0.02 per GB crossing; EC2 to RDS across AZs $0.01/GB, charged on the EC2 side only; RDS Multi-AZ and read-replica replication is free)
| Hop | Crosses an AZ | Monthly |
|---|---|---|
| Bytes per month | 270,000 × 2 KB = 540 MB/s × 2,592,000 s | 1,399,680 GB |
| Application → DBProxy, AZ-unaware | ⅔ of the bytes × $0.02 | $18,662 |
| DBProxy → pooler, AZ-unaware | ⅔ × $0.02 | $18,662 |
| Pooler → primary | ⅔ (the primary is in one AZ) × $0.01 | $9,331 |
| All AZ-unaware | $46,656 | |
| Same-AZ routing on the first two hops | only the last hop crosses | $9,331 |
Keeping application → DBProxy → pooler inside one AZ saves about $37,300 a month. The last hop can't be avoided for two-thirds of traffic, because a primary lives in exactly one AZ.
Quotas: check them before the split, not during it. The default quota is 40 DB instances per Region (L-7B6409FD) and 100,000 GB of total storage across all RDS instances (L-7ADDB58A), both adjustable. Counting primaries and read replicas (assumptions: 12 partitions averaging ~3,000 GB, one read replica each): 24 instances and 72,000 GB before the split; the four new databases add 4 instances and 24,000 GB, reaching 28 instances and 96,000 GB. The next split, 4 → 8, would pass the storage quota. Request increases ahead of each split.
R3.7 Trade-Offs
Build a proxy on RDS Postgres, or adopt a sharded database?
| In-house sharding on RDS Postgres (Figma's choice) | Vitess | CockroachDB, TiDB, Spanner (NewSQL) | A Postgres sharding extension or Aurora Limitless Database | |
|---|---|---|---|---|
| Data migration | None to a new store; splits reuse the move tooling | To MySQL: a full double read and write migration | A full migration of the most critical data | Depends: a migration if it means leaving RDS for Postgres |
| SQL surface | A subset (90% of queries) | MySQL, broad | Broad, with distributed transactions | Broad Postgres |
| Cross-shard transactions | Not atomic (for now) | Best effort by default; two-phase commit as an option | Yes | Yes, with limits |
| Team expertise | Years of RDS Postgres | New | New | Partly new |
| Time to first value | ~9 months to the first table | Longer, given the migration | Longer, given the migration | — |
Figma's stated reasons (2024): switching stores meant "a complex data migration to ensure consistency and reliability across two different database stores", rebuilding "domain expertise from scratch", and "only months of runway". "We favored known low-risk solutions over potentially easier options with much higher uncertainty, where we had less control over the outcome." Figma's posts don't mention Postgres sharding extensions, so we can't say how it weighed them. Amazon now offers Aurora PostgreSQL Limitless Database (sharded Postgres on Aurora); it isn't part of Figma's story. Figma also said it would revisit the choice: "NewSQL stores have continued to evolve and mature. We will finally have bandwidth to reevaluate the tradeoffs".
Logical sharding through views vs going straight to physical
| Views first (chosen) | Physical first | |
|---|---|---|
| Finding application bugs | While rollback is a flag | While rollback is a data migration |
| Cost | View overhead up to 10%; months longer | Faster, if nothing goes wrong |
Full vs filtered logical replication
| Full copy to every new shard (chosen) | Filtered: each shard gets its rows | |
|---|---|---|
| Build effort | Reuses existing replication | Needs row filters by hash range and careful setup |
| Storage | N × the table until cleanup; RDS won't shrink it | ~1× |
| Safety | Ownership enforced by fences and views | Ownership built into the copy |
Polling vs tailing the log for change consumers. LiveGraph (2021) chose to "tail the database replication log" because polling would "multiply the load on the database" and forces each query to pick a polling interval; the same reasoning applies to step 3.4's outbox relay. A relay that polls the outbox table every 500 ms runs a query per shard every interval even when nothing changed, adds a delay of 0 to 500 ms (250 ms on average) to every event, and must mark or delete the rows it has sent, which is more write load and more dead rows for vacuum. A relay that tails the WAL (Debezium-style, through a logical replication slot) reads changes the database already wrote, in commit order, with no extra queries. The cost of tailing is what Round 2 found: the reader must follow the topology, and a slot pins WAL until it's read.
What changed from Round 1. Round 1 took the database off the keystroke path. Round 3 made the database itself a routed system, and it did so by keeping every property Round 2 relied on: moves by replication, a pause of seconds, a permanent refusal on the side being left, a way back. The lesson of the whole loop: grow in the order that keeps each step reversible, and put the safety of every move in the database that's being left, not in the clients that might still point at it.
R3.8 Failure Modes
| Trigger | What you'd see | How the design responds |
|---|---|---|
| A DBProxy parsing or planning bug | Wrong shard, or a query rejected that used to work | Before the physical split: flip the table's flag back to the unsharded path, within seconds (published). After it: the WITH CHECK OPTION views and fence rows stop a misrouted write from landing in the wrong range; roll DBProxy back; shadow planning on new query shapes before they ship. |
| A hot logical shard | One physical database at high CPU or IOPS while others idle | Split that physical database's logical shards onto more databases. If one key is the hot spot, placement can't help; move its tables to a finer-keyed colo (step 3.1). |
| One of four targets fails to catch up during a split | The LSN wait times out on one target | Abort before publishing v+1: un-fence the old database, resume there, investigate. No target ever served a write. |
| A problem after v+1 | Errors on the new shards | A reverse move: fence the new side, wait for the old database to replay past the new sides' LSNs through reverse replication, publish v+2 pointing back. Never "just flip the topology back". |
| A stale DBProxy | It routes a write to the old database | The fence row says moved: rollback, retryable error, topology reload. Safe however stale. |
| Orphans after a partial commit | A favorite pointing at a deleted file | Readers skip dangling references; the sweeper checks the owning shard and deletes the orphan only if the target is really gone. |
| Duplicate IDs | Unique violations after a split | Each target's sequence starts above the copied maximum on its own residue (step 3.5), so the four never overlap; new tables sharded only with globally unique IDs. |
| A schema change reaches some shards | Mixed schemas; logical replication stops if DDL runs mid-split | Additive, backward-compatible changes rolled out shard by shard, and none during a split. Figma lists "horizontally sharded schema updates" as future work. |
| Multixact age climbs on a shard | Aggressive vacuums start more often | The fence rows' shared locks are the source; watch mxid_age(datminmxid), keep transactions short, or switch to the advisory-lock variant under READ COMMITTED. |
| A LiveGraph invalidator misses a shard's stream | Some clients stop getting live updates for rows on that shard | The invalidators follow the topology like DBProxy does; after a split, a stream per new database. |
R3.9 Runbook and Incident Response
| Signal | Alarm | Severity | First action |
|---|---|---|---|
| CPU per physical shard | > 70% for 10 min | P2 | Which logical shards? Plan a split |
WriteIOPS vs provisioned IOPS | > 70% for 30 min | P2 | Same; check for a runaway writer |
Replication slot lag (OldestReplicationSlotLag) | Growing for 15 min, or above half of max_slot_wal_keep_size | P2 | Which consumer? Is a split's copy stuck? |
| DBProxy p99 and rejects | p99 > 2× baseline; any new reject rule firing | P2 | A new query shape from a deploy? |
| Share of scatter-gather queries | Rising week over week | P3 | Find the call sites missing a shard key |
XID age (MaximumUsedTransactionIDs) | > 500 million | P2 | Which table? Is vacuum running, throttled, blocked? |
| Multixact age | > 400 million | P3 | Long transactions holding fence locks? |
| Multiplayer journal persistence | p95 above ~600 ms (Figma's published figure) | P2 | DynamoDB throttling? Ownership conflicts? |
| Multiplayer reconnect rate | A spike outside deploys | P2 | A server crash, or a network event? |
A blocking query on a shard OPS 10
-
Find who blocks whom:
sqlSELECT pid, usename, state, now() - xact_start AS xact_age, pg_blocking_pids(pid) AS blocked_by, left(query, 80) AS query FROM pg_stat_activity WHERE state <> 'idle' ORDER BY xact_start; -
If the blocker is an application query,
SELECT pg_cancel_backend(<pid>);stops the statement. If it's idle inside a transaction, cancel does nothing:SELECT pg_terminate_backend(<pid>);ends the session and rolls its transaction back. -
If the blocker is an autovacuum "(to prevent wraparound)", don't kill it; it will restart and the XID age keeps rising. Wait, or move the conflicting work.
A shard's primary fails over REL 11
- RDS Multi-AZ promotes the standby; AWS documents failover as "typically 60–120 seconds". Only that shard's key range is affected; other shards and replicas keep serving.
- Confirm the new primary and AZ (command 1). The pooler reconnects by DNS name.
- Confirm that the replication slots used by LiveGraph or an outbox relay exist on the new primary. A Multi-AZ DB instance's standby is a synchronous copy of the storage, so its slots should survive with the volume; a read-replica promotion keeps them only with RDS for PostgreSQL 17+ failover slots. If a slot is missing, recreate it and republish from the stored source.
- Watch DBProxy errors for that shard fall back to zero.
Go deeper: CLI playbook
Commands an on-call engineer runs one at a time; replace names and times with real ones.
text# 1. Status, Multi-AZ and AZs of one shard's primary aws rds describe-db-instances --db-instance-identifier files-shard-03 --query "DBInstances[0].[DBInstanceStatus,MultiAZ,AvailabilityZone,SecondaryAvailabilityZone]" # 2. Highest transaction-ID age on that shard, per minute, for an hour aws cloudwatch get-metric-statistics --namespace AWS/RDS --metric-name MaximumUsedTransactionIDs --dimensions Name=DBInstanceIdentifier,Value=files-shard-03 --start-time 2026-09-28T09:00:00Z --end-time 2026-09-28T10:00:00Z --period 60 --statistics Maximum # 3. WAL held back by the most-lagging replication slot aws cloudwatch get-metric-statistics --namespace AWS/RDS --metric-name OldestReplicationSlotLag --dimensions Name=DBInstanceIdentifier,Value=files-shard-03 --start-time 2026-09-28T09:00:00Z --end-time 2026-09-28T10:00:00Z --period 60 --statistics Maximum # 4. A planned Multi-AZ failover of one shard (a drill, or before maintenance) aws rds reboot-db-instance --db-instance-identifier files-shard-03 --force-failover # 5. Quotas to check before a split: DB instances, and total storage in GB aws service-quotas get-service-quota --service-code rds --quota-code L-7B6409FD aws service-quotas get-service-quota --service-code rds --quota-code L-7ADDB58A
R3.10 Pillar Check
| Pillar | What Round 3 adds |
|---|---|
| Reliability | A table's rows spread over independent databases, so one failover affects one key range REL 10; splits rolled out as logical first, then physical, with an abort point and a reverse move REL 8; quotas checked before each split REL 1 |
| Performance Efficiency | Writes and IOPS spread by hashed shard key; queries pushed down to one shard; scatter-gathers measured and discouraged PERF 3 · PERF 1 |
| Security | The application role can reach only the views of shards a database serves; a move revokes access on the side being left SEC 3 |
| Cost Optimization | A 26% higher bill for a 35% higher IOPS ceiling now and room to split to 64 databases; same-AZ routing saves about $37,300 a month COST 8; building on the team's RDS expertise instead of a new store, a cost of effort Figma weighed explicitly COST 11 |
| Operational Excellence | Percentage rollouts behind flags, shadow planning and shadow reads before any risky step OPS 6; per-shard signals and runbooks OPS 8 · OPS 10 |
| Sustainability | Non-production environments keep production's logical topology on far fewer physical databases (published), and capacity added one split at a time instead of a large new fleet up front SUS 2; each shard's instance class sized to its own IOPS and CPU needs rather than to the whole table's SUS 5 |
Figma (2024) on that sustainability point: "in our non-production environments, we can keep the same logical topology as production, but serve the data from many fewer physical databases."
R3.11 Round 3 Rubric and Follow-Ups
What an architect (L7) answer adds over L6
- Picks shard keys from the data model (a few keys, colos) and explains hashing vs ranges with the ID formats in play.
- Separates logical from physical sharding, and uses it to find bugs while rollback is cheap.
- Designs the router: parse, plan logically, plan physically; a supported subset chosen from production traffic; the cost of scatter-gathers in load and in tail latency.
- Replaces cross-shard transactions with ordered writes, operation records, an outbox and a sweeper, and knows that IDs and unique indexes change too.
- Makes a 1 → N split safe: full copies, an abort point, a topology version as the commit point, and a permanent fence on the old copy checked under a lock.
- Prices it: provisioned IOPS, full-copy storage, cross-AZ hops, quotas.
Follow-up questions
-
"Why did Figma write DBProxy in Go?" Answer: the post says only that it's "a new golang service"; it doesn't give reasons. A fair guess, labeled as one: a network service handling many concurrent connections is Go's home ground, and LiveGraph's newer services are in Go too. Don't state a reason Figma didn't give.
-
"The Oregon Region is lost. What happens?" Answer: Figma hasn't published a multi-region failover design, so this is ours. Metadata: an RDS cross-Region replica is asynchronous, so the recovery point is its lag; promoting it needs capacity in the other Region (instances, storage and quotas already raised) and must respect any data-residency commitments. Files: Figma (2022) keeps checkpoints replicated across Regions and checkpoints journal changes within 30 minutes, so file edits could lose up to that window, plus however far S3's cross-Region replication is behind. Deciding the Region is gone must be done from outside it, and switching DNS should be a gated, human-approved step. Anything with a single writer (a file's multiplayer owner, the topology) needs a new epoch in the new Region: a new topology version and new ownership UUIDs. The epoch alone doesn't stop the old Region, though, because its databases and its DynamoDB lock table are separate copies that never saw the new epoch. So the returning Region must be fenced for good before anything in it can take traffic: keep it cut off from clients (DNS and load balancers stay pointed away, security groups closed to the application), and on its databases mark every logical shard
movedinshard_fenceand revoke the application role, then rebuild them as replicas of the new primaries rather than letting them rejoin as writers. Its multiplayer fleet stays stopped; files are re-homed only from the new Region's copies. A Region that comes back is a source of stale writers, never a candidate primary, until it has been rebuilt from the survivor. -
"A table has no good shard key. What do you do?" Answer: leave it unsharded on its own vertical partition as long as it fits; if it doesn't, add a key during a planned data-model change, accepting the backfill that Figma avoided for its first tables. Don't force it into a colo whose key it doesn't have.
-
"Isn't hashing the key bad for 'list my recent files'?" Answer: only if the list is a range over the shard key. "Recent files of user U" is a query in the user colo filtered by
user_idand sorted by time within one shard: one shard, pushed down. A range overfile_keyitself would scatter, which Figma found rare enough to accept.
Interview gotchas from this round's wrong answers
| Gotcha | Why it's wrong |
|---|---|
| "One shard key for everything" | Needs a new column on every table, backfills and refactoring, which Figma avoided. |
| "Shard by ID ranges" | Auto-increment and Snowflake IDs put all new rows on the newest shard. |
| "Split physically first" | Every application bug is found when rollback is a data migration. |
| "The topology updates in under a second, so the old copy is safe" | Propagation time bounds staleness; only a refusal in the old database makes it safe. |
| "Two-phase commit across shards" | Slowest-shard latency and a blocking coordinator; Figma built none. |
| "Clean up the extra rows right after the split" | With reverse replication on, the deletes replicate back to the old database. |
Loop Closer: Interview Strategy for All Three Rounds
How to Run Each 60-Minute Round
| Time | Round 1 | Round 2 | Round 3 |
|---|---|---|---|
| 0–5 min | Scoping: two workloads, 33 ms edits, one Postgres | Restate Round 1 in 60 seconds | Restate Rounds 1–2 in 60 seconds |
| 5–15 min | Requirements; the WebSocket messages; an idempotent POST /v1/files | Scope raise → what breaks | Scope raise → what breaks |
| 15–40 min | Steps 1.0–1.3: rows per keystroke → a process per file → last-writer-wins per property → a bigger machine, replicas, PgBouncer | Steps 2.1–2.4: vertical partitioning → choosing tables → the live move with a permanent fence → vacuum and XIDs | Steps 3.1–3.5: colos → logical sharding through views → DBProxy → partial commits and IDs → the 1 → N split |
| 40–50 min | Fan-out per file, edits vs the database, checkpoints, CPU runway, Little's law for the pooler | Copy time, slot WAL, the pause chain, error budget, headroom, cost per vCPU | IOPS runway, shard math, full-copy storage, cost, fleet per AZ, tail latency, transfer, quotas |
| 50–60 min | Failures (the journal and its fence) and pillar check | Failures, gotchas, pillar check | Failures, runbook, pillar check |
For how to spend a single 45-minute round, see the 45-minute interview blueprint. For another company that split a growing database while it kept running, see the Uber loop (Schemaless) and the Discord loop (a migration of trillions of rows). For bucket moves fenced on the old side, see the hotel and wallet loops.
The Two Sentences That Matter Most
- Opening any round: "Live editing and relational metadata are two different workloads, so I'll keep each open file in memory on one server that orders edits and checkpoints it, and keep Postgres off the keystroke path entirely."
- When scale arrives: "I'll grow the database in reversible steps: move table groups by logical replication with a pause of seconds, then shard big tables by colo, logical before physical, behind a query-routing proxy, and every move makes the old copy refuse writes permanently, in the database, not in a cache."
Well-Architected Review Sheet
Interviewers rarely ask "which pillar is this?". They ask the pillar's question in plain words. Rehearse one sentence per row.
| Pillar | Question you'll hear | One-sentence answer | Round | Backed by |
|---|---|---|---|---|
| Reliability | "A multiplayer server crashes. What's lost?" (REL 11) | Unacknowledged changes are re-sent by clients; acknowledged ones since the last checkpoint were at risk (up to 60 s), until the journal cut that to under a second for 95% of changes. | 1 | R1.9 |
| "How do you move live data between databases?" (REL 8) | Logical replication, a pause of seconds, revoke and drain on the old side, an LSN check, switch, and reverse replication for rollback. | 2–3 | Steps 2.3, 3.5 | |
| "How do you limit the blast radius of one database?" (REL 10) | Table groups on separate primaries, then key ranges on separate shards, so a failover hits one slice. | 2–3 | Steps 2.1, 3.5 | |
| "What stops you mid-split?" (REL 1) | The RDS instance and storage quotas; we raise them before each split. | 3 | R3.6 | |
| Performance | "Why not store edits in Postgres?" (PERF 3) | Edits alone would roughly double the busiest database's writes; in memory, one owner per file handles them at memory speed. | 1 | Step 1.1, R1.7 |
| "What does sharding do to latency?" (PERF 1) | About 0.3 ms per query for the proxy hop, and much worse tails for scatter-gathers: at 64 shards, nearly half of them hit some shard's p99. | 3 | R3.6 | |
| Security | "How do you make sure nothing writes the old copy?" (SEC 3) | Revoke the application role's privileges there, and keep a never-deleted moved fence row that every write checks under a lock. | 2–3 | Steps 2.3, 3.5 |
| Cost | "Is sharding cheaper?" (COST 6) | No: capacity costs the same per vCPU; sharding makes more of it possible, and the first split costs about 26% more for a 35% higher IOPS ceiling now, plus room to keep splitting. | 2–3 | R2.6, R3.6 |
| "Where does transfer cost hide?" (COST 8) | In AZ-unaware hops between the application, the proxy and the poolers: about $37,300 a month in our sizing. | 3 | R3.6 | |
| "Why build it instead of buying a distributed database?" (COST 11) | Months of runway, years of RDS expertise, and a migration of the most critical data made the new store the riskier path. | 3 | R3.7 | |
| Operations | "How do you roll out something this risky?" (OPS 6) | Logical sharding behind flags, percentage rollout, shadow planning and shadow reads, rollback in seconds before any data moves. | 3 | Step 3.2 |
| "How do you know a shard is healthy?" (OPS 8) | CPU and IOPS per shard, slot lag, proxy p99 and rejects, XID and multixact age, journal persistence time. | 2–3 | R3.9 | |
| Sustainability | "Do you run production-sized fleets everywhere?" (SUS 2) | No: non-production keeps the same logical topology on far fewer physical databases. | 3 | R3.10 |
Rubric Across Levels
| Dimension | L5 (Round 1) | L6 (Round 2) | L7 (Round 3) |
|---|---|---|---|
| Workload split | Keeps the database off the keystroke path, with numbers | Protects that split as features grow | Keeps it while the database becomes a routed system |
| Concurrency | Last-writer-wins per property in server order | Drain proofs: a lock that waits out open transactions | Fence rows checked under FOR SHARE; multixact cost |
| Data movement | Checkpoints and their loss window | Logical replication, slots, sequences, LSN checks, reverse replication | Full copies 1 → N, abort before the commit point, reverse moves, cleanup after the window |
| Correctness across stores | One database, ACID | No transactions across groups | Partial commits, operation records, outbox, sweepers, global IDs |
| Honesty | Labels assumptions | Separates Figma's steps from its own additions | Says what Figma didn't publish, and what the numbers can't show |
Sources
All sources used on this page, oldest first.
- Wallace, Rust in production at Figma, Figma blog, May 2, 2018.
- Wallace, How Figma's multiplayer technology works, Figma blog, October 16, 2019.
- Goel, Under the hood of Figma's infrastructure, Figma blog, November 21, 2019.
- Chen and Kim, GraphQL, meet LiveGraph: a real-time data system at scale, Figma blog, October 14, 2021.
- Tsung, Making multiplayer more reliable, Figma blog, October 20, 2022.
- Liang, The growing pains of database architecture, Figma blog, April 4, 2023.
- Steele, How Figma’s databases team lived to tell the scale, Figma blog, March 14, 2024.
- Bandi, Keeping it 100(x) with real-time data at scale, Figma blog, May 17, 2024.
- PostgreSQL documentation: Routine vacuuming, Vacuuming settings, Replication settings, Logical replication restrictions, CREATE VIEW, checked September 2026.
- PgBouncer, Usage (
PAUSE,RESUME), checked September 2026. - AWS documentation and pricing: Multi-AZ failover, RDS DB instance storage, Quotas for Amazon RDS, RDS CloudWatch metrics, Amazon RDS pricing, and the AWS Price List for RDS and S3 in
us-west-2, checked September 2026.