Skip to content
System Design

CAP Theorem interview questions and answers

15 questions, 9 with code. About 1 h to read them all.

By TestLab · Updated

Plan for an interview
15 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

What is the CAP theorem, and why does it matter for distributed systems?

The CAP theorem states that when a network partition happens, a distributed data store must choose between consistency (every read sees the latest write, i.e. linearizability) and availability (every request to a non-failed node gets a non-error response). Because partitions can always happen in a real network, it is not “pick any two of three”: partition tolerance is a requirement, and the real choice is how the system behaves while a partition lasts. It matters because it forces designers to decide, per feature, whether a stale or conflicting answer is worse than an error or a wait.

Read full answer

Applies toreplicated databases, caches, coordination services and any service whose state is copied across nodes or data centres

The Three Properties (as Defined in the Proof)

Property Precise meaning
Consistency © Linearizability: operations appear to happen in a single order that respects real time; a read returns the most recent completed write
Availability (A) Every request received by a non-failing node returns a non-error response (no time bound, but it must eventually answer)
Partition tolerance (P) The system keeps operating even though messages between nodes may be lost or delayed indefinitely

Eric Brewer proposed it in 2000; Seth Gilbert and Nancy Lynch proved it formally in 2002.

A Concrete Scenario

Two data centres, DC-East and DC-West, each hold a replica of account balances. The link between them fails. A client in the east writes balance = 50.

  • CP choice: DC-West cannot confirm it has the latest value, so it rejects or blocks reads and writes for that data (or only the side with a majority continues). Users in the west see errors or timeouts, but nobody reads a stale balance.

  • AP choice: both sides keep serving. DC-West still returns the old balance = 100, and may even accept its own writes. After the link heals, replicas reconcile (last-writer-wins, merge, or application logic).

Why “Pick Two” Is Misleading

  • Partitions are not optional: switches fail, cables are cut, GC pauses and overloaded nodes look like partitions. A system that “gives up P” simply has undefined behaviour when they occur.

  • A single-node database is often labelled “CA”, but that only means it is not distributed; if its one node fails, it is neither consistent-and-available nor partition-tolerant in any useful sense.

  • The choice is per operation and configurable: Cassandra and DynamoDB let you choose consistency levels per request, and MongoDB uses read and write concerns.

Beyond CAP: PACELC

CAP only describes behaviour during a partition. PACELC (Daniel Abadi) adds: Else, when the network is fine, the system trades Latency vs Consistency. Synchronous replication across regions gives strong consistency but adds a round trip to every write; asynchronous replication is faster but can serve stale reads.

Why It Is Important in Practice

Design question CAP-informed answer
Can a user see an old value briefly? If yes (likes, feeds), AP plus eventual consistency is fine
Is a wrong value dangerous? If yes (balances, inventory reservations, leader election), prefer CP behaviour
What happens during a partition? Define it explicitly: errors, read-only mode, or accept writes and reconcile
How fast must writes be across regions? PACELC: strong consistency costs latency even without failures

Interview Tip

State CAP precisely (linearizability vs availability during a partition), reject the “pick two” framing, walk through the two-data-centre scenario, and add PACELC for the normal case. Mentioning that real systems tune this per operation shows practical understanding.

2

How does the CAP theorem define consistency in a distributed system?

In the CAP theorem, consistency means linearizability: the system behaves as if there were a single copy of the data, and every operation takes effect atomically at some instant between its start and its completion. So once a write completes, every later read, on any node, returns that value or a newer one. This is much stronger than the “C” in ACID (which means the database keeps its integrity constraints) and stronger than eventual or causal consistency.

Read full answer

Applies toreplicated databases, key-value stores, coordination services (etcd, ZooKeeper), and distributed locks or counters

What Linearizability Requires

  1. Single global order: all operations can be placed in one sequence.

  2. Real-time respect: if operation A finishes before B starts, A comes before B in that order.

  3. Reads return the latest write in that order.

Example: What Is and Is Not Linearizable

Time  ->
Client 1:  write(x = 1) [-------]
Client 2:                          read(x) -> 0     (NOT linearizable: the write had finished)
Client 3:       read(x) -> 1 [---]                 (allowed: overlaps the write)

If a read overlaps a write, it may return the old or new value. Once it returns the new value, no later read may return the old one (“no going back in time”).

CAP Consistency vs Other Meanings

Term Meaning Relation to CAP’s C
ACID consistency Transactions keep constraints valid (foreign keys, checks) Different concept entirely
Linearizability Single-copy, real-time ordering This is CAP’s C
Sequential consistency One global order consistent with each client’s order, but not real time Weaker
Causal consistency Causally related operations are seen in order by everyone Weaker; achievable while available
Read-your-writes A client always sees its own writes Weaker, per-session guarantee
Eventual consistency Replicas converge if writes stop Much weaker

Why It Conflicts With Availability During a Partition

Suppose replicas R1 and R2 are partitioned and a write x = 1 succeeds at R1. A client then reads from R2:

  • To stay linearizable, R2 must refuse or wait, because it cannot know whether a newer write exists.

  • To stay available, R2 must answer, and it can only return the stale x = 0.

That is the core of the CAP proof.

How Systems Provide It

  • Consensus protocols (Raft, Paxos): etcd, ZooKeeper, Consul, CockroachDB and Spanner order writes through a majority, so a minority side of a partition cannot serve linearizable reads or writes.

  • Leader-based replication with synchronous acknowledgement plus reads from the leader (with a lease or read index check, so a deposed leader does not answer).

  • Quorum reads and writes (R + W > N) give overlap between read and write sets, which helps but is not sufficient for full linearizability on its own (concurrent writes, sloppy quorums and clock-based conflict resolution can still break it).

Costs

  • Latency: each write (and some reads) needs a round trip to a majority, which is significant across regions (PACELC’s “else consistency costs latency”).

  • Reduced availability during partitions for the minority side.

Interview Tip

Say clearly that CAP’s C is linearizability, define it with the “single copy plus real-time order” idea, and contrast it with ACID consistency and eventual consistency. Showing why a partitioned replica must either refuse or return stale data demonstrates you understand the theorem, not just its name.

3

What does availability mean in the CAP theorem?

In the CAP theorem, availability means every request received by a non-failing node must eventually get a non-error response, even while the network is partitioned. The response does not have to contain the latest data; it just cannot be an error, a refusal, or a wait that never ends. This is a strict, theoretical definition: a system that serves only the majority side of a partition is not “available” in the CAP sense, even if it has excellent uptime.

Read full answer

Applies toreasoning about replicated databases and services under network partitions; contrast with operational availability (uptime and SLOs), which is a separate topic

Key Points of the Definition

Aspect CAP availability
Who must answer Every node that has not crashed
What it must return A non-error response (reads may be stale; writes must be accepted)
Timing Eventually; the formal definition has no time bound, though real systems need timeouts
During partitions Must hold on both sides of the split

CAP Availability vs Uptime

CAP availability Operational availability
A property of the algorithm during a partition A measured percentage (for example 99.9 percent of requests succeed)
Binary: either every live node answers or not Continuous: measured over time
A CP system fails it during partitions A CP system can still achieve very high uptime if partitions are rare and the majority side keeps serving

A system like etcd or Spanner is CP under CAP, yet can have very high practical availability, because most clients can reach the majority side most of the time.

Scenario

Replicas in DC-East and DC-West lose their link. A user in the west sends GET /cart.

  • AP system (for example Cassandra at consistency level ONE): DC-West returns the cart it has, possibly missing an item added in the east a moment ago. Writes are accepted locally and reconciled later.

  • CP system (for example etcd, or Cassandra at QUORUM if the west side holds a minority of replicas): DC-West cannot reach a majority, so it returns an error or times out.

What Makes a System CAP-Available

  • Any replica can accept reads and writes without coordinating synchronously with others (multi-leader or leaderless designs).

  • Conflicts are handled after the fact: last-writer-wins timestamps, version vectors with merge logic, or CRDTs.

  • Clients accept weaker guarantees: eventual consistency, or session guarantees such as read-your-writes when they stay on one replica.

Trade-offs of Choosing Availability

  • Stale reads while the partition lasts.

  • Conflicting writes that need resolution; last-writer-wins can silently lose an update.

  • Application complexity: business logic must tolerate and reconcile divergence (for example, a merged shopping cart).

Practical Nuance

  • Timeouts: a request that never returns is unavailable in practice, so real systems define timeouts and treat slow responses as failures.

  • Choice per operation: a system may keep reads available (serving possibly stale data) while rejecting writes that need strong guarantees.

  • Degraded modes: read-only mode or cached responses during a partition are common compromises.

Interview Tip

Give the precise definition (every non-failing node answers with a non-error response), then contrast it with uptime percentages. Pointing out that CP systems can still have high real-world availability because they serve the majority side is a sign of mature understanding.

4

What is partition tolerance in a distributed system under the CAP theorem?

Partition tolerance means the system continues to operate even when the network drops or delays messages between nodes, splitting them into groups that cannot communicate. In the CAP theorem, a partition is any situation where some nodes cannot hear from others, and the theorem says that during such a partition the system must give up either linearizable consistency or availability. Because real networks always can partition, partition tolerance is not an optional property for a distributed system; what you choose is the behaviour during the partition.

Read full answer

Applies toany system with data or state spread across machines, racks, availability zones or regions

What Counts as a Partition

Cause Example
Link or switch failure A top-of-rack switch fails; one rack is cut off
Inter-region network issues A trans-continental link degrades between two data centres
Misconfiguration A firewall rule or routing change blocks traffic between subnets
Long pauses A garbage collection pause or VM freeze makes a node unresponsive for seconds
Overload Packet loss and timeouts under heavy load look like a split
Asymmetric failures A can reach B but B cannot reach A

From the inside, a node cannot tell whether another node crashed or is merely unreachable or slow. That uncertainty is what makes partitions hard.

What the System Must Decide

Scenario: DC-East and DC-West are partitioned; a write stock = 4 arrives at DC-East.

Behaviour What happens Category
East accepts only if it has a majority; West refuses No stale data; West users get errors CP
Both sides accept reads and writes Everyone gets answers; data diverges; reconcile later AP
Both refuse all writes, serve cached reads A degraded, partial compromise (still not CAP-available for writes) Design choice

Why “Giving Up P” Is Not Real

A design that “assumes no partitions” (for example, synchronous replication with no plan for a lost link) will, when a partition occurs, either:

  • block forever (lost availability), or

  • split brain: both sides think they are primary and accept conflicting writes (lost consistency).

So “CA” is really a description of a single-node or tightly coupled system, which avoids the question by not being distributed, not a third option.

Techniques for Tolerating Partitions

  • Quorums and consensus (Raft, Paxos): only a majority side can make progress, preventing split brain.

  • Leases and fencing tokens: a leader’s authority expires, and storage rejects writes carrying stale tokens from a deposed leader.

  • Failure detectors with timeouts (heartbeats, phi-accrual in Cassandra and Akka) decide when to treat a peer as unreachable.

  • Conflict resolution for AP systems: version vectors, last-writer-wins, CRDTs.

  • Anti-entropy and hinted handoff to repair replicas after the partition heals.

  • Placement across failure domains so that a single partition leaves a majority intact (for example three zones, not two).

Interview Tip

Define a partition broadly (lost or delayed messages, including pauses), explain that nodes cannot distinguish slow from dead, and stress that partition tolerance is required, so the real trade-off is C vs A during the split. Mentioning split brain, quorums and fencing shows practical depth.

5

Which real systems favour consistency over availability during a partition?

Systems that favour consistency (CP behaviour) refuse or delay requests on the side of a partition that cannot confirm it has the latest data, usually because it cannot reach a majority. Classic examples are coordination services built on consensus: etcd, ZooKeeper and Consul, and distributed SQL databases such as Google Spanner, CockroachDB, YugabyteDB and TiDB. HBase and a MongoDB replica set with majority read and write concerns also behave this way. Many are configurable, so the label describes their default or recommended setup, not a fixed property.

Read full answer

Applies toleader election, configuration and metadata stores, financial ledgers, inventory and any data where a wrong value is worse than an error

Examples

System How it stays consistent What happens during a partition
etcd (Raft) Writes and linearizable reads go through the Raft leader with majority acknowledgement The minority side cannot elect a leader; requests fail or time out
ZooKeeper (ZAB) Leader orders all writes; quorum of servers must agree Minority servers stop serving writes; clients reconnect to the majority
Consul (Raft) Raft for the catalog and KV store Minority servers lose leadership; stale reads possible only if explicitly requested
Google Spanner Paxos groups plus TrueTime for externally consistent transactions Replicas cut off from a Paxos majority cannot serve strongly consistent operations
CockroachDB, YugabyteDB, TiDB Raft per range; serializable or snapshot transactions Ranges whose majority is unreachable become unavailable until the majority returns
HBase One RegionServer owns each region; ZooKeeper coordinates ownership Regions on an isolated server are unavailable until reassigned
MongoDB (majority concerns) Writes acknowledged by a majority; reads with readConcern: "linearizable" or "majority" from the primary A primary isolated in the minority steps down; the majority side elects a new one

Concrete Scenario: Kubernetes and etcd

A Kubernetes cluster uses a three-member etcd cluster across three zones. Zone C loses its network.

  • Zones A and B still form a majority (2 of 3): etcd keeps accepting writes, so the API server works.

  • The etcd member in zone C cannot reach the majority: it rejects writes and linearizable reads.

  • When zone C reconnects, it catches up from the Raft log. No conflicting cluster state exists to reconcile.

If two zones were lost, the remaining member would refuse writes entirely: the cluster chooses correctness over availability.

Configurability

  • MongoDB: writeConcern: {w: 1} and reads from secondaries can return data that is later rolled back; w: "majority" with majority or linearizable reads gives CP behaviour.

  • Cassandra and DynamoDB are usually described as AP, but quorum reads and writes (or DynamoDB strongly consistent reads) shift individual operations toward consistency, at the cost of availability when replicas are unreachable.

  • Consul and etcd offer stale read modes that trade consistency for availability on reads.

Why Choose CP

  • Leader election and locks: two leaders at once would be dangerous.

  • Money and inventory: a double spend or oversold item is unacceptable.

  • Cluster metadata: conflicting configuration is hard to repair.

Costs

  • Requests fail on the minority side during partitions.

  • Write latency includes a majority round trip (PACELC), which is significant across regions.

Interview Tip

Name consensus-based systems (etcd, ZooKeeper, Spanner, CockroachDB), explain the majority mechanism and what the minority side does, and note that labels depend on configuration. Walking through a three-zone etcd example makes the answer concrete.

6

Which systems prefer availability over consistency during a partition?

Systems that favour availability (AP behaviour) let every reachable replica keep serving reads and writes during a partition, accepting that replicas may diverge temporarily and must be reconciled later. Well-known examples are Amazon’s Dynamo design and the databases it inspired: Apache Cassandra, ScyllaDB and Riak, plus Amazon DynamoDB with eventually consistent reads, CouchDB with multi-master replication, and DNS. Several are tunable, so a single request can ask for stronger guarantees and give up some availability.

Read full answer

Applies toshopping carts, user activity feeds, session data, metrics and IoT ingestion, product catalogues, content delivery, and any data where a briefly stale answer is acceptable

Examples

System How it stays available How divergence is resolved
Amazon Dynamo (2007 paper) Leaderless replication, sloppy quorums, hinted handoff Vector clocks; application merges conflicts (for example the shopping cart union)
Apache Cassandra, ScyllaDB Any replica can coordinate; consistency level per request (ONE, LOCAL_QUORUM, QUORUM, ALL) Last-writer-wins by timestamp; read repair, hinted handoff and anti-entropy repair
Riak KV Leaderless with sloppy quorums Siblings with vector clocks, or CRDT data types (counters, sets, maps)
DynamoDB Eventually consistent reads by default; global tables replicate across regions Last-writer-wins between regions for global tables
CouchDB Multi-master replication, works offline Deterministic winning revision plus stored conflicts for the application
DNS Caches serve records until TTL expires Changes propagate as caches expire

Concrete Scenario: Shopping Cart

DC-East and DC-West lose their link. A customer adds “headphones” in the east; a few seconds later their phone, routed to the west, adds “charger”.

  • AP result: both writes succeed immediately. Each side shows a cart containing only its own addition for a while.

  • After healing: replicas exchange versions. With a merge rule (union of items), the cart becomes {headphones, charger}. With plain last-writer-wins, one addition could be lost, which is why Dynamo let the application merge.

Tunable Consistency

-- Cassandra (cqlsh)
CONSISTENCY ONE;           -- most available, may read stale data
SELECT * FROM carts WHERE user_id = 'u42';

CONSISTENCY LOCAL_QUORUM;  -- majority of replicas in the local data centre
SELECT * FROM carts WHERE user_id = 'u42';

With replication factor 3, writing and reading at QUORUM (2 of 3) means read and write sets overlap, so reads usually see the latest acknowledged write, but requests fail if two replicas are unreachable.

Why Choose AP

  • User experience: an always-working “add to cart” is worth more than perfect consistency.

  • Multi-region low latency: writes complete locally without waiting for distant replicas (PACELC: low latency in the normal case too).

  • Offline and edge use: mobile apps and edge nodes keep working while disconnected.

Costs

  • Stale reads and conflicting writes.

  • Lost updates with last-writer-wins and clock skew.

  • No multi-key invariants: “never oversell stock” or “unique username” cannot be guaranteed without coordination.

  • Repair overhead: anti-entropy repairs must run regularly (Cassandra recommends repairs within gc_grace_seconds to avoid deleted data reappearing).

Interview Tip

Name Dynamo-style systems (Cassandra, Riak, DynamoDB), explain how they keep accepting writes and how conflicts are reconciled, and mention tunable consistency levels. The shopping cart merge example shows you understand both the benefit and the reconciliation cost.

7

What does eventual consistency mean in the context of the CAP theorem?

Eventual consistency is a guarantee that if no new updates are made to a piece of data, all replicas will eventually converge to the same value. It is the consistency model that AP systems typically offer: during a partition (and even during normal operation with asynchronous replication) replicas may return different or stale values, but once messages flow again, they exchange updates and converge. It says nothing about how long convergence takes or which value wins, so applications usually add stronger session guarantees or conflict-resolution rules.

Read full answer

Applies toDynamo-style databases (Cassandra, Riak, DynamoDB eventually consistent reads), DNS, CDN and cache invalidation, asynchronous read replicas, and offline-first or multi-region apps

How It Relates to CAP

  • During a partition, an AP system keeps accepting writes on both sides, so replicas diverge.

  • Eventual consistency promises that after the partition heals and writes quiet down, divergence disappears.

  • Linearizability (CAP’s C) is not provided: a read may not reflect a completed write.

Example Timeline

t0  R1: x=1   R2: x=1          (in sync)
t1  partition begins
t2  client writes x=2 at R1    R1: x=2   R2: x=1   (read at R2 returns stale 1)
t3  partition heals; replication / anti-entropy runs
t4  R1: x=2   R2: x=2          (converged)

Mechanisms That Make Replicas Converge

Mechanism Purpose
Asynchronous replication Pushes writes to other replicas in the background
Hinted handoff A node stores writes for an unreachable replica and delivers them later
Read repair A read that sees differing versions updates stale replicas
Anti-entropy (Merkle trees) Periodic comparison of replicas to find and fix differences
Conflict resolution Last-writer-wins timestamps, version vectors plus merge, or CRDTs

Convergence Is Not Correctness

If two replicas accept conflicting writes, they will converge, but to what?

  • Last-writer-wins: simple, but silently discards one update and depends on clock accuracy.

  • Application merge: for example union of shopping cart items.

  • CRDTs: data types designed so concurrent updates merge deterministically (counters, sets, registers, maps). This gives strong eventual consistency: replicas that received the same updates are equal, regardless of order.

Stronger Guarantees Built on Top

Pure eventual consistency can confuse users, so systems add session guarantees, all achievable without giving up availability in many designs:

Guarantee Meaning
Read-your-writes A user always sees their own updates
Monotonic reads A user never sees data go backward in time
Monotonic writes A user’s writes are applied in order
Causal consistency If B depends on A (a reply to a post), nobody sees B without A

Typical techniques are sticky sessions to one replica, client-tracked version tokens, or reading from the leader after a recent write.

Where It Fits and Where It Does Not

Good fit: like counts, view counters, social feeds, product reviews, presence indicators, analytics, caches, DNS.

Poor fit: account balances, inventory reservations, unique usernames, leader election, anything with invariants spanning several records.

Interview Tip

Define eventual consistency as convergence once updates stop, connect it to AP behaviour during partitions, explain how replicas converge and how conflicts are resolved, and mention session guarantees and CRDTs. Stating that it gives no time bound and no rule for which value wins shows precise understanding.

8

What design trade-offs does the CAP theorem force in distributed systems?

The CAP theorem forces designers to decide what each part of the system does when replicas cannot communicate: refuse or delay requests to stay correct (CP), or keep answering and accept stale or conflicting data (AP). Combined with PACELC, it also forces a second trade-off in normal operation: lower latency or stronger consistency. In practice this shapes replication strategy, data placement, conflict resolution, user experience during failures and operational complexity, and the best systems make these choices per feature, not for the whole system.

Read full answer

Applies toreplicated databases, multi-region services, caches, coordination services and microservice data ownership

The Main Trade-offs

Trade-off Leaning consistent Leaning available or fast
Behaviour during partition Minority side returns errors or read-only responses All sides accept requests; reconcile later
Normal-case latency (PACELC) Synchronous replication to a majority, possibly cross-region Local writes with asynchronous replication
Data correctness No stale reads, no conflicting writes Stale reads, conflicts, possible lost updates
Application complexity Simpler logic; handle errors and retries Conflict resolution, idempotency, merge logic
User experience Occasional “try again later” Always responds, sometimes with outdated data
Invariants Can enforce uniqueness, balances, stock limits Cannot enforce cross-replica invariants without coordination
Operations Need odd-numbered quorums, careful placement across failure domains Need repairs, anti-entropy, conflict monitoring

Scenario: One Product, Two Kinds of Data

An online store runs in two regions that lose their link.

  • Product reviews and recommendations: serve locally from AP storage. A review written in one region appears in the other a little later. Nobody is harmed.

  • Stock reservation for the last unit: allowing both regions to sell it would oversell. Route reservations to the region that owns that item’s stock (a single leader or a consensus group); the other region shows “temporarily unable to reserve” or queues the order for confirmation.

The same system uses both behaviours.

Common Design Decisions Driven by CAP

  1. Classify data by tolerance for staleness: money, identity and inventory vs feeds, counters and caches.

  2. Choose a replication model: single leader, multi-leader or leaderless.

  3. Pick consistency levels per operation: Cassandra LOCAL_QUORUM for writes, ONE for non-critical reads; MongoDB read and write concerns; DynamoDB strongly consistent reads where needed.

  4. Define partition behaviour explicitly: errors, read-only mode, queued writes, or local acceptance with later reconciliation.

  5. Plan conflict resolution for AP data: last-writer-wins, version vectors with merges, or CRDTs.

  6. Make operations idempotent so retries after timeouts do not double-apply.

  7. Place replicas across failure domains (three zones rather than two) so one failure leaves a majority.

Alternatives That Soften the Trade-off

  • Compensation instead of prevention: accept orders available-first, then cancel or apologise if stock runs out (common in e-commerce and airline overbooking).

  • Escrow or quotas: give each region a share of stock it can sell without coordination.

  • CRDTs for counters and sets that merge safely.

  • Causal or session consistency, which can be offered while staying available and fixes many user-visible anomalies.

Interview Tip

Frame CAP as a per-feature decision about partition behaviour, add the PACELC latency trade-off, and give an example where one application uses CP for some data and AP for others. Mentioning compensation, escrow and CRDTs as ways to soften the trade-off shows senior-level thinking.

9

How would you design a highly available system, and what CAP trade-offs follow?

To design for high availability under the CAP theorem, you let every replica keep serving reads and writes during a partition (AP behaviour), replicate data asynchronously across failure domains, route users to any healthy replica, and handle the consequences: stale reads, conflicting writes and later reconciliation. The trade-off is giving up linearizable consistency for that data, so you must decide which data can tolerate it and keep a small, strongly consistent core for the rest.

Read full answer

Applies tomulti-region user-facing services, shopping carts, feeds, session and profile data, IoT ingestion, and content platforms

Design Steps

1. Classify data by tolerance for staleness

Data Tolerance Storage choice
Feeds, likes, view counts High AP store, eventual consistency
Shopping cart, user preferences Medium (merge acceptable) AP store with merge rules or CRDTs
Payments, stock reservation, unique usernames Low CP store or single owner per key

2. Replicate across failure domains

Run replicas in at least three availability zones and, if needed, several regions. Use leaderless or multi-leader replication (Cassandra, DynamoDB global tables, Riak, CouchDB) so any replica can accept writes.

3. Route to any healthy replica

Geo-DNS or anycast plus local load balancers send users to the nearest healthy region. Clients retry on another replica if one fails.

4. Choose consistency levels per operation

-- Cassandra: available and fast for feed reads
CONSISTENCY LOCAL_ONE;
SELECT * FROM feed WHERE user_id = 'u42' LIMIT 50;

-- Stronger within the local data centre for profile updates
CONSISTENCY LOCAL_QUORUM;
UPDATE profiles SET display_name = 'Asha' WHERE user_id = 'u42';

LOCAL_QUORUM keeps each region independent during an inter-region partition while still overlapping reads and writes inside the region.

5. Plan conflict resolution

  • Last-writer-wins for fields where losing a concurrent update is acceptable.

  • Merge functions or CRDTs (sets, counters) for carts and counters.

  • Version vectors and application-level resolution where business rules matter.

6. Add session guarantees

Read-your-writes (route a user to the same region or pass a version token), monotonic reads, and causal ordering for replies, so users do not see confusing results even though the system is eventually consistent.

7. Make writes idempotent

Clients retry after timeouts, so every write carries an idempotency key or is naturally idempotent (set a value, add to a set).

Trade-offs You Accept

Trade-off Impact Mitigation
Stale reads Users briefly see old data Short replication lag, read-your-writes
Conflicting writes Two regions update the same record Merge rules, CRDTs, version vectors
Lost updates with last-writer-wins Silent data loss Avoid LWW for important fields; use merges
No global invariants Cannot guarantee uniqueness or stock limits Route those operations to a CP component
Operational repair work Anti-entropy repairs, monitoring divergence Scheduled repairs, metrics on replication lag
Harder testing Concurrency and reconciliation edge cases Chaos and partition tests

Example: Partition Between Two Regions

  • Feed and cart: both regions continue normally; carts are merged after healing.

  • Checkout: the order is accepted locally, but payment and stock reservation go to the owning region. If it is unreachable, the order is queued as “pending confirmation”, or the user is asked to retry. This is a deliberate CP island inside an AP system.

Availability Beyond CAP

CAP covers only partitions. Real high availability also needs redundancy for crashes, health checks, automatic failover, load shedding, graceful degradation and capacity planning; those are reliability concerns rather than CAP trade-offs.

Interview Tip

Describe the AP architecture (multi-zone replicas, any-replica writes, per-operation consistency, conflict resolution, session guarantees), list the trade-offs clearly, and show that critical operations stay on a CP path. That balance is what interviewers look for.

10

What strategies keep a service working during a network partition?

During a partition you can keep a service working by deciding in advance, per operation, how each side behaves: let the majority side continue (quorum-based CP), let both sides continue and reconcile later (AP), or degrade gracefully (read-only mode, cached responses, queued writes). Supporting techniques include quorums and leases to prevent split brain, hinted handoff and anti-entropy repair, CRDTs and merge rules, idempotent retries, circuit breakers and timeouts, and routing users to the side that can serve them.

Read full answer

Applies toreplicated databases, multi-zone and multi-region services, microservices that depend on remote data, and edge or mobile clients

Scenario

Two data centres, DC-East (2 replicas) and DC-West (1 replica), lose their link. Orders, product catalogue and user sessions are replicated across both.

Strategy 1: Majority Side Continues (CP)

  • Consensus (Raft, Paxos) or quorum writes: DC-East, holding 2 of 3 replicas, keeps accepting writes; DC-West rejects writes for that data.

  • Leases and fencing tokens stop an old leader in DC-West from accepting writes after its lease expires.

  • Global load balancing redirects users in the west to DC-East if the client-to-East path still works.

Best for: orders, payments, inventory, configuration.

Strategy 2: Both Sides Continue and Reconcile (AP)

  • Each side accepts writes locally (Cassandra at LOCAL_QUORUM or ONE, DynamoDB global tables, multi-leader replication).

  • Hinted handoff stores updates for unreachable replicas; anti-entropy repair with Merkle trees fixes the rest after healing.

  • Conflict resolution: CRDTs for counters and sets, application merges for carts, last-writer-wins only where loss is acceptable.

Best for: carts, feeds, likes, sessions, telemetry.

Strategy 3: Graceful Degradation

Technique Example
Read-only mode Serve the catalogue but disable editing on the minority side
Cached or stale responses Show the last known balance labelled “as of 10:42”
Queue writes for later Accept an order as “pending” and confirm when the link returns
Feature flags Temporarily switch off non-essential features that need cross-region data
Bounded local authority Escrow: each region may sell up to its pre-allocated share of stock

Strategy 4: Protect the Rest of the System

  • Timeouts and circuit breakers so calls to the unreachable side fail fast instead of tying up threads.

  • Bulkheads to isolate partition-affected dependencies.

  • Idempotency keys so client retries do not create duplicate orders when the network recovers.

  • Backpressure and load shedding if the surviving side receives redirected traffic.

import time

class CircuitBreaker:
    def __init__(self, threshold=3, reset_after=5.0):
        self.failures, self.threshold, self.reset_after = 0, threshold, reset_after
        self.opened_at = None

    def call(self, fn, fallback):
        if self.opened_at and time.monotonic() - self.opened_at < self.reset_after:
            return fallback()                       # fail fast while open
        try:
            result = fn()
            self.failures, self.opened_at = 0, None
            return result
        except ConnectionError:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time.monotonic()
            return fallback()

def remote_inventory():
    raise ConnectionError("DC-East unreachable")

cb = CircuitBreaker()
for _ in range(4):
    print(cb.call(remote_inventory, lambda: "cached: 12 in stock (may be stale)"))
# cached: 12 in stock (may be stale)   (printed 4 times)

After the Partition Heals

  • Run repairs and reconcile conflicts; monitor replication lag until it returns to normal.

  • Replay queued writes in order, using idempotency keys.

  • Review conflict and error metrics: how many writes were rejected or merged.

Interview Tip

Organise the answer as majority-continues (CP), both-continue (AP) and graceful degradation, with supporting techniques such as fencing, hinted handoff, circuit breakers and idempotent retries. Choosing differently for orders and for feeds in one scenario shows practical judgement.

11

How would you apply the CAP theorem to a financial transaction system?

For financial transactions, the core ledger should prefer consistency (CP behaviour): during a partition, it is better to reject or delay a payment than to allow a double spend, overdraft or lost transfer. You keep balances and ledger entries on strongly consistent storage (a single-leader database with synchronous replication or a consensus-based distributed SQL database), make every operation idempotent and auditable, and push availability-friendly work (notifications, analytics, statement views) to eventually consistent paths. Many real payment systems also use asynchronous workflows with authorisation holds and later settlement to remain responsive without breaking correctness.

Read full answer

Applies tobanking ledgers, wallets, payment processors, trading and brokerage systems, and billing platforms

Why CP for the Ledger

Risk under AP behaviour Example
Double spend Two regions both approve withdrawals from the same 100 balance
Lost updates Last-writer-wins discards a deposit
Invariant violations Account goes negative, or debits and credits do not balance
Reconciliation cost Manual correction, regulatory and customer trust issues

Scenario

Regions East and West lose their link. A customer has a balance of 100 and sends two 80 payments, one routed to each region.

  • AP design: both regions approve; the balance becomes -60 after reconciliation. Unacceptable.

  • CP design: the account is owned by one consensus group whose majority is in East. East processes the first payment; West cannot reach the majority, so its request fails or is retried against East. The second payment is declined for insufficient funds.

Architecture

  1. Strongly consistent ledger: PostgreSQL or similar with synchronous replication and serializable or carefully locked transactions, or distributed SQL (Spanner, CockroachDB, YugabyteDB) with replicas in three zones so one zone failure keeps a majority.

  2. Double-entry, append-only ledger: every transfer writes balanced debit and credit entries; corrections are new entries, never edits.

  3. Idempotency keys: retries after timeouts return the original result instead of charging twice.

  4. Single owner per account: route all writes for an account to its home shard or region to avoid cross-region coordination on every transaction.

  5. Sagas for cross-service flows: payment, fraud check and settlement coordinated with compensating actions rather than distributed locks.

  6. Event outbox: committed ledger changes are published reliably to downstream systems.

  7. Eventually consistent read models: statements, dashboards and notifications built from events can lag safely.

-- Debit only if funds suffice, atomically, with idempotency
BEGIN;
INSERT INTO payments (idempotency_key, account_id, amount)
VALUES ('req-7f3a', 'acc-1', 80)
ON CONFLICT (idempotency_key) DO NOTHING;

UPDATE accounts
SET balance = balance - 80
WHERE id = 'acc-1' AND balance >= 80;   -- 0 rows updated means insufficient funds: roll back
COMMIT;

Production code checks both statements’ affected-row counts and rolls back if either did nothing.

Staying Usable During Partitions

  • Fail clearly: return “payment temporarily unavailable, please retry”, never a false success.

  • Authorisation holds: card networks approve a hold that is settled later; limits are enforced at authorisation time against a consistent balance.

  • Bounded offline limits: some systems allow small offline transactions up to a pre-approved amount (an escrow-like budget), accepting limited risk.

  • Queue non-critical actions: receipts and emails wait until the system recovers.

PACELC Considerations

Synchronous replication across regions adds latency to every write. Common compromises: keep a majority within one region (across zones) for low latency, and replicate to another region asynchronously for disaster recovery, accepting a small recovery point objective after a full regional loss, or pay cross-region latency where regulations require zero data loss.

Interview Tip

Choose CP for balances and explain why with a double-spend example, then show how idempotency, double-entry ledgers, sagas and asynchronous read models keep the system practical. Noting the latency cost of synchronous replication (PACELC) shows a complete view.

12

When might a system shift from CA-like behaviour to AP during operation?

A system looks “CA” only while the network is healthy: every replica is reachable, so it can be both consistent and available. The label is misleading, because that is just the normal no-partition state, and CAP only forces a choice when a partition happens. A system “switches to AP” when a partition or degraded link occurs and its configuration tells it to keep serving on every side instead of refusing requests, for example a multi-region database that falls back from synchronous to asynchronous replication, or a Cassandra client that downgrades its consistency level when replicas are unreachable.

Read full answer

Applies tomulti-region databases, replication setups with fallback modes, Cassandra and DynamoDB style tunable consistency, and caches in front of authoritative stores

Why “CA” Is a Misleading Label

  • With no partition, any well-designed system can be consistent and available; that is not a design choice.

  • A single-node database is called “CA” because it never partitions internally, but if the node or its network fails, it is simply unavailable.

  • The meaningful question is what happens when a partition occurs: refuse (CP) or answer with possibly stale data (AP).

Scenario: Two Data Centres Lose Their Link

An e-commerce platform replicates its database between DC-East (primary) and DC-West (replica) with synchronous replication, so every committed write exists in both places. While the link is healthy, reads in the west are up to date and every request succeeds: the “CA-like” state.

At 14:05 the inter-DC link fails.

Configuration Behaviour CAP category
Strict synchronous replication East cannot commit without West’s acknowledgement, so writes block; West serves only stale reads or none CP (and unavailable for writes)
Fallback to asynchronous replication East keeps committing alone; West serves reads that become stale; if West is promoted, writes diverge AP
Both sides accept writes (multi-primary) Both continue; conflicts reconciled after 14:30 when the link returns AP

PostgreSQL synchronous replication blocks commits if the synchronous standby is unreachable, unless the operator or tooling changes synchronous_standby_names. Some managed setups deliberately downgrade to asynchronous after a timeout to keep accepting writes, which is a switch to AP-like behaviour with risk of data loss on failover.

Other Real Triggers

  • Client-side consistency downgrade: a Cassandra driver policy retries at a lower consistency level (QUORUM to ONE) when not enough replicas respond. Newer drivers removed the built-in downgrading policy because it silently weakens guarantees.

  • Cache fallback: when the authoritative database is unreachable, the service serves cached data (“stale-if-error”), favouring availability.

  • Mobile and offline-first apps: while online they read and write through the server; offline, they write locally and sync later.

  • DynamoDB: a strongly consistent read fails if the leader replica for the item is unavailable; clients may fall back to eventually consistent reads.

  • Operator decisions: during an incident, operators promote a replica in an isolated region to restore service, knowingly accepting possible divergence.

Risks of Switching

  • Data loss or divergence when the partition heals (writes accepted on both sides).

  • Silent weakening of guarantees if the fallback is automatic and unmonitored.

  • Split brain if two primaries exist without fencing.

Making It Safe

  • Make the switch explicit and observable: alerts, metrics and logs when consistency is downgraded.

  • Use it only for data that tolerates staleness; keep critical data on strict paths.

  • Plan reconciliation: version vectors, merge logic or manual review.

  • Use fencing tokens so a former primary’s writes are rejected after failover.

Interview Tip

Point out that “CA” just means “no partition right now”, then use the two-data-centre scenario to show how fallback replication, consistency downgrades or cache fallbacks move the system to AP behaviour. Highlighting the risks and the need to monitor such switches shows practical judgement.

13

How do quorums balance consistency and availability, and how do they relate to CAP?

A quorum is the minimum number of replicas that must take part in an operation. With N replicas, a write must be acknowledged by W and a read must contact R; if R + W > N, every read set overlaps every write set, so a read normally sees the latest acknowledged write. Choosing R and W lets you move along the consistency/availability spectrum: larger quorums give stronger consistency but fail when too many replicas are unreachable, which is exactly the CAP trade-off during a partition. Majority quorums (more than N/2) also prevent split brain, which is why consensus protocols use them.

Read full answer

Applies toDynamo-style databases (Cassandra, ScyllaDB, Riak), consensus systems (Raft, Paxos, ZooKeeper), MongoDB write concerns, and replicated storage

The Overlap Rule

With N = 3:

Setting R + W > N? Behaviour
W = 3, R = 1 Yes Fast reads; writes fail if any replica is down
W = 2, R = 2 Yes Balanced; tolerates one unreachable replica for both reads and writes
W = 1, R = 3 Yes Fast writes; reads fail if any replica is down
W = 1, R = 1 No Most available; reads can be stale
from itertools import combinations

def always_overlap(n, r, w):
    nodes = range(n)
    return all(set(rs) & set(ws)
               for rs in combinations(nodes, r)
               for ws in combinations(nodes, w))

for r, w in [(2, 2), (1, 3), (1, 1)]:
    print(f"N=3 R={r} W={w} overlap={always_overlap(3, r, w)}")
# N=3 R=2 W=2 overlap=True
# N=3 R=1 W=3 overlap=True
# N=3 R=1 W=1 overlap=False

Relation to CAP

Scenario: N = 3 replicas, two in DC-East and one in DC-West; the link fails.

  • QUORUM (R = W = 2): East can still form quorums, so it keeps working consistently. West, with one replica, cannot reach a quorum, so its requests fail: CP behaviour for the minority.

  • ONE (R = W = 1): both sides accept reads and writes, so the system is available everywhere but reads in the west can be stale and writes can conflict: AP behaviour.

Quorums do not escape CAP; they let you pick where to sit, per operation.

Majority Quorums and Consensus

  • Raft and Paxos need a majority to elect a leader and commit entries. Two disjoint majorities cannot exist, so at most one side of a partition makes progress, preventing split brain.

  • With 2f + 1 replicas, the system tolerates f failures (3 nodes tolerate 1; 5 tolerate 2).

  • This is why clusters use odd sizes and spread replicas across three failure domains.

Limits of R + W > N

Overlap alone does not guarantee linearizability:

  • Concurrent writes may be ordered by timestamps, and clock skew can make the “latest” write lose.

  • Sloppy quorums with hinted handoff (Dynamo, Riak) accept writes on substitute nodes during failures, so the overlap may not include the real replicas.

  • Partial writes: a write that reached only one replica before failing may appear in some reads and not others.

  • Read repair timing: without it, a later read can return an older value (non-monotonic reads).

Consensus protocols add a log and leader ordering to provide true linearizability.

Multi-Data-Centre Variants

Cassandra’s LOCAL_QUORUM uses a quorum within the local data centre: fast and resilient to inter-DC partitions, but cross-DC reads can be stale. EACH_QUORUM requires a quorum in every data centre: stronger but unavailable if any data centre is cut off.

Interview Tip

Explain the R + W > N overlap rule with N = 3 examples, map quorum choices to CP or AP behaviour in a partition scenario, and mention majority quorums in consensus. Adding the limits (clock skew, sloppy quorums) shows depth beyond the formula.

14

How do Cassandra and DynamoDB handle the trade-offs of the CAP theorem?

Cassandra and DynamoDB both descend from Amazon’s Dynamo design and default to availability and low latency, but neither has a fixed CAP label: they offer tunable consistency. Cassandra lets each request choose a consistency level (ONE, LOCAL_QUORUM, QUORUM, ALL and others), and supports lightweight transactions (Paxos-based compare-and-set). DynamoDB offers eventually consistent reads by default, strongly consistent reads on request within a region, ACID transactions across items, and global tables for multi-region replication. Both use replication, partitioning and repair mechanisms to stay available, and both make you choose consistency per operation.

Read full answer

Applies tohigh-throughput, multi-zone and multi-region workloads such as user profiles, carts, sessions, IoT data, messaging and time-series data

Apache Cassandra

Mechanism Role
Leaderless replication (replication factor N, often 3 per data centre) Any node can coordinate a request; no single leader to lose
Tunable consistency levels per request Choose how many replicas must respond
LOCAL_QUORUM Majority within the local data centre; isolates regions from cross-DC partitions
Hinted handoff Stores writes for replicas that are briefly down
Read repair and anti-entropy repair (Merkle trees) Bring replicas back in sync
Last-write-wins by timestamp Resolves conflicting writes per cell
Lightweight transactions (IF NOT EXISTS, IF col = x) Linearizable compare-and-set using Paxos, at higher latency
CONSISTENCY LOCAL_QUORUM;
INSERT INTO users (user_id, email) VALUES ('u42', 'asha@example.com');

-- Unique username claim: linearizable, uses Paxos (needs a quorum of replicas)
INSERT INTO usernames (name, user_id) VALUES ('asha', 'u42') IF NOT EXISTS;

During a partition with RF = 3 in each of two data centres:

  • LOCAL_QUORUM keeps both data centres working independently (AP across regions, consistent within each), with cross-DC divergence repaired later.

  • QUORUM or EACH_QUORUM across both data centres may fail on the cut-off side (CP behaviour).

  • ONE works almost anywhere but can read stale data.

Amazon DynamoDB

Feature Trade-off
Data replicated across three availability zones in a region Survives a zone failure
Eventually consistent reads (default) Lower cost and latency; may return slightly stale data
Strongly consistent reads (ConsistentRead=True) Returns the latest committed write in the region; costs more and can fail if the item’s leader replica is unavailable
Transactions (TransactWriteItems, TransactGetItems) ACID across up to 100 items, at roughly double the capacity cost
Conditional writes Optimistic concurrency with condition expressions and version attributes
Global tables Multi-region, multi-active replication, typically within about a second; conflicts resolved by last-writer-wins
# boto3 example (requires AWS credentials and a table)
import boto3

table = boto3.resource("dynamodb").Table("Accounts")
item = table.get_item(Key={"id": "acc-1"}, ConsistentRead=True)["Item"]
table.update_item(
    Key={"id": "acc-1"},
    UpdateExpression="SET balance = balance - :a, version = version + :one",
    ConditionExpression="version = :v AND balance >= :a",
    ExpressionAttributeValues={":a": 80, ":v": item["version"], ":one": 1},
)

For multi-region global tables, writes in two regions to the same item during a partition are both accepted and resolved by last-writer-wins: AP across regions. Strong consistency for critical items is achieved by writing them in one “home” region.

Common Patterns in Both

  • Per-operation choice: weak consistency for feeds and analytics, strong or conditional writes for money, inventory and uniqueness.

  • Single-region ownership for data that needs strict guarantees, with global replication for reads.

  • Idempotent writes and conditional updates to tolerate retries.

  • PACELC: both lean toward low latency in normal operation (local acknowledgement), with stronger options costing extra round trips.

Limitations

  • Last-writer-wins can lose concurrent updates and depends on accurate clocks (Cassandra).

  • Cross-partition or multi-region transactions are limited or costly.

  • Strong reads or quorum operations reduce availability during failures.

Interview Tip

Explain that both systems are tunable rather than fixed AP, give Cassandra’s consistency levels and lightweight transactions, DynamoDB’s eventually vs strongly consistent reads, transactions and global tables, and walk through partition behaviour. Choosing different settings for different data shows real-world understanding.

15

Why do idempotency, commutativity and convergence matter in CAP-driven designs?

In systems that favour availability (or that retry across unreliable networks), the same operation may be delivered more than once, operations may arrive in different orders at different replicas, and replicas may diverge during a partition. Idempotency makes repeated delivery harmless, commutativity makes order irrelevant, and convergence guarantees that replicas that have seen the same set of operations reach the same state. Together they let AP systems accept writes on both sides of a partition and still end up correct, which is the basis of CRDTs and of robust retry and messaging designs.

Read full answer

Applies toeventually consistent databases, multi-region and offline-first apps, message processing with at-least-once delivery, CRDT libraries (Automerge, Yjs, Riak data types, Redis Enterprise active-active), and payment and order APIs

Why the Properties Are Needed

Problem in distributed systems Property that solves it
Client retries after a timeout; message brokers redeliver Idempotency
Replicas receive updates in different orders Commutativity (and associativity)
Replicas diverge during a partition and later sync Convergence

Idempotency

An operation is idempotent if applying it twice has the same effect as applying it once.

  • Naturally idempotent: set status = shipped, add item X to a set, DELETE /orders/42.

  • Not idempotent: balance = balance - 80, counter += 1, POST /payments.

  • Making it idempotent: attach an idempotency key; the server records processed keys and returns the stored result for duplicates.

Commutativity

Operations commute if applying them in any order gives the same result: “add 5” and “add 3” commute; “set x = 5” and “set x = 3” do not. If replicas apply operations in different orders, only commutative operations guarantee equal results without coordination.

Convergence

Replicas converge if, once they have received the same updates, they are in the same state. Strong eventual consistency means convergence is guaranteed regardless of delivery order and duplication, which CRDTs provide.

Example: Grow-Only Counter CRDT

class GCounter:
    """Each replica increments only its own slot; merge takes the max per slot."""
    def __init__(self, replica_id):
        self.id, self.counts = replica_id, {}

    def increment(self, n=1):
        self.counts[self.id] = self.counts.get(self.id, 0) + n

    def merge(self, other):
        for rid, c in other.counts.items():
            self.counts[rid] = max(self.counts.get(rid, 0), c)

    def value(self):
        return sum(self.counts.values())

east, west = GCounter("east"), GCounter("west")
east.increment(3)          # during the partition
west.increment(2)

east.merge(west); west.merge(east)   # partition heals
east.merge(west)                     # duplicate merge is harmless (idempotent)
print(east.value(), west.value())    # 5 5

The merge (max per replica slot) is idempotent, commutative and associative, so replicas converge however many times and in whatever order they exchange state.

Applying the Ideas in Real Systems

  • APIs: require idempotency keys on POST /payments and POST /orders; retries return the first result.

  • Event consumers: with at-least-once delivery, store processed event IDs or make updates conditional on versions.

  • Data modelling: represent a shopping cart as a set with add and remove operations (an OR-Set CRDT) instead of overwriting the whole cart.

  • Counters: use per-replica counters (G-Counter, PN-Counter) instead of read-modify-write.

  • Collaborative editing: sequence CRDTs (Yjs, Automerge) let users edit offline and merge automatically.

Limits

  • Not every business rule fits: “balance must never go negative” or “username must be unique” needs coordination (a CP path), because those invariants cannot be preserved by merging concurrent updates.

  • CRDT metadata can grow (tombstones, per-replica state) and needs compaction.

  • Convergence guarantees equal states, not necessarily the state users intended; merge semantics must be chosen carefully.

Interview Tip

Define each property, link it to a concrete distributed-system problem (retries, reordering, divergence), and show a CRDT whose merge is idempotent and commutative. Noting that invariants like non-negative balances still require coordination shows a balanced understanding.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 15 CAP Theorem answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All System Design subjects