Consistency in Distributed Databases

ACID describes the guarantees of a single database. The moment your data lives on more than one node — replicas, shards, a cluster spanning regions — a new constraint takes over: the CAP theorem. This note covers what CAP actually says (it’s routinely misquoted), the latency trade-off it leaves out (PACELC), and the consistency models that live between “strong” and “eventual.”

The three properties

CAP is about three properties of a distributed data store:

• Consistency — every read sees the most recent write. All nodes agree on the current value. (Note: this is not the C in ACID — here it means all replicas return the same, latest data.)
• Availability — every request gets a non-error response, even if it might be stale.
• Partition tolerance — the system keeps working when the network drops or delays messages between nodes.

What CAP actually says

The popular phrasing — “pick 2 of 3” — is misleading. In any real distributed system, network partitions will happen; they’re a fact of networks, not a design option you can decline. So partition tolerance isn’t really on the menu.

The theorem is better stated as:

When a network partition occurs, you must choose between Consistency and Availability. You cannot have both.

That’s the whole trade-off. During a partition, a node that can’t reach its peers faces a binary choice: refuse the request to avoid serving stale or conflicting data (choose C), or answer anyway from what it has (choose A).

Partition: node A and node B can't talk. A write hits A.
A read hits B.

  CP  → B refuses / errors the read   (won't risk stale data)
  AP  → B answers with its old value  (stays up, may be stale)

CP vs AP in practice

The choice maps to what your data can’t tolerate:

• CP — sacrifice availability for correctness. During a partition, reject requests rather than serve wrong data. Right when stale reads are unacceptable: account balances, inventory counts, distributed locks. Examples: etcd, ZooKeeper, HBase, and (by config) most consensus-based stores.
• AP — sacrifice consistency for uptime. During a partition, keep answering and reconcile later. Right when being down is worse than being briefly stale: shopping carts, social feeds, DNS, telemetry. Examples: Cassandra, DynamoDB (in its default mode), Riak.

PACELC: the half CAP leaves out

CAP only describes behavior during a partition — which is rare. It says nothing about the trade-off the other 99% of the time. PACELC completes it:

If there’s a Partition, trade between Availability and Consistency; Else (normal operation), trade between Latency and Consistency.

The key insight is the “else”: even with a healthy network, keeping replicas strongly consistent costs latency — every write must reach and be acknowledged by other nodes before it’s confirmed. Relax consistency and you can answer from the nearest replica immediately.

PACELC classifications:
  PC/EC  → consistency always (e.g. strongly-consistent stores)
  PA/EL  → availability during partition, low latency otherwise (e.g. Cassandra, Dynamo)
  PC/EL  → consistent under partition, but fast (stale) reads normally

PACELC is the more useful lens day to day, because the latency-vs-consistency decision is one you make on every write path, partition or not.

Consistency models: it’s a spectrum

“Consistent vs eventual” is a false binary — there’s a useful middle. From strongest to weakest:

• Strong (linearizable) — a read always returns the latest committed write, as if there were one copy. Simplest to reason about, most expensive.
• Read-your-writes — you always see your own prior writes, though others’ may lag. (Why your comment appears instantly to you but not yet to everyone.)
• Monotonic reads — once you’ve seen a value, you never see an older one; no going backwards in time.
• Eventual — with no new writes, all replicas eventually converge. No timing guarantee, just the promise they won’t diverge forever.

Back to transactions

This is exactly where the ACID note’s cliffhanger led. A transaction spanning nodes can’t lean on single-node atomicity, so distributed systems reach for:

• Two-phase commit (2PC) — a coordinator asks all participants to prepare, then commit only if all agree. Gives atomicity across nodes, but blocks if the coordinator dies mid-round and ties availability to the slowest participant — a CP-leaning choice.
• Sagas — break the distributed transaction into a sequence of local transactions, each with a compensating action to undo it. No global lock, so it scales and stays available (AP-leaning), at the cost of only eventual consistency and having to write the compensations yourself.

The pattern to notice: distributed correctness is the same CAP dial. 2PC buys consistency by giving up availability; sagas buy availability by relaxing consistency.

At a glance

Concept	The one-liner
CAP	On a partition, choose Consistency or Availability — not both
Partition tolerance	Not optional; networks fail, so it’s a given
CP	Refuse requests to avoid stale/wrong data
AP	Stay up and answer, reconcile later
PACELC	Even without a partition, trade Latency vs Consistency
Strong consistency	Reads always see the latest write
Eventual consistency	Replicas converge given enough time
2PC	Atomic across nodes, but blocking (CP-leaning)
Saga	Local txns + compensations; available but eventually consistent (AP-leaning)

The model worth keeping: there is no free distributed consistency. CAP names the choice you’re forced into during a partition; PACELC names the one you’re making on every request otherwise. Good distributed design is being deliberate about that dial per operation — not picking a database and hoping.