Skip to content
System Design

Caching interview questions and answers

50 questions, 11 with code. About 5 h 4 min to read them all.

By TestLab · Updated

Plan for an interview
50 questions

Track what you know and what to revise.

Sign in to track progress

You can open a few full answers without signing in. Sign in to read all

1

What does caching mean in programming?

Caching is the practice of storing a temporary copy of data in a faster, closer storage layer so that repeated requests for that same data can be served without re-running the full, more expensive retrieval or computation.

Read full answer

Applies toall versions

The Problem It Solves

Before caching, every request would have to go all the way to its original source to get data. For example, a news feed could hit a database for every user refresh, or a product page could recompute the same pricing for thousands of views. These paths are often orders of magnitude slower: a memory read is roughly ~100 ns, an SSD read roughly ~100 µs, a same-datacenter round trip roughly ~0.5 ms, and a cross-region round trip roughly ~50-150 ms (all rough orders of magnitude). Caching cuts down on this cost by serving from a closer tier.

Where It Fits In A Real System

Caching can exist at nearly every layer depending on the goal:

  • Browser/HTTP cache: Uses headers like Cache-Control, ETag, max-age, and s-maxage to avoid re-downloading static assets.

  • CDN: Caches edgeable responses closer to users across regions.

  • Reverse-proxy cache: Sits in front of your services (e.g. Nginx) to cache safe, repeatable HTTP responses.

  • Application cache: In-process or shared caches held by your API to avoid calling other services repeatedly.

  • Distributed cache: Shared across many API instances (commonly used to hold serialized objects) to prevent every instance from re-fetching the same data.

  • Database query/buffer cache: Internal to the database to speed up repeated reads, but this is more of a database internals topic than an application caching design choice.

Trade-Offs

Nothing is free when introducing caching:

  • Latency vs consistency: You gain speed, but the copy can become stale the moment the source changes.

  • Complexity vs simplicity: You now have two sources of truth to reason about, with invalidation rules, keys, and expiry.

  • Cost vs savings: Caching can save database I/O and egress, but memory for a distributed cache costs money and adds operational effort.

  • Availability vs correctness: Serving stale data can keep a read path up during partial failures, but it can also serve incorrect data for sensitive flows.

For this reason, you would not use caching for highly volatile, per-user sensitive actions where the smallest bit of staleness is unacceptable (e.g. showing the exact remaining balance in a payment service). The choice must be based off of the workload.

Rules To Follow When Applying It

  • Invalidation is mandatory: A cache without a plan to remove or update its entries is almost always the mistake candidates make. Expiry alone is not an “ignore invalidation” strategy.

  • Use eviction intentionally: LRU, LFU, FIFO, or TTL all behave differently under pressure. Pick based off of access patterns, not hype.

  • Plan for cache-specific failures: Watch for cache stampedes/thundering herd, hot keys, and cache penetration. Protect them with techniques such as locking, request coalescing, early expiry with jitter, caching negative results, or bloom filters when appropriate.

  • Be strict with what goes in: Never put raw per-user secrets, one-time tokens, or anything that must be strictly fresh across all users onto a shared cache. Hit-ratio can be a useful metric, but chasing 100% is a red herring if it comes at the cost of correctness.

Interview Tips

  • What interviewers look for: The ability to explain the problem before the solution, state concrete trade-offs, and talk about invalidation without jumping straight to a tool.

  • Common follow-up: “When would you not cache this?”

  • Biggest mistake: Adding a cache to “make it faster” without defining consistency, keying, and invalidation for that exact data flow.

2

What are the main purposes of caching in software applications?

A cache stores a reusable copy of data closer to the requester so an application can avoid repeated computation, disk access, or network calls. Its main purposes are lower latency, less backend load, higher scalability and throughput, and sometimes graceful degradation when the origin is unavailable.

Read full answer

Applies togeneral caching concepts (all versions); Redis 7+ and Memcached 1.6.x for product details

Problems Caching Solves

Without caching, every request may repeat expensive work or retrieve the same unchanged data. For example, a product page might query a database, call services, and render the same response for thousands of users.

Caching helps by:

  • Reducing latency: A local memory lookup may take roughly 100 ns, while an SSD read is around 100 µs and a same-datacenter network round trip around 0.5 ms. These are rough orders of magnitude, not guarantees.

  • Reducing load: If 90% of requests hit the cache, the origin database receives roughly 10% of the original request volume for cacheable data.

  • Increasing throughput: Cached responses consume fewer database connections, CPU cycles, and downstream service calls.

  • Improving resilience: A stale cached response can sometimes serve traffic during a backend outage, although this is appropriate only when slightly stale data is acceptable.

  • Absorbing traffic spikes: A CDN or distributed cache can protect origins from sudden increases in demand.

The costs are additional memory or infrastructure, lower effective capacity, stale-data risk, invalidation complexity, and additional failure modes.

Where Caches Reside

Layer Examples Best suited for
Browser cache HTTP Cache-Control, ETag Repeated visits controlled by the client
CDN or reverse proxy Nginx, Envoy, cloud CDN Public assets and shared HTTP responses
Application cache In-process map or library Request-scoped or service-local reusable data
Distributed cache Redis, Memcached Shared data across application instances
Database cache Query cache, buffer pool, materialized views Repeated database work or disk access

For HTTP caching, max-age defines freshness, s-maxage can override it for shared caches, and private prevents a shared cache from storing an individualized response. ETag supports conditional requests but does not itself specify a freshness lifetime.

Caching Strategies

Strategy Behavior Trade-off
Cache-aside Application checks the cache and loads misses from the database Most control, but application must implement loading and invalidation
Read-through Cache loads missing data from the database automatically Simpler application code; ties the cache to its backing store
Write-through Writes update the cache and backing store synchronously Stronger read-after-write behavior, but higher write latency
Write-back Cache is updated immediately and the backing store is written later Fast writes, but data loss and consistency risks after failures
Write-around Write bypasses the cache and updates or invalidates it separately Useful for write-heavy data that is rarely read

Cache-aside is common because it keeps the persistence source authoritative. A cache should generally not be treated as the sole source of truth unless the system explicitly designs for that durability model.

Invalidation, Eviction, and Failure Handling

  • TTL expiry bounds staleness automatically, but a TTL alone does not tell the application when an entry changed.

  • Explicit invalidation deletes or updates cached entries after a source-of-truth change. Event-driven invalidation is often better than synchronously updating every cache layer, but it introduces propagation delay.

  • Eviction controls memory pressure. Common policies include LRU, LFU, FIFO, and TTL expiration. Implementations differ, so verify the behavior of the chosen product rather than assuming every cache uses LRU.

  • Cache stampede: Many requests may miss the same popular key simultaneously. Prevent this with request coalescing, a short-lived distributed lock, or early recomputation with randomized TTL jitter.

  • Cache penetration: Requests for nonexistent data repeatedly reach the origin. Cache negative results briefly and, at larger scale, consider a Bloom filter.

  • Hot keys: A single popular key can overload one shard or cache node. Add controlled expiration, local caching, key replication, or request coalescing.

  • Partial failure: Set connection and operation timeouts, use bounded retries with exponential backoff and jitter, and make the origin path degrade predictably when the cache is unavailable.

Redis and Memcached

Concern Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and others Primarily key-value items with byte values
Persistence Optional snapshots and append-only file persistence No persistence
Replication Built-in replication and Redis Cluster No built-in replication
Concurrency Command execution is primarily single-threaded; I/O threading does not make commands multi-threaded Uses worker threads to serve requests
Common use Caching plus queues, counters, locks, and data processing Large-scale, simple shared object caching

Best Practices and Pitfalls

  • Measure the hit ratio, miss cost, eviction rate, key cardinality, and tail latency; a cache with a low hit ratio may add cost without meaningful benefit.

  • Avoid caching passwords, one-time tokens, or security decisions. Sensitive per-user data on a shared cache requires strict key isolation, authorization, encryption where appropriate, and short lifetimes.

  • Do not cache correctness-critical values such as inventory balances or payment state unless stale reads are explicitly acceptable.

  • Use bounded TTLs, cache size limits, and backpressure so a traffic spike cannot exhaust memory.

  • Design invalidation before deployment. The common interview mistake is adding Redis without defining source-of-truth ownership, expiry behavior, or what happens when the cache is empty or unavailable.

3

What are cache hits and cache misses in caching?

A cache hit occurs when a request finds a usable cached value and can return it without querying the underlying source. A cache miss occurs when no valid value is available, so the system normally reads from the database or other authoritative source and may populate the cache.

Read full answer

Applies toall versions

Cache Hit and Cache Miss

A cache lookup uses a cache key, such as a product ID or URL, to determine whether a suitable entry exists.

Result What happened Typical next step
Cache hit A valid entry was found Return the cached value immediately
Cache miss The entry was absent, expired, invalidated, or otherwise unusable Read from the source and optionally cache the result

A miss is not an error. It is an expected event that indicates either a cold cache, an expired entry, an invalidation, or a workload accessing many distinct keys.

For example, when a request looks up product 123, the service first checks the cache. If product:123 exists and has not expired, that is a hit. If it does not exist, the miss causes a database read; the service can then store the result for future requests.

Where Caches Are Used

Caching can occur at several layers:

  • Browser cache: Stores responses on the client. Cache-Control, max-age, and ETag determine reuse.

  • CDN cache: Stores HTTP responses closer to users, reducing network latency and origin load.

  • Reverse proxy cache: Nginx or Envoy can cache selected responses before they reach the application.

  • Application cache: Stores frequently accessed objects in local or shared memory.

  • Distributed cache: Redis or Memcached allows multiple application instances to share cached data.

  • Database cache: Buffer pools and query caches reduce the cost of repeated reads.

A cache hit avoids expensive work such as a database query, remote API call, or computation. A local memory read is roughly nanoseconds to microseconds, while a same-datacenter service call may take hundreds of microseconds and a cross-region call tens to hundreds of milliseconds. These are rough orders of magnitude and depend on the workload.

Hit Ratio and Trade-offs

The hit ratio is:

hit ratio = cache hits / total cache lookups

For example, 950 hits and 50 misses produce a 95% hit ratio. A high ratio usually reduces source load and response latency, but it is not automatically better. Cached values consume memory, invalidation adds complexity, and stale data may violate business requirements.

The cost of a cache includes:

  • Memory and infrastructure cost: Especially for large objects or many keys.

  • Operational complexity: Monitoring, eviction, invalidation, and deployment coordination.

  • Consistency risk: The cache can temporarily disagree with the source of truth.

  • Failure risk: A cache outage may increase database load dramatically if the application has no fallback.

Do not cache data that is expensive to store but rarely reused, or data whose freshness requirements cannot tolerate stale results. Avoid placing per-user sensitive data in a shared cache unless keys, access controls, encryption, and isolation are designed carefully.

Invalidation and Consistency

A cache becomes stale when the underlying data changes. Common controls include:

  • TTL expiration: Entries expire after a defined lifetime.

  • Explicit invalidation: The writer deletes or updates the affected key after a successful database transaction.

  • Versioned keys: A changed version is included in the key, making old entries naturally irrelevant.

  • Event-driven invalidation: A change event reaches cache nodes and removes affected entries.

The normal pattern is cache-aside:

  1. Read from the cache.

  2. Return the value on a hit.

  3. On a miss, read from the authoritative source.

  4. Store the value in the cache.

  5. Invalidate or update it when the source changes.

Other strategies include read-through, where the cache loads missing data itself; write-through, where writes update the cache and source together; write-back, where the cache is updated first and persisted asynchronously; and write-around, where writes bypass the cache. Write-back improves write latency but risks data loss if the cache fails. Write-through simplifies read-after-write behavior but increases write latency.

Stampedes, Penetration, and Hot Keys

  • Cache stampede or thundering herd: Many concurrent requests miss the same popular key and overload the database. Use request coalescing, distributed locking, stale-while-revalidate, or early expiration with jitter.

  • Cache penetration: Repeated requests for nonexistent keys bypass the cache. Cache negative results briefly or use a Bloom filter where appropriate.

  • Hot keys: A single key receives excessive traffic. Use replicas, sharding, key splitting, or request collapsing.

  • Eviction: When memory is full, policies such as LRU, LFU, FIFO, or TTL-based expiration decide which entries to remove. LRU favors recent access; LFU favors popular entries.

Practical Design Guidance

Choose a cache because the same data is read repeatedly and the measured latency or source load justifies the added complexity. Define the cache key, TTL, eviction policy, invalidation owner, and stale-data tolerance before implementation. Monitor hit ratio, miss latency, source load, evictions, and error rates together.

Interviewers commonly look for a trade-off discussion rather than the definition alone. The most common mistake is adding a cache without specifying how entries expire, how updates are propagated, and what happens when the cache is unavailable.

4

How does cache size affect system performance?

Cache size affects performance by determining whether the active working set remains resident, which influences the hit ratio, eviction rate, memory cost, and load on the source system. A larger cache helps only when it retains entries that will be reused.

Read full answer

Applies togeneral cache-sizing principles; Redis 8.x and Memcached 1.6.x

How Size Changes Performance

The hit ratio is cache hits / (cache hits + cache misses).

  • Too small: Frequently reused entries are evicted before reuse, causing cache churn, repeated expensive reads, and additional database or upstream-API traffic.

  • Appropriately sized: The active working set remains resident, producing a high hit ratio and predictable latency.

  • Too large: Memory and operational costs rise. An application cache may create heap or allocator pressure, while a distributed cache retains more data and may increase snapshot or replication overhead.

As rough orders of magnitude, an in-process lookup may take around 100 ns, a same-datacenter Redis request around 0.5 ms, and a database query commonly takes milliseconds. Cache capacity has no effect after an entry expires or is invalidated, but insufficient capacity can evict an entry before its natural TTL.

Effects by Cache Layer

Cache layer Effect of insufficient capacity Cost or risk of excessive capacity
Application cache Repeated computation or database reads Higher heap usage and possible GC pressure
Redis or Memcached More source-system traffic More memory and greater snapshot or replication cost
CDN More origin requests Higher edge storage cost
Reverse proxy More requests reach application servers More memory per proxy
Browser and HTTP cache More network requests because entries are absent Browser-managed memory or disk use and browser-specific eviction
Database query and buffer cache More query work or physical disk reads Less memory available for other database structures or the host

For HTTP caching, Cache-Control: max-age controls freshness, ETag supports conditional requests, and s-maxage overrides max-age in shared caches such as CDNs.

Sizing the Cache

  • Working set: Measure the keys and values actively reused within a relevant interval.

  • Miss cost: A larger cache is more valuable when misses trigger expensive queries or remote calls.

  • Entry overhead: Include per-key metadata, allocator overhead, and serialized value size.

  • Traffic skew: A few hot keys may provide most hits, so capacity for the long tail may add little value.

  • Headroom: Allow for traffic spikes, deployments, and larger values.

  • Eviction policy: LRU favors recent access, LFU favors frequent access, FIFO ignores access frequency, and TTL expiration is based on entry age rather than access time.

Monitor hit ratio, evictions, memory usage, source latency, and source load. Increase capacity when evictions repeatedly remove entries that would soon be reused, not merely because configured memory remains unused.

Strategies and Consistency

Strategy Behavior Typical fit
Cache-aside Application checks cache and loads the source on a miss Read-heavy services with explicit invalidation
Read-through Cache loads the source when it encounters a miss Systems that want source access hidden behind the cache
Write-through Cache and source are updated synchronously Read-after-write consistency at the cost of write latency
Write-back Cache acknowledges the write and persists it asynchronously High write rates where a bounded data-loss window is acceptable
Write-around Write bypasses the cache and goes to the source Avoiding cache pollution from write-only data, at the risk of a subsequent miss

For cache-aside, define the database as the source of truth, commit before invalidating or updating the cache, and use versioned values to prevent an older concurrent fill from becoming current. TTLs bound staleness but do not provide strong consistency.

Failure Modes and Trade-offs

  • Stampede: Coalesce concurrent misses, use short locks, or refresh slightly before expiry with randomized jitter.

  • Hot keys: Capacity alone cannot help one dominant key; consider request collapsing, read replicas, or partitioning.

  • Penetration: Cache negative results briefly or use a Bloom filter when appropriate.

  • Sensitive data: Do not place per-user or authorization-sensitive data in a shared cache without explicit key isolation and access controls.

  • Availability: Treat non-critical cache failure as graceful degradation; use timeouts, circuit breakers, bounded fallback concurrency, and source reads.

  • When not to cache: Avoid caching cheap-to-recompute data, low-reuse workloads, or data requiring strict immediate consistency unless the source is too expensive to query repeatedly.

Redis Versus Memcached

Aspect Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and other structures Simple opaque string values keyed by strings
Persistence Optional RDB snapshots and/or append-only file persistence No built-in persistence
Replication Built-in asynchronous replication, including within Redis Cluster No built-in replication; clients commonly shard instances
Threading Primarily single-threaded command execution, with optional I/O threads Multithreaded server

Neither is automatically faster for every workload. Choose based on required data structures, durability, topology, and operational simplicity, then size both around the measured working set rather than attempting to hold every source record.

5

How does caching reduce data retrieval latency?

Caching reduces retrieval latency by storing frequently requested data closer to the requester, allowing a cache hit to return without reading the original database or calling an expensive remote service. The benefit comes with trade-offs: stale data, invalidation complexity, extra infrastructure, and potentially higher costs.

Read full answer

Applies toall versions

How It Works

A typical request flow is:

Client → CDN or reverse proxy → application → in-process or distributed cache → database

On a cache miss, the application retrieves the data from the database, stores it in the cache, and returns it. On later requests, the cached value avoids the database round trip and associated query, computation, or service call.

The following are rough orders of magnitude to illustrate the difference:

  • Local memory access: about 100 ns

  • SSD read: about 100 µs

  • Same-datacenter network round trip: about 0.5 ms

  • Cross-region round trip: about 50–150 ms

The improvement depends on where the cache sits. An in-process cache avoids network overhead, while Redis or Memcached usually saves database work but still requires a network request. A CDN can reduce latency substantially by serving content from an edge location near the user.

The hit ratio is the percentage of requests served from cache:

hit ratio = cache hits / total cache requests

A 90% hit ratio avoids 90% of origin reads, but the remaining misses can still create latency spikes. Cache effectiveness therefore depends on request frequency, read cost, and acceptable staleness—not just the raw hit ratio.

Where Caches Are Placed

  • Browser cache: Stores resources on the user’s device. Cache-Control, max-age, ETag, and conditional requests control reuse.

  • HTTP and reverse-proxy cache: Caches complete HTTP responses. s-maxage can allow shared caches to retain an object longer than an individual browser.

  • CDN: Caches static or cacheable dynamic content geographically close to users and offloads the origin.

  • Application cache: Holds computed results, configuration, or frequently accessed objects inside the service process.

  • Distributed cache: Provides a shared cache across instances.

  • Database cache: PostgreSQL and other databases use shared buffers, query plans, and operating-system page caches to reduce physical disk reads.

Aspect Redis Memcached
Data structures Rich: strings, hashes, lists, sets, sorted sets, streams, and more. Primarily simple string key-value pairs.
Persistence Optional persistence (RDB snapshots and/or AOF) can survive restarts, but is not automatic by default in every deployment. In-memory only by design. No built-in persistence; data is lost on restart.
Replication Supports replication with a primary/replica model in typical deployments. No built-in replication mechanism. High availability must be handled externally.
Threading Primarily single-threaded for command execution (with some threading used for I/O/pipelining features), which avoids lock contention but can become a bottleneck for certain CPU-heavy operations. Multithreaded by design, which can scale well across multiple CPU cores for simple key-value operations.
Best fit When you need richer data structures, optional persistence, or replication for a shared cache. When you need a simple, fast, memory-focused key-value cache with multithreaded serving and no need for built-in persistence.

Neither removes the need for invalidation, capacity planning, or protection from hot keys.

Invalidation and Consistency

Caches must be invalidated when source data changes. Common strategies include:

  • Cache-aside: Application reads from cache and loads from the database on a miss. This is widely used because the application controls population and invalidation.

  • Read-through: The cache loads missing data from the database automatically. It is convenient but adds library-specific behavior.

  • Write-through: Writes update the cache and database together. Reads are fast, but write latency and write availability depend on both operations.

  • Write-back: Writes update the cache first and persist asynchronously. It improves write throughput but creates a window where acknowledged data exists only in memory.

  • Write-around: New writes bypass the cache so only data requested later is loaded. It helps when existing cache entries are unlikely to be reused.

No strategy makes a distributed cache and database atomically consistent without additional coordination. TTL-based expiration bounds staleness, while explicit invalidation provides faster updates but can fail or arrive late. A practical design often combines short TTLs with event-driven invalidation.

Failure Modes and Trade-offs

  • Cache stampede: Many requests miss the same expired key and load it simultaneously. Prevent this with request coalescing, short-lived locks, staggered refreshes, or early recomputation with jitter.

  • Cache penetration: Requests repeatedly ask for nonexistent data. Cache negative results briefly, validate inputs, or use a Bloom filter.

  • Hot keys: One popular key overwhelms a cache shard or database. Replicate read-heavy keys, partition them intentionally, or use local caching.

  • Eviction pressure: A bounded cache must discard entries. LRU favors recent access, LFU favors popular items, FIFO ignores usage patterns, and TTL removes expired entries.

  • Shared-cache security: Do not store per-user sensitive responses under shared keys. Include authorization context, avoid caching responses incorrectly, and encrypt data where required.

When to Use Caching

Caching is most valuable when data is read frequently, retrieval is expensive, and some staleness is acceptable. A product-detail catalog, CDN-hosted images, or a news-feed response are common examples. It is less useful for constantly changing records, low-traffic data, or operations where the added complexity exceeds the latency benefit.

Common interview mistakes are adding a cache without an invalidation plan or treating a high hit ratio as automatically cost-effective. Strong answers identify the workload, measure hit ratio and origin load, define acceptable staleness, and include failure behavior such as timeouts, bounded retries, and graceful fallback to the source of truth.

6

How do local and distributed caches differ in system design?

A local cache stores data inside an application instance or process, while a distributed cache stores data in a shared service accessed over the network. Local caching minimizes access latency; distributed caching shares cached results across instances but adds network latency, coordination, and operational complexity.

Read full answer

Applies toGeneral system-design concepts across versions; Redis 8.x and Memcached 1.6.x for tool comparisons

Core Differences

Aspect Local cache Distributed cache
Scope Private to one process or application instance Shared by multiple application instances
Typical storage In-process memory or local storage Redis, Memcached, or another network-accessible service
Access cost Very low; an in-memory read is roughly 100 ns Higher; a same-datacenter round trip is roughly 0.5 ms
Capacity Memory must be added to every application instance Capacity can be added to the cache cluster independently
Failure Usually affects only the owning instance A shared outage can affect every instance unless they degrade gracefully
Consistency Every instance may hold a different stale copy Clients normally share one logical view, subject to replication consistency
Restart behavior In-memory entries are usually lost Application restarts do not delete entries; durability depends on cache configuration

Availability is the ability to serve requests, reliability is consistent operation over time, and durability is the survival of acknowledged data. Cache replication primarily improves availability and reliability; persistence improves durability.

Where the Caches Fit

  • Browser and HTTP client caches: Local to the client; Cache-Control, ETag, max-age, and s-maxage control freshness and reuse.

  • CDN caches: Geographically distributed edge caches that reduce origin traffic and user latency.

  • Reverse-proxy caches: May run per node or as a shared cluster, depending on deployment.

  • Application caches: Often local for low latency, or distributed when instances must reuse the same result.

  • Database caches: Buffer, page, and internal query-plan caches are normally local to each database node.

  • Distributed caches: Shared services such as Redis or Memcached used for sessions, rate limits, and expensive shared lookups.

Invalidation and Write Strategies

The authoritative database should remain the source of truth. Local invalidation is harder because copies exist in every instance; use short TTLs, versioned keys, or invalidation events. Distributed invalidation is more centralized, although replicas may observe updates eventually.

  • Cache-aside: The application checks the cache, loads a miss from the database, and populates the cache.

  • Read-through: The cache invokes a loader on a miss, hiding cache and storage access from the application.

  • Write-through: The cache writes synchronously to the database before acknowledging the operation, at the cost of higher write latency.

  • Write-back: The cache writes first and persists later, reducing write latency but risking data loss if the cache fails before persistence.

  • Write-around: A write bypasses the cache and goes to the database, suitable for write-once data that will be read later.

Failure and Contention Scenarios

  • Cache stampede: When a hot key expires, many requests can reload it simultaneously. Use request coalescing or single-flight refreshes; distributed locks need timeouts. Early refresh with randomized jitter reduces synchronized expiry.

  • Hot key: A single key receives disproportionate traffic. Neither Redis Cluster nor Memcached automatically splits one logical key across nodes, so use key replication, request coalescing, or application-level partitioning.

  • Cache penetration: Repeated requests for nonexistent data bypass the cache and load the database. Cache negative results briefly, validate inputs, or use a Bloom filter for a large predictable key space.

  • Cache outage: Fall back to the database with strict timeouts, concurrency limits, and a circuit breaker; otherwise cache failure can become database failure.

Eviction and Tool Choices

  • LRU: Appropriate for changing access patterns.

  • LFU: Appropriate when a small subset of keys dominates traffic.

  • FIFO: Simple, but it favors insertion order rather than popularity.

  • TTL: Controls freshness rather than capacity and should be combined with a capacity policy.

Capability Redis 8.x Memcached 1.6.x
Data model Strings, hashes, lists, sets, sorted sets, streams, and other structures Primarily opaque key-value entries
Persistence Optional RDB snapshots and append-only files Memory-only; no built-in persistence
Replication and sharding Primary-replica replication and Redis Cluster hash-slot sharding No built-in replication or server-side sharding; clients normally hash keys
Concurrency Primarily single-threaded command execution, with I/O and background work using threads Multi-threaded server

Monitor hit ratio, evictions, miss latency, and backend load. Hit ratio is hits / (hits + misses); at a 90% hit ratio, 10% of lookups still miss. Caching is worthwhile only when avoided backend work and latency exceed the memory, invalidation, and operational costs.

Choosing and Operating the Cache

Use local caching for instance-independent data, immutable configuration, templates, and computations where slight staleness is acceptable. Use distributed caching when instances must share results or avoid repeating expensive work after every deployment.

A two-level design often works well: a small local cache serves the hottest data, while a distributed cache reduces repeated database access across instances. Do not place sensitive per-user data in a shared cache without carefully designed keys, isolation, access controls, encryption, and expiry.

For an e-commerce catalog, a distributed cache can retain product descriptions while short local TTLs limit staleness. The common interview mistake is adding a cache without specifying the source of truth, invalidation policy, TTL, eviction policy, stampede protection, and fallback behavior.

7

What is cache eviction and which strategies are common?

Cache eviction removes entries when a cache reaches capacity or when they expire, bounding resource use while retaining data likely to be requested again. The policy must balance hit ratio against memory, CPU, freshness, and operational complexity.

Read full answer

Applies togeneral cache-eviction concepts across all cache layers; Redis 4.0+ with the feature-specific versions noted below; Memcached 1.6+

Why Eviction Is Necessary

Without a policy, a cache fills; useful writes may fail and reads fall through to the database. Eviction trades hit ratio and freshness against memory, CPU, and complexity. As rough orders of magnitude, an in-process hit may take about 100 ns–1 µs, while a same-datacenter network cache lookup may take about 0.5 ms. Workload matters: news becomes irrelevant quickly, while stable configuration may remain useful longer.

Common Eviction Strategies

Strategy Selection Best fit Main limitation
LRU Removes the least-recently used entry General workloads Large hot entries can crowd out many smaller useful entries
LFU Removes the least-frequently used entry Stable, skewed hot-key workloads New entries need time to establish their frequency
FIFO Removes the oldest entry regardless of use Simple queues and inexpensive policies Ignores actual popularity
TTL Removes entries after their validity period Data with a known validity window Simultaneous expirations can cause reload bursts
Random Removes an arbitrary eligible entry Fallback when metadata and CPU are constrained Can evict hot entries without regard to popularity

TTL is usually combined with a capacity policy: expiration controls freshness, while capacity eviction bounds resource consumption. TTL alone is sufficient only when the set of simultaneously live entries is inherently bounded.

Redis 4.0+ offers approximate LRU and LFU policies. The volatile-lru and volatile-lfu policies consider only keys with an expiry, while allkeys-lru and allkeys-lfu consider all keys. Memcached uses LRU within its slab classes and does not offer LFU.

Redis and Memcached also differ beyond eviction:

Capability Redis Memcached
Data structures Strings, hashes, lists, sets, sorted sets; Streams from Redis 5.0+ Opaque key/value byte strings plus atomic counters
Persistence Optional RDB snapshots and AOF None
Replication Built-in asynchronous replication; Sentinel for failover and Cluster for sharding No built-in replication; clients must coordinate it
Threading Optional network I/O threads from Redis 6.0+; core command execution remains primarily on the main thread Multithreaded workers with dedicated I/O-thread support

Eviction Versus Invalidation

These solve different problems:

  • Eviction removes an entry because capacity or expiry requires it.

  • Invalidation removes an entry because its source data changed.

  • Refresh replaces an outdated value that is still popular.

For example, when a product price changes, the application should invalidate or update its cache entry. An unrelated entry may instead be removed through capacity eviction. An entry can therefore be valid but evicted; eviction is not a substitute for invalidation.

Interaction With Caching Strategies

All common population strategies still require rules for bounded storage:

  • Cache-aside: The application loads missing data from the database. It suits general read-heavy services and keeps write-path complexity low, but misses create database load.

  • Read-through: The cache loads missing data itself. It simplifies application reads but couples the cache layer to the source and its failure modes.

  • Write-through: The cache and primary store are updated synchronously. It favors immediate cache consistency but increases write latency.

  • Write-back: Writes remain in the cache until a later flush. It reduces write latency, but evicting a dirty entry without first persisting it can lose data.

  • Write-around: Writes bypass the cache. It avoids polluting the cache with values not immediately read, but subsequent reads may miss.

Cache-aside is common in application and distributed caches. A miss, expiry, or capacity eviction can each trigger another source request, so stampede protection remains necessary.

Common Failure Modes and Best Practices

  • Cache stampede: Many requests load the same expired entry concurrently. Use request coalescing, a short distributed lock, or early refresh with jitter.

  • Hot keys: One key receives excessive traffic. Use replicas, request collapsing, or locally cached immutable data.

  • Cache penetration: Requests repeatedly seek nonexistent keys. Cache negative results briefly and consider a Bloom filter for a large key space.

  • Sensitive data: Do not place per-user or authorization-sensitive data in a shared cache without explicit partitioning, key isolation, and access controls.

  • Unbounded TTLs: Use both expiry and a capacity limit so temporary traffic cannot create permanent memory pressure.

  • Poor key design: Include every dimension affecting the response, such as tenant, locale, authorization context, and API version.

  • Invalidation gaps: Define how updates propagate through application, CDN, proxy, and browser caches.

Measure hit ratio, eviction rate, source-load duration, stale-read rate, and database fallback traffic. Choose the policy from the workload rather than assuming LRU or LFU is universally best.

8

How does a cache key identify and retrieve cached data?

A cache key is a stable identifier that maps to one cache entry containing a value and metadata such as its TTL or size. The application constructs the key, the cache locates the matching entry, and the application uses the cached value or obtains it from the authoritative source and stores it for reuse.

Read full answer

Applies togeneral cache-key semantics across cache systems; Redis 8.x and Memcached 1.6.x for the product-specific comparison

How Cache Lookup Works

A typical cache-aside lookup is:

  1. The application normalizes the request into a deterministic key.

  2. The cache hashes the key to locate a bucket, slot, or shard.

  3. It compares the complete stored key, so a hash collision does not return the wrong entry.

  4. On a hit, the cache returns the value and associated metadata.

  5. On a miss, the application reads the authoritative source, stores the result with a TTL, and returns it.

For example, a product service might use catalog:v3:tenant-42:product-123. An in-process lookup is roughly 100 ns; a same-datacenter distributed-cache request is often about 0.5–2 ms; and a database query may take several milliseconds. These are workload-dependent orders of magnitude.

Designing a Cache Key

A good key should be:

  • Deterministic: the same logical request produces the same key.

  • Complete: it includes every input that can change the response.

  • Canonical: it normalizes parameter order, encoding, and casing only when that does not change semantics.

  • Scoped: it distinguishes tenants, locales, regions, permissions, and API versions where relevant.

  • Bounded: it avoids unnecessary high cardinality and arbitrary unbounded input.

Use an unambiguous serialization format. Joining arbitrary values with : can cause collisions if a value contains :; length-prefixed fields, canonical JSON, or a digest of a canonical representation avoid that ambiguity.

Do not put secrets or raw authorization tokens in keys because keys may be logged or exposed through monitoring. Avoid sensitive per-user data in shared caches unless isolation, authorization, and encryption are enforced. A user ID in a key identifies a cache partition; it does not replace an authorization check.

Cache Read and Write Strategies

  • Cache-aside: The application reads and populates the cache. It is simple and common, but misses and cache/database consistency races must be handled by the application.

  • Read-through: The cache fetches missing data from the source. This centralizes loading but ties the cache implementation to the source.

  • Write-through: A write updates the cache synchronously, often with the source. It reduces stale misses but increases write latency.

  • Write-back: The cache acknowledges a write and persists it later. It reduces write latency and source load but risks lost writes and requires careful recovery.

  • Write-around: A write bypasses the cache. It prevents rarely read values from evicting useful entries, but the next read will usually miss.

Eviction, Invalidation, and Consistency

TTL controls entry age, while eviction chooses what to remove when capacity is exhausted: LRU favors recently used entries, LFU favors frequently used entries, and FIFO removes entries in insertion order. The appropriate policy depends on access patterns.

Invalidation options include explicit deletion, versioned keys, and event-driven updates. For cache-aside, update the database and then delete the affected key. A race can still repopulate stale data, so short TTLs, versioned keys, locking, or transactional updates may be necessary. Do not invalidate unrelated entries when one targeted key can be removed.

Keys Across Caching Layers

Layer Typical key or identifier Important behavior
Browser Method, URI, selected headers, and response metadata Uses HTTP freshness and validators
CDN or reverse proxy URI, host, query, and headers named by Vary Shared responses must vary correctly by tenant and representation
Application Namespaced object identifier Usually local to one process
Redis Binary-safe string key Can select a cluster slot through hashing
Memcached Restricted key of at most 250 bytes No whitespace or control characters; values may be binary
Database buffer pool Physical page or block identifier Managed internally by the database

For HTTP, an ETag is a validator rather than necessarily the cache key. Cache-Control: max-age controls freshness, while s-maxage lets a shared cache retain a response longer than client caches.

Concern Redis Memcached
Data types Strings, hashes, lists, sets, sorted sets, and streams Primarily opaque values
Persistence Optional RDB snapshots and AOF Memory-only
Replication Built-in replication and cluster mechanisms No equivalent built-in replicated-cache model
Concurrency Event-driven I/O with mainly single-threaded command execution Multi-threaded request processing
Best fit Shared state, structured data, counters, locks, and streams Simple low-latency object caching

Trade-offs, Failure Modes, and When Not to Cache

The effective benefit depends on the hit ratio, calculated as hits divided by total lookups, and on the cost saved per hit. Measure hit ratios by key class because popular and rarely accessed entries behave very differently.

  • Cache stampede: Many requests miss one hot key together. Use request coalescing, a short-lived lock with failure fallback, stale-while-revalidate, or early refresh with randomized jitter.

  • Cache penetration: Repeated requests target nonexistent data. Cache negative results briefly or use a Bloom filter for a large keyspace.

  • Hot keys: One key exceeds a shard’s capacity or request rate. Route reads to replicas where supported, split the value, or distribute it across subkeys.

  • Unbounded keys: Random identifiers consume memory. Normalize keys, cap cardinality, and enforce length limits.

  • Stale data: Define acceptable staleness and combine TTLs with targeted or versioned invalidation.

  • Cache outage: Use timeouts, circuit breakers, and bulkheads so cache failure degrades performance rather than necessarily taking down the authoritative service.

Do not add a cache when reuse is low, the source lookup is already cheap, or the application requires immediate strict consistency. A distributed cache adds memory cost, a network hop, invalidation complexity, and operational failure modes, so its saved source work must exceed those costs.

9

Why do caches need expiration and how is expiration managed?

Cache expiration limits how long a cached value may be served without being refreshed or revalidated. It provides bounded staleness, reclaims storage, and acts as a safety net when explicit invalidation fails, but it trades freshness for cache-hit rate and infrastructure cost.

Read full answer

Applies toall versions

Why Expiration Matters

Caches can contain data that changes after it was stored. Without expiration, a product price, permission, or news feed could remain stale indefinitely. Expiration also prevents expired entries from unnecessarily consuming memory and reduces the impact of missed invalidation messages.

Expiration is especially important when:

  • Freshness requirements vary: a search result might tolerate 30 seconds, while account permissions may require immediate revocation.

  • Invalidation is unreliable: a process crash or network partition can prevent a delete or event from reaching every cache node.

  • Memory is finite: expired values eventually become reclaimable, although eviction handles capacity pressure separately.

  • Traffic spikes are expected: expiration limits how long an outdated value can be amplified across many requests.

Expiration does not guarantee that data is perfectly fresh. It only bounds staleness according to the configured TTL.

Expiration Versus Eviction

These solve different problems:

  • Expiration removes or makes an item unavailable after its TTL. It primarily controls freshness and limits stale-data lifetime.

  • Eviction removes an item when capacity is needed. Common policies include LRU, LFU, and FIFO.

For example, Redis can expire keys by TTL while also evicting keys under memory pressure. A cache with no TTL can still evict entries, but it may serve stale data for much longer.

Managing Expiration

The expiration policy should follow the data’s freshness requirement:

Policy Behavior Typical use
Absolute TTL Expires a fixed duration after insertion or update Product catalogs, configuration
Sliding TTL TTL is refreshed when an item is accessed Frequently changing but tolerable data
Short TTL Refreshes frequently, reducing staleness Dynamic feeds or prices
Long TTL plus invalidation Explicit deletion provides immediate freshness; TTL is the fallback Data that must normally be current
Stale-while-revalidate Serves stale data briefly while refreshing in the background Read-heavy systems where a little staleness is acceptable

Sliding TTL requires care: a popular key might never expire and monopolize cache space.

Typical management practices include:

  • Store an expiry timestamp or TTL with each cache entry.

  • Choose TTLs from the acceptable staleness window, update frequency, and expected cache-hit ratio.

  • Add TTL jitter so many keys created together do not expire simultaneously.

  • Use shorter TTLs for negative results and permission-sensitive data.

  • Delete or version cache keys after database commits rather than depending only on expiration.

  • Monitor hit ratio, expiration rate, stale responses, origin load, and eviction rate.

Caching Layers and Strategies

Expiration is applied differently across layers:

  • Browser cache: Cache-Control: max-age=... controls browser freshness; ETag supports conditional revalidation.

  • Shared caches and CDNs: s-maxage controls shared-cache freshness and takes precedence over max-age for shared caches.

  • Application cache: the service chooses TTLs based on business rules.

  • Distributed cache: Redis or Memcached stores expiration metadata with entries.

Cache-writing strategies also affect expiration:

  • Cache-aside: On a miss, the application reads the database and stores the result with a TTL. This is common, but stale values can be repopulated after an invalidation race.

  • Read-through: The cache loads missing data from the database. TTL handling stays inside the cache layer.

  • Write-through: Writes update the cache synchronously. The TTL should be reset on each successful update.

  • Write-back: Writes remain in cache until later persistence. This improves write latency but increases stale-data exposure and potential data loss.

  • Write-around: Writes bypass the cache, and later reads populate it. This suits write-heavy workloads where values are rarely reused.

Failure Modes and Tool Trade-offs

A short TTL can overload the database with cache misses; a long TTL increases staleness. Explicit invalidation gives faster freshness but adds messaging and operational complexity.

Additional risks include:

  • Cache stampede: Many requests miss one hot key simultaneously. Use request coalescing, a short distributed lock, early refresh, or jittered expiration.

  • Cache penetration: Repeated requests for nonexistent keys cause repeated origin lookups. Cache negative results briefly and consider a Bloom filter for large key spaces.

  • Hot keys: A single popular key can overload one shard or server. Use key replication, request coalescing, or deliberate partitioning.

  • Sensitive data: Do not place personalized or permission-sensitive values in a shared cache unless access is explicitly isolated and responses are marked private.

Redis supports rich data structures, optional persistence, and built-in replication. Memcached provides a simpler key-value model, no persistence, and multithreaded request handling. Both support TTL-based expiration, so expiration behavior depends more on policy and deployment design than on the product.

Practical Guidance

Use explicit invalidation for correctness-sensitive changes and TTL as a bounded fallback. For an e-commerce catalog, a publish event can invalidate the product key immediately while a five-minute TTL limits stale data if that event is lost. The main mistake is choosing a TTL without estimating the origin load it creates or defining how stale data is handled during refresh.

10

How does cache invalidation prevent stale data in distributed systems?

Cache invalidation removes or refreshes stale cached copies when the authoritative source changes. TTLs and version checks can bound staleness, but they do not replace a deliberate invalidation policy when updates must be reflected promptly.

Read full answer

Applies toall versions of the general invalidation model; Redis 8.x and Memcached 1.6.x for tool-specific details

Why Stale Data Appears

A request may traverse several cached layers rather than the source of truth:

Client → CDN → reverse proxy → application cache → Redis → database

A price update in the database may leave an application cache serving the old price. Caching reduces latency and origin load, but introduces a consistency window. Local memory access is roughly 100 ns and a same-datacenter cache request roughly 0.5 ms, both far below many database queries.

Invalidation Mechanisms

  • TTL expiry: Removes an entry after its configured lifetime. It bounds age-based staleness only while expiry is enforced; it does not immediately detect a source change.

  • Explicit invalidation: After a source write commits, delete or update affected keys. It is fast but couples the writer to the cache and can fail.

  • Event-driven invalidation: CDC or a broker publishes invalidation events. This scales across services but adds delay, duplicates, ordering issues, and at-least-once delivery concerns.

  • Versioned entries: Put a source version in the key or metadata; readers reject older versions, avoiding some invalidation races.

  • HTTP revalidation: ETag lets a client or shared cache validate its copy and receive 304 Not Modified when unchanged.

  • Early refresh: Refresh popular entries before expiry, with jitter to avoid synchronized expiry.

Eviction is not invalidation: LRU, LFU, and FIFO control capacity pressure; they do not guarantee freshness. TTL controls age.

Cache Strategies and Consistency

Strategy Behavior Best fit and cost
Cache-aside Application loads the source on a miss and invalidates after writes Flexible default; adds miss latency and invalidation logic
Read-through Cache loads missing data from the source Managed access; couples the cache to a loader
Write-through Synchronously writes cache and source Read-heavy data; higher write latency and no atomicity unless coordinated
Write-back Writes cache, then persists asynchronously High write throughput; allows a data-loss window
Write-around Writes only to the source Write-once/read-later data; the later first read misses

Cache-aside is common. Strong read-after-write requirements need source-level coordination, not a cache assumption.

Invalidating Across Cache Layers

  • Browser/HTTP: Cache-Control: max-age controls browser freshness; s-maxage applies to shared caches. Use no-store when a response must not be stored.

  • CDN: Purge changed URLs. Versioned URLs work only after references switch; old URLs can still be requested and serve stale content.

  • Reverse proxy: Nginx or Envoy may require a purge API for application-specific invalidation.

  • Application cache: Per-process or per-user entries need cross-instance invalidation through events, broadcasts, or versioned keys.

  • Distributed cache: Redis or Memcached support key deletion or expiry, but invalidation can fail or race with concurrent fills.

  • Database cache: Buffer-pool pages are managed by the database; application code must not invalidate them manually.

Do not expose one user’s personalized or sensitive response through a shared cache. Scope keys and use HTTP private or no-store where shared storage is unsafe.

Hit Ratio and Distributed Cache Choice

Hit ratio is hits / (hits + misses). Estimate it from reuse frequency and key skew, then evaluate hit ratio, miss cost, and origin load together; a high ratio only pays off when avoided misses are expensive enough.

Aspect Redis 8.x Memcached 1.6.x
Data model Strings, hashes, lists, sets, sorted sets, streams Primarily scalar key-value entries
Persistence Optional RDB snapshots and AOF None
Replication Built-in primary-replica replication None
Threading Main-thread command execution, plus background and optional I/O threads Multithreaded server

Both are eviction-based memory caches, not systems of record. Redis’s persistence options do not change that architectural role.

Failure Modes and When Not to Cache

  • Stampede: Coalesce concurrent misses, use a bounded lock, serve stale while refreshing, or refresh early with jitter.

  • Penetration: Cache negative results briefly or use a Bloom filter; a Bloom filter can produce false positives.

  • Hot keys: Use local replicas, key splitting, request coalescing, or workload-specific partitioning.

  • Invalidation failure: Commit the source first, then invalidate the cache and retry through an outbox or event stream. Versioned keys prevent an old concurrent read from repopulating a newer entry; TTLs bound the failure window.

Do not cache rarely reused or write-heavy data, strongly read-after-write data, or sensitive per-user data in a shared cache. Payment authorization or inventory reservation may need transactional coordination, not a TTL. Interviewers look for a named invalidation owner, consistency target, and failure handling; adding Redis and a TTL without defining invalidation is the common mistake.

11

What are the steps to implement a basic cache system?

A basic cache system stores reusable data closer to consumers so it can avoid repeated work against a slower origin such as a database. A practical default is cache-aside, but cache keys, expiry, invalidation, consistency, and failure behavior must be designed with it.

Read full answer

Applies toall cache implementations; Python 3.12 for the code example

Implementation Steps

  1. Select cacheable work: Favor frequent, expensive reads whose results tolerate a defined staleness window. Do not cache one-off operations, continuously changing data, or results that are difficult to key consistently.

  2. Choose the layer:

  3. Browser/HTTP: Use Cache-Control, max-age, ETag, and s-maxage.

  4. CDN or reverse proxy: Cache shareable HTTP responses.

  5. Application: Local memory is fast but separate for each instance.

  6. Distributed: Redis or Memcached shares entries across instances.

  7. Database: Buffer pools, query-plan caches, and materialized results reduce database work.

  8. Define canonical keys: Include every input affecting the response, such as entity ID, locale, API version, and authorization scope.

  9. Choose a read strategy: With cache-aside, read the cache first; on a miss, load the origin, populate the cache, and return the result.

  10. Define writes and invalidation: Decide whether writes update or invalidate cached entries and specify the acceptable staleness window.

  11. Bound usage: Set TTLs, entry-size and memory limits, eviction policies, metrics, and key-abuse protections.

  12. Design failure behavior: Test cache outages, slow dependencies, expired entries, hot keys, and concurrent misses.

A local lookup may take hundreds of nanoseconds; a same-datacenter distributed-cache request may take hundreds of microseconds to low milliseconds. These are rough orders of magnitude, and the benefit depends on the origin work avoided.

Cache Access and Write Strategies

Strategy Behavior Best fit Main cost
Cache-aside Application loads and fills on a miss; writes update the origin and invalidate cache General application caching Some requests still wait for the origin
Read-through Cache loads missing data from the origin Consistent cache encapsulation Tight coupling to the origin
Write-through Cache and origin are updated synchronously Read-heavy data needing prompt cache visibility Higher write latency and failure coupling
Write-back Cache is updated first and origin later Counters or aggregatable data tolerating delay Data loss if cache persistence fails
Write-around Write bypasses the cache and invalidates matching entries Data unlikely to be read immediately More misses if it later becomes popular

Cache-aside is usually the simplest starting point because it keeps application and cache infrastructure loosely coupled.

Invalidation and Failure Handling

  • TTL expiry bounds staleness, although expired entries may remain physically stored until accessed or evicted.

  • Event-driven invalidation can react to changes such as user.updated, but delivery can be delayed or lost; TTLs remain a safety net.

  • Write invalidation can race with an older read repopulating stale data. Versioned keys or atomic, version-aware writes provide stricter consistency.

  • Stampede protection uses per-key locking, request coalescing, or early refresh with randomized jitter.

  • Cache penetration from nonexistent keys can be reduced with short-lived negative caching or a Bloom filter.

  • Hot keys may require request coalescing, replication, or sharding when one key overloads a shard.

  • Cache failure should normally bypass the cache while rate limiting, circuit breakers, and bulkheads protect the origin. Serve stale data only when correctness permits.

Eviction and Technology Choices

Policy Behavior Caveat
TTL Removes entries based on age Staleness remains until expiry
LRU Favors recently used entries May discard repeatedly useful entries
LFU Favors frequently used entries Can retain obsolete popularity
FIFO Evicts in insertion order Does not adapt to access patterns
Concern Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, and streams Primarily opaque key-value entries
Persistence Optional RDB snapshots and append-only files None
Replication Built-in replication and cluster support None
Concurrency Command execution is primarily single-threaded, with I/O and background threads Multithreaded server
Typical fit Richer features and clustered deployments Simple, low-latency memory caching

Measure hit ratio, origin load, eviction rate, stale-data rate, and latency. A high hit ratio is worthwhile only when the avoided origin work exceeds cache latency and operational complexity.

Code Example

This Python cache implements TTL expiry, LRU eviction, and per-key request coalescing:

import threading
from collections import OrderedDict
from time import monotonic
from typing import Any, Callable, Generic, TypeVar, cast

T = TypeVar("T")
_MISSING = object()


class TTLCache(Generic[T]):
    def __init__(self, max_entries: int):
        if max_entries <= 0:
            raise ValueError("max_entries must be positive")

        self._entries: OrderedDict[str, tuple[T, float]] = OrderedDict()
        self._max_entries = max_entries
        self._lock = threading.RLock()
        self._key_locks: dict[str, tuple[Any, int]] = {}

    def _get(self, key: str) -> object:
        with self._lock:
            entry = self._entries.get(key)
            if entry is None:
                return _MISSING

            value, expires_at = entry
            if expires_at <= monotonic():
                del self._entries[key]
                return _MISSING

            self._entries.move_to_end(key)
            return value

    def set(self, key: str, value: T, ttl_seconds: float) -> None:
        if ttl_seconds <= 0:
            raise ValueError("ttl_seconds must be positive")

        with self._lock:
            self._entries[key] = (value, monotonic() + ttl_seconds)
            self._entries.move_to_end(key)

            while len(self._entries) > self._max_entries:
                self._entries.popitem(last=False)

    def get_or_set(
        self,
        key: str,
        loader: Callable[[], T],
        ttl_seconds: float,
    ) -> T:
        cached = self._get(key)
        if cached is not _MISSING:
            return cast(T, cached)

        with self._lock:
            key_lock, users = self._key_locks.get(key, (threading.Lock(), 0))
            self._key_locks[key] = (key_lock, users + 1)

        try:
            with key_lock:
                cached = self._get(key)
                if cached is not _MISSING:
                    return cast(T, cached)

                value = loader()
                self.set(key, value, ttl_seconds)
                return value
        finally:
            with self._lock:
                key_lock, users = self._key_locks[key]
                if users == 1:
                    del self._key_locks[key]
                else:
                    self._key_locks[key] = (key_lock, users - 1)


cache = TTLCache[str](max_entries=10_000)
result = cache.get_or_set(
    "user:42",
    lambda: "value-loaded-from-the-origin",
    ttl_seconds=60,
)

This coalesces loads within one process. Distributed deployments need a shared coordination mechanism or rely on a cache feature that provides atomic operations.

Best Practices and Interview Points

  • Define the acceptable staleness window before choosing TTLs and invalidation rules.

  • Treat the cache as an optimization, not the authoritative source, unless the design explicitly says otherwise.

  • Do not store per-user sensitive data on shared infrastructure. Cache-Control: private instructs compliant shared caches not to store it; it does not make storage in Redis or Memcached safe. Use strict isolation and appropriate encryption if shared caching is unavoidable.

  • Never add a cache without an invalidation plan, memory limit, origin fallback, and metrics. The common interview mistake is focusing on cache speed while ignoring consistency, stampedes, and failure modes.

12

How do you synchronize caches across distributed systems?

Cache synchronization means keeping cached copies within a defined staleness bound of their authoritative data source while tolerating delayed messages, node failures, and concurrent requests. The usual approach is to combine a write strategy, versioned invalidation events, and TTLs as a safety net.

Read full answer

Applies toDistributed caching with Redis 8.x and Memcached 1.6.x; the example uses ioredis 5.x and KafkaJS 2.x

Consistency Model

Choose how stale an entry may be before selecting an invalidation mechanism:

Strategy Behaviour Main trade-off Best fit
Cache-aside Application loads from the database on a miss Simplest, but temporary staleness is possible General read-heavy services
Read-through A cache loader obtains missing data Standardizes access but adds loader coupling Services with a common data layer
Write-through Synchronously updates cache and database Lower read staleness, but cross-store atomicity is not automatic Data that must be immediately readable
Write-back Updates cache first and persists asynchronously Lower write latency, but failure can lose unflushed writes Tolerant, high-write workloads
Write-around Bypasses the cache for writes Reduces cache pollution; any existing entry must still be invalidated Write-heavy or rarely read data

Strict cross-region coordination is expensive because a cross-region round trip is roughly 50–150 ms. Most systems therefore define bounded staleness instead of promising instantaneous consistency.

Invalidation and Event Propagation

A reliable cache-aside update flow is:

  1. In one database transaction, update the entity and insert an invalidation event into a transactional outbox.

  2. Commit the transaction.

  3. Publish outbox records to a durable topic such as Kafka, using the entity key as the partition key.

  4. Consumers delete or refresh the affected shared or local cache entries.

  5. Let a TTL remove stale entries if an event is delayed or lost.

The outbox prevents the database-commit/event-publish gap. Delivery is generally at least once, so consumers must be idempotent.

Include a monotonically increasing entity version in events. Consumers should atomically reject an older version, and deletions should retain a short-lived version tombstone so a delayed update cannot recreate deleted data. Tombstone retention should cover the maximum expected event delay and duplication window.

A common stale-refill race occurs when a reader fetches an old value, a writer commits and invalidates it, and the reader then writes the old value back. A per-key lock works only if both the writer’s invalidation and the reader’s refill participate. A version-aware compare-and-set gate is usually simpler; request coalescing alone only reduces duplicate database reads and does not close this race.

Failure and Traffic Protection

  • Stampede protection: Use per-key locking, request coalescing, stale-while-revalidate, or early expiry with randomized jitter.

  • Hot keys: Replicate a read-mostly hot key into local caches or split a large value into subkeys. A single logical key normally maps to one shard.

  • Cache penetration: Cache negative results briefly and consider a Bloom filter for large key spaces.

  • Cache outage: Apply short timeouts, fall back to the database with rate limiting, and use a circuit breaker.

  • Eviction: TTL controls expiration, while size limits control eviction pressure. LRU favors recent access, LFU favors frequently accessed entries, and FIFO is simple but can evict hot entries. Redis 8.x supports approximate LRU and LFU policies; upstream Memcached 1.6.x uses LRU.

Redis and Memcached

Property Redis 8.x Memcached 1.6.x
Data model Strings, hashes, lists, sets, sorted sets, atomic operations Simple key-value objects
Persistence Available Not provided by upstream Memcached
Replication Available, with asynchronous replication by default Not provided by upstream Memcached
Concurrency Primarily single-threaded command execution with background I/O support Multithreaded request handling

Redis provides more functionality but generally requires more operational care. Memcached remains attractive for a simple low-latency object cache. Neither automatically provides cross-region consistency or stampede protection.

Code Example

This consumer uses a durable Kafka topic to invalidate a shared Redis cache. Cache values are JSON objects containing a numeric version.

import Redis from "ioredis";
import { Kafka } from "kafkajs";

const redisUrl = process.env.REDIS_URL;
const brokerText = process.env.KAFKA_BROKERS;

if (!redisUrl || !brokerText) {
  throw new Error("REDIS_URL and KAFKA_BROKERS are required");
}

const brokers = brokerText.split(",").map((value) => value.trim()).filter(Boolean);
if (brokers.length === 0) {
  throw new Error("KAFKA_BROKERS must contain at least one broker");
}

const cache = new Redis(redisUrl);
const kafka = new Kafka({ clientId: "cache-invalidator", brokers });
const consumer = kafka.consumer({ groupId: "cache-invalidator-v1" });

const invalidateIfNotNewer = `
local raw = redis.call('GET', KEYS[1])
if not raw then return 0 end

local ok, entry = pcall(cjson.decode, raw)
if ok and type(entry) == 'table' then
  local currentVersion = tonumber(entry.version)
  if currentVersion and currentVersion > tonumber(ARGV[1]) then
    return 0
  end
end

redis.call('DEL', KEYS[1])
return 1
`;

try {
  await consumer.connect();
  await consumer.subscribe({
    topic: "cache.invalidations",
    fromBeginning: false,
  });

  await consumer.run({
    eachMessage: async ({ message }) => {
      if (!message.value) return;

      const event = JSON.parse(message.value.toString()) as {
        key: string;
        version: number;
      };

      if (!event.key || !Number.isSafeInteger(event.version)) {
        throw new Error("Invalid cache invalidation event");
      }

      await cache.eval(invalidateIfNotNewer, 1, event.key, event.version);
    },
  });
} finally {
  await consumer.disconnect().catch(() => undefined);
  await cache.quit().catch(() => undefined);
}

Practical Guidance

Use cache-aside with TTLs for most services, adding durable invalidation only when its consistency benefit exceeds its operational cost. Keep the database authoritative, use access controls and tenant isolation for sensitive cached data, and monitor hit ratio, evictions, invalidation lag, stale-data incidents, and database fallback load.

Do not cache rapidly changing data, data that cannot tolerate the selected TTL, or data that is inexpensive to read. The key design decision is not merely adding a cache; it is defining invalidation, consistency bounds, eviction, and recovery from missed or delayed synchronization.

13

How do hash maps implement caching?

Hash maps provide constant-time average lookup from a cache key to its cached value. They make cache access fast, but expiration, invalidation, eviction, consistency, and distributed coordination require additional mechanisms.

Read full answer

Applies toJava 21 standard collections; hash-map caching principles are implementation-independent

How Hash Maps Provide Fast Lookup

A cache avoids expensive work such as querying PostgreSQL, calling an external service, rendering a page, or recomputing a result. Its hit path should be much faster than the original operation.

A hash map computes a key’s hash and uses it to locate a bucket:

  • Average lookup: O(1)

  • Average insertion and deletion: amortized O(1)

  • Worst case: O(n) when many keys collide

  • Memory cost: keys, values, hash-table metadata, and any expiration or ordering metadata

A local map access is typically tens to hundreds of nanoseconds, while a same-datacenter Redis request is often around 0.5 ms. This difference makes caching worthwhile for frequently requested data.

Where Hash Maps Are Used

  • Browser and HTTP caches: Index cached responses by URL and selected headers, with metadata such as ETag, max-age, and s-maxage.

  • CDNs and reverse proxies: Map request variants to cached responses.

  • Application caches: Store objects in local maps for extremely fast access, although every application instance then has separate state.

  • Distributed caches: Use hash-based indexing to locate values in Redis or Memcached; clustered systems also partition keys across nodes.

  • Database caches: Buffer pools and metadata structures often use specialized indexes and hash tables internally.

Tool Data model Persistence and replication Concurrency model
Redis Strings, hashes, lists, sets, sorted sets, streams, and other structures RDB snapshots, AOF persistence, replicas, and cluster mechanisms Most command execution uses a single main thread; some work is threaded
Memcached Primarily opaque key-value objects No built-in persistence or replication Multiple worker threads

Cache Update Strategies

The map stores values, but an application or cache library determines the update and consistency semantics.

Strategy Behavior Best fit
Cache-aside Read from cache; on a miss, load from the database and populate the cache General-purpose read-heavy applications
Read-through The cache calls a loader when a key is absent Centralized loading logic
Write-through Update cache and database synchronously Read-after-write needs with acceptable write latency
Write-back Update the cache and flush to the database later High throughput when some data loss is tolerable
Write-around Write directly to the database and bypass the cache Avoiding cache pollution from write-heavy data

For example, an e-commerce product catalog can use cache-aside with a short TTL. Product details may be cached, while checkout totals and payment state should remain authoritative because stale values can cause incorrect transactions.

Eviction, Expiration, and Consistency

A plain HashMap has no automatic capacity limit, TTL, or eviction policy. Production caches add policies such as:

  • LRU: Removes the least recently used entry.

  • LFU: Removes the least frequently used entry.

  • FIFO: Removes the oldest entry regardless of usage.

  • TTL: Expires entries after a specified duration.

Invalidation options include explicit deletion after a database change, versioned keys that make old entries unreachable, and event-driven invalidation. Expiration can be lazy, with expired entries removed when accessed, or active, with a background process scanning for expired entries.

Code Example

This Java example implements a bounded local LRU cache with an access-ordered LinkedHashMap. Collections.synchronizedMap protects individual map operations, including the ordering change performed by get.

import java.util.Collections;
import java.util.LinkedHashMap;
import java.util.Map;

public final class LruCacheExample {
    static final class LruCache<K, V> extends LinkedHashMap<K, V> {
        private final int maxEntries;

        LruCache(int maxEntries) {
            super(16, 0.75f, true); // accessOrder = true
            if (maxEntries <= 0) {
                throw new IllegalArgumentException(
                        "maxEntries must be positive");
            }
            this.maxEntries = maxEntries;
        }

        @Override
        protected boolean removeEldestEntry(Map.Entry<K, V> eldest) {
            return size() > maxEntries;
        }
    }

    public static void main(String[] args) {
        Map<String, String> cache = Collections.synchronizedMap(
                new LruCache<String, String>(2));

        cache.put("product-1", "Laptop");
        cache.put("product-2", "Phone");

        String product = cache.get("product-1");
        System.out.println(product); // Laptop
    }
}

Best Practices and Pitfalls

  • Define invalidation first: Select TTL, explicit deletion, or versioned keys according to the consistency requirement.

  • Prevent stampedes: Use request coalescing, locking, stale-while-revalidate, or early expiry with jitter when many requests miss one hot key.

  • Handle penetration: Cache negative results briefly or use a Bloom filter when invalid keys cause repeated database lookups.

  • Distribute hot keys: Replicate or partition exceptionally popular entries.

  • Protect sensitive data: Keep payment details and authorization decisions out of shared caches; isolate user data by tenant and user.

  • Design for failure: Fall back to the origin with timeouts, rate limiting, and a circuit breaker instead of making the cache a hard dependency.

  • Avoid unnecessary caching: Low reuse rates, large objects, frequently changing data, and strongly consistent operations may cost more than they save.

  • Measure results: Track hit ratio, latency, evictions, stale reads, memory use, and origin load.

The common interview mistake is adding a cache without specifying ownership, TTL, invalidation, eviction, stampede protection, and behavior when the cache is unavailable.

14

What are the main caching algorithms and their trade-offs?

Common caching algorithms determine which entries remain in a cache and are evicted when capacity is reached. LRU is a common default, but the best policy depends on access patterns, freshness requirements, and the cost of maintaining cache metadata.

Read full answer

Applies toRedis 8.x and Memcached 1.6.x for product-specific details; caching concepts are version-independent

Eviction and Expiration Algorithms

Algorithm Behavior Best fit Main cost
LRU Evicts the entry used least recently General-purpose workloads with changing popularity Requires recency tracking; distributed implementations often use an approximate version
LFU Evicts the entry used least frequently Stable hot keys, such as popular products Frequency counters add overhead and may preserve obsolete entries
FIFO Evicts the oldest entry regardless of usage Simple, inexpensive caches with uniform value Does not reflect actual popularity
TTL Removes entries after an expiration time Data with a known freshness window Expiration does not control which valid entry is evicted first
Random Evicts a random entry Avoiding recency or frequency metadata Can evict useful hot entries
ARC Balances recency and frequency adaptively Mixed workloads where popularity changes frequently More metadata and implementation complexity
  • Hit ratio is the percentage of requests served without reaching the origin. For example, a 95% hit ratio means only 5% of requests load data from the database.

  • Eviction and TTL are complementary: TTL controls freshness, while LRU, LFU, or another policy controls capacity.

  • Exact policies require additional bookkeeping. Systems such as Redis commonly implement approximate LRU or LFU sampling to reduce coordination overhead.

Cache Population and Write Strategies

Strategy How it works Advantages Trade-offs
Cache-aside Application checks the cache and loads from the database on a miss, then populates the cache Simple, and cache failures can be handled by falling back to the source Application manages reads, population, and invalidation
Read-through Cache loads missing data itself Application uses one data-access interface Cache implementation must know how to load the source
Write-through Cache and database are updated synchronously Cache reflects successful writes Write latency increases and partial failures require handling
Write-back Cache is updated immediately; database update happens asynchronously Fast writes and reduced database load Data can be lost if the cache fails before flushing; eviction semantics are more complex
Write-around Writes go directly to the database without updating the cache Avoids filling the cache with entries that may not be read The next read pays the full cache-miss cost

Cache-aside is widely used for web and database-backed applications. Write-through suits systems requiring synchronous cache visibility, while write-back is common where durability and failure recovery are carefully engineered.

Where Caching Fits

Layer Typical use Important controls
Browser Caches static assets and API responses Cache-Control: max-age, ETag, request validators
CDN Serves shared content from locations near users s-maxage, cache keys, provider-specific cache tags or surrogate keys, purge APIs
Reverse proxy Caches responses from backend services Vary, cache-key normalization, request collapsing
Application cache Stores local data in process memory Low latency, but entries are not shared across instances
Distributed cache Shares objects across service instances Redis or Memcached, serialization, network latency, cluster topology
Database cache Reuses pages, indexes, query results, or buffers Internal cache invalidation and memory pressure

max-age primarily controls freshness in a browser, while s-maxage controls shared caches such as CDNs. An ETag allows a client to revalidate an entry and receive 304 Not Modified when it has not changed.

Redis and Memcached

Characteristic Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and other structures Mostly opaque key-value byte strings with limited CAS support
Persistence Optional snapshots and append-only logging No built-in durable persistence
Replication Supports replication and cluster distribution No equivalent built-in replication feature
Expiration and eviction TTL expiration plus configurable eviction policies, including approximate LRU and LFU TTL expiration plus LRU eviction
Best fit Shared state, counters, streams-based queues, sessions, coordination, and rich data structures Simple low-latency object caching at scale
Operations Richer feature set with more operational complexity Simpler operation and generally lower per-item overhead

Memcached uses worker threads for request handling. Redis is event-driven, can use additional I/O threads, and executes commands primarily on the main thread. Redis also supports background persistence work, which must be monitored because cache persistence does not necessarily provide the same durability guarantees as a primary database.

Reliability and Common Pitfalls

  • Stale data: Delete or update entries after database commits. Use a transactional outbox or durable event stream when cache invalidation must survive process failures. TTLs provide a safety net, not a complete consistency mechanism.

  • Cache stampede: Many requests may load the same expired key. Protect misses with request coalescing, a distributed lock, or probabilistic early recomputation with jitter.

  • Cache penetration: Repeated requests for nonexistent keys can reach the database. Cache negative responses briefly and use a Bloom filter when the keyspace is large.

  • Hot keys: A single popular key can overload one shard. Replicate reads, split large values, or maintain a short-lived local cache in front of the distributed cache.

  • Sensitive data: Avoid placing plaintext per-user or security-sensitive values in a shared cache. Scope keys correctly, minimize stored data, and apply appropriate encryption where required.

  • Poor cacheability: Do not cache frequently changing data with little reuse, authorization decisions, or values that require strict immediate consistency without a robust invalidation design.

  • Operational cost: Caching reduces origin load and latency but adds memory usage, invalidation logic, monitoring, serialization overhead, and another failure boundary.

A strong design starts with the access pattern and required freshness, estimates the expected hit ratio, defines cache keys, and specifies what happens on misses, stale entries, evictions, and cache outages. The most common interview mistake is adding a cache without an invalidation or failure plan.

15

How do you design a cache for high concurrency?

A high-concurrency cache design starts with a clear source of truth, a cache-aside or equivalent read path, and explicit protection against misses overwhelming the database. The goal is low tail latency and high hit ratio without allowing cache failure, expiry, or invalidation to become a database outage.

Read full answer

Applies toGeneral cache architecture; Redis 8.0 and Memcached 1.6.x for product-specific behavior; Java 17 with Caffeine 3.1.8 in the code example

Request Path and Layers

A typical flow is:

Client → CDN or reverse proxy → application → local or distributed cache → database

  • Browser and CDN: Use Cache-Control: max-age for browser freshness and s-maxage for shared caches. ETag with If-None-Match supports conditional requests. CDN caching works well for public assets and cacheable API responses.

  • Reverse proxy: Nginx or Envoy can cache shared HTTP responses, reducing application load.

  • Application cache: In-process memory has the lowest latency but is limited to one instance. Redis or Memcached provides a shared cache across instances.

  • Database cache: Buffer or page caches, indexes, and read replicas reduce disk and query work; they do not remove the need for an application-level cache.

Memory access is roughly 100 ns, a same-datacenter cache call around 0.5 ms, and a database query potentially milliseconds or longer. These are rough orders of magnitude. For example, an e-commerce product page can be cached publicly while checkout and payment data bypass shared caches.

Strategies and Consistency

Strategy Behavior Best fit
Cache-aside Read the cache; on a miss, read the backing store and populate the cache. Writes update the store and invalidate the entry. General-purpose designs and explicit application control
Read-through The cache library loads missing values from the backing store. Systems wanting cache-loading logic centralized in a library
Write-through Update the cache and backing store synchronously. Writes that must be immediately readable through the cache; partial-failure handling is still required
Write-back Update the cache and persist asynchronously. Workloads tolerating bounded staleness and possible loss of recent writes during failure
Write-around Bypass the cache for writes and update the store directly. Data unlikely to be read soon enough to justify occupying cache space

The database should normally remain the source of truth. Invalidating immediately after a commit can race with an older read that repopulates the cache with stale data. Use per-key ordered invalidation, monotonic versioned keys, or carefully bounded TTLs to reduce that risk.

Invalidation, Eviction, and Miss Protection

  • Capacity and eviction: Use TTLs to bound staleness and an eviction policy such as LRU, LFU, or FIFO. LRU suits changing access patterns; LFU favors consistently popular items; FIFO is simple but does not model popularity.

  • Invalidation: Prefer writer-driven invalidation or ordered change events where possible. TTL-only invalidation is simpler but permits staleness for the TTL duration.

  • Stampede protection: Coalesce concurrent requests for one key with local singleflight or a distributed lock. Stale-while-revalidate, early renewal, and randomized jitter are alternatives. Distributed locks require lease expiry and safe unlock behavior.

  • Penetration protection: Cache negative results briefly and use a Bloom filter when the possible key universe is known.

  • Hot keys: Spread reads across replicas, localize immutable data, precompute popular results, and apply admission control so one workload cannot evict everything.

  • Capacity and hit ratio: One million 1 KB entries require roughly 1 GB before metadata, fragmentation, and replication overhead. Measure actual memory use and weight hit ratio by the database work avoided, not by the percentage alone.

Redis and Memcached

Concern Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and other structures Primarily opaque key-value pairs
Persistence Optional RDB or AOF, with durability and latency trade-offs None; designed as an in-memory cache
Replication Built-in asynchronous replication No built-in server replication; clients or the 1.6 proxy can write copies to several servers
Concurrency Mainly main-thread command execution, with background work and optional I/O threads Multithreaded request processing
Clustering Native Redis Cluster Usually sharded across independent servers

Choose Redis when persistence, richer structures, scripting, or streams are useful. Choose Memcached for a simple shared key-value cache with straightforward client-side sharding.

Code Example

This local application cache uses Caffeine’s atomic get operation to coalesce concurrent loads for the same key and caches negative results with Optional.

import com.github.benmanes.caffeine.cache.Cache;
import com.github.benmanes.caffeine.cache.Caffeine;
import java.time.Duration;
import java.util.Optional;

public final class UserCache {
  record User(long id, String name) {}
  interface UserRepository { User findById(long id); }

  private final UserRepository repository;
  private final Cache<Long, Optional<User>> cache;

  public UserCache(UserRepository repository) {
    this.repository = repository;
    this.cache = Caffeine.newBuilder()
        .maximumSize(100_000)
        .expireAfterWrite(Duration.ofSeconds(30))
        .build();
  }

  public Optional<User> find(long id) {
    // Concurrent misses for this key share one local computation.
    return cache.get(id, key -> Optional.ofNullable(repository.findById(key)));
  }

  public void invalidate(long id) {
    cache.invalidate(id);
  }
}

Reliability and Best Practices

  • Treat the cache as an optimization, not the sole source of truth.

  • Set cache timeouts and use circuit breakers, bounded fallbacks, rate limits, and bulkheads to protect the database.

  • Add TTL jitter so unrelated keys do not expire simultaneously.

  • Monitor hit ratio, miss latency, evictions, memory pressure, hot-key distribution, replication lag, and database load.

  • Never assume cache and database updates are atomic across failures.

  • Do not place per-user sensitive data in a shared cache unless keys, access controls, and isolation prevent cross-user leakage.

  • Avoid caching cheap, rarely reused, or impossible-to-invalidate data. A cache adds memory, operational complexity, and consistency risk, so it should not be added merely because the system is distributed.

16

How to prevent cache stampede under high load?

Cache stampede protection prevents concurrent requests for the same expired or missing key from rebuilding the same value simultaneously. Use per-key request coalescing or a bounded distributed lock, and optionally serve stale data while one worker refreshes it.

Read full answer

Applies toall cache systems and versions; the code example uses Python 3.12

Why a Stampede Happens

In a cache-aside design, a hot key can expire while thousands of requests are arriving. Every request observes the miss and independently calls the database, turning one origin request into thousands. A same-datacenter network round trip is roughly 0.5 ms and a cross-region round trip roughly 50–150 ms, while database queries may take milliseconds or much longer; these are rough orders of magnitude.

The goal is not merely to increase the TTL. It is to ensure that only one request rebuilds the value while others wait, reuse the newly cached value, or receive a controlled stale fallback.

Core Techniques

  • Single-flight or request coalescing: Keep one in-flight loader per key. All concurrent misses await the same result. Process-local coalescing protects one application instance; multiple instances require shared coordination.

  • Distributed locking: Allow one instance to refresh while others wait with a bounded timeout, then recheck the cache. Use a lease, ownership token, safe compare-and-delete release, and preferably renewal or fencing so an expired owner cannot overwrite a newer value.

  • Stale-while-revalidate: Maintain a soft TTL and hard TTL. Serve stale data immediately while one coordinated background worker refreshes it. This reduces latency but permits stale reads.

  • Early refresh with jitter: Refresh shortly before the soft TTL expires and randomize the schedule across keys. Jitter alone does not protect one shared hot key; the refresh itself must still use coalescing or locking.

  • Cache warming: Prepopulate predictable hot keys before a launch or expected spike. This consumes extra origin capacity and cannot cover arbitrary traffic.

  • Negative caching: Short-lived negative results can prevent repeated database lookups for nonexistent objects, which addresses cache penetration rather than stampede protection.

Trade-Offs and Example

Technique Benefit Main cost or risk
Single-flight Fresh value with one origin load Tail latency for waiters; limited to one process unless distributed
Distributed lock Cluster-wide single refresh Lease expiry, duplicate work, and operational complexity
Stale-while-revalidate Low latency and graceful degradation Temporarily stale or semantically invalid data
Early refresh Most requests hit fresh data Wasted refreshes if the key becomes cold

For a popular URL shortener, one process might coalesce all misses for short->long. Across several application instances, Redis or another shared coordination service can provide a distributed lock; after the owner writes the cache, waiters must read the new value rather than each rebuilding it. Stale redirects may be acceptable for a public feed, but not for payment destinations, authorization decisions, or inventory reservations.

Code Example

This Python example demonstrates process-local single-flight for concurrent misses:

import asyncio
from collections.abc import Awaitable, Callable
from dataclasses import dataclass
from time import monotonic
from typing import Generic, TypeVar

T = TypeVar("T")


@dataclass
class Entry(Generic[T]):
    value: T
    expires_at: float


class SingleFlightCache(Generic[T]):
    def __init__(self, ttl_seconds: float) -> None:
        self._ttl_seconds = ttl_seconds
        self._entries: dict[str, Entry[T]] = {}
        self._inflight: dict[str, asyncio.Task[T]] = {}

    async def get_or_load(
        self,
        key: str,
        loader: Callable[[str], Awaitable[T]],
    ) -> T:
        entry = self._entries.get(key)
        if entry is not None and monotonic() < entry.expires_at:
            return entry.value

        task = self._inflight.get(key)
        if task is None:
            task = asyncio.create_task(self._load_and_store(key, loader))
            self._inflight[key] = task
            task.add_done_callback(
                lambda done: self._remove_inflight(key, done)
            )

        # Cancelling one waiter does not cancel the shared load.
        return await asyncio.shield(task)

    async def _load_and_store(
        self,
        key: str,
        loader: Callable[[str], Awaitable[T]],
    ) -> T:
        value = await loader(key)
        self._entries[key] = Entry(
            value=value,
            expires_at=monotonic() + self._ttl_seconds,
        )
        return value

    def _remove_inflight(
        self,
        key: str,
        task: asyncio.Task[T],
    ) -> None:
        if self._inflight.get(key) is task:
            del self._inflight[key]


async def main() -> None:
    calls = 0

    async def loader(key: str) -> str:
        nonlocal calls
        calls += 1
        await asyncio.sleep(0.05)
        return f"value-for-{key}"

    cache = SingleFlightCache[str](ttl_seconds=30)
    values = await asyncio.gather(
        *(cache.get_or_load("hot-key", loader) for _ in range(1000))
    )
    print(len(values), calls)  # 1000 1


asyncio.run(main())

Best Practices and Common Pitfalls

  • Bound lock waits and refresh operations; never let waiters block indefinitely.

  • Keep a last-known-good value where stale reads are acceptable and use it on timeout.

  • Do not serve stale data for payments, authorization, or other correctness-critical operations.

  • Monitor hit ratio, origin queries per key, lock wait time, refresh latency, and rejected or failed waits.

  • If one key remains a bottleneck, shard or partition the data rather than only extending its TTL.

  • Avoid blindly placing sensitive per-user data in a shared cache; stampede protection does not address confidentiality or revocation.

17

What are the trade-offs of read-heavy vs write-heavy caching strategies?

Read-heavy caching strategies prioritize serving frequent reads from cache, while write-heavy caching strategies prioritize keeping cached data correct under frequent updates, and each sacrifices the opposite priority.

Read full answer

Applies toall versions

The Problem Each Solves

  • Read-heavy: Solves read amplification. For example, on an e-commerce product listing, a database read can take roughly ~0.5 ms in the same datacenter compared to a memory read of ~100 ns (rough orders of magnitude). Caching removes repeated trips for data that changes infrequently.

  • Write-heavy: Solves stale-data risk. For example, during a flash sale on a payment-adjacent inventory view, serving an outdated count can be incorrect. Here, the strategy must account for every update instead of only optimizing for cache misses.

Core Trade-Off Comparison

Aspect Read-heavy oriented (cache-aside, read-through) Write-heavy oriented (write-through, write-back)
Main Goal Maximize hit ratio and minimize read latency. Maximize freshness and minimize the chance of serving inconsistent data.
Write Path Cheap. Often writes go straight to the database, with the cache only populated or invalidated on reads. Expensive. Writes must update the cache and/or database on the write path, creating write amplification.
Read Latency Extremely fast on hits, but can suffer if hot keys expire at the same time. Fast on hits, but there is less room to rely on long-lived stale entries.
Consistency Fits eventual consistency well. Safe for most read-mostly entities. Needs stronger alignment between cache and database. Drift is the main threat if not handled carefully.
Invalidation Simpler, often handled by TTLs or explicit eviction when writes happen elsewhere. Central to the design. Every relevant write must invalidate or update the cached entry.
Failure Impact A cache outage mostly increases DB load and latency. A failed flush or cache failure can be far more damaging, especially with write-back where not all writes may be durable yet.

Key Considerations

  • Hit-ratio reasoning: For read-heavy, hit ratio is important, but chasing it with uncoordinated expiries invites a cache stampede. For write-heavy, optimizing hit ratio can be counterproductive if it forces unsafe staleness.

  • Eviction & pressure: Read-heavy amplifies hot key pressure. Write-heavy amplifies write-path pressure, so backpressure, timeouts and idempotency keys matter more.

  • Data suitability: Do not cache per-user sensitive data on shared caches for either strategy. Only cache data whose staleness tolerance you can explicitly justify based on the workload.

  • Other notes: Write-around can be useful in specific cases, but it trades cache population for avoiding cache pollution rather than actively optimizing for a write-heavy correctness model.

When To Use Which

  • Use read-heavy caching when reads vastly outnumber writes, the data has clear staleness tolerance, and you can afford eventual consistency (e.g. a news feed’s public article metadata).

  • Use write-heavy caching when writes are frequent and correctness is strict for the cached view (e.g. checkout availability counters). Be ready for the added synchronization work.

  • Use neither if the workload is write-dominated with zero staleness tolerance and no clear hot set, or if you cannot commit to an invalidation plan.

Interview Tips & Common Pitfalls

  • Ask first: Always ask “how stale can this be?” before picking a strategy, and state trade-offs against a concrete example instead of assuming universal superiority.

  • Biggest mistake: Adding a cache without an invalidation plan. For write-heavy paths especially, that plan is mandatory, not optional.

18

What is a cache manifest in a web application?

A cache manifest is a legacy plain-text file used by the HTML Application Cache that tells the browser which resources to store locally so a web application can load or partially operate without a network connection. It provides offline access and faster repeat visits, but it introduces storage, update, and cache-invalidation complexity.

Read full answer

Applies toHTML Application Cache in supporting legacy browsers (removed in Chromium 95); Service Worker and Cache Storage in current browsers

Role in a Web Application

Without a local cache, every visit may require network requests for the HTML, JavaScript, stylesheets, images, and other resources. This increases latency and makes the application unusable when connectivity is poor or unavailable.

A cache manifest solves this by declaring three groups of resources:

  • CACHE: Resources the browser downloads and stores for offline use, such as the application shell, static assets, and frequently accessed data.

  • NETWORK: Resources that should be fetched from the network rather than stored in the offline cache, such as live inventory or payment APIs.

  • FALLBACK: Alternative resources used when the network fails, such as an offline page or a cached response.

For example, a field-service application might precache its HTML shell, CSS, JavaScript, and map assets while keeping work-order submissions network-only.

Request Flow

On the first visit, the browser:

  1. Loads the page and discovers the manifest through the manifest attribute.

  2. Downloads the manifest, which legacy browsers serve with the text/cache-manifest MIME type.

  3. Downloads resources listed in the CACHE section.

  4. Stores those resources in the browser’s Application Cache.

On later visits, cached resources can be served locally. When the manifest changes, the browser attempts to download the updated resources and update its cache. This gives basic offline support, but it does not provide the fine-grained control available through modern service workers.

Invalidation and Consistency

A cache manifest has limited runtime controls; it does not provide precise per-entry TTLs or cache policies. Versioned resource paths or URLs are usually needed so updated assets are treated as new entries.

The main consistency risk is serving an old application shell alongside newer APIs or business logic. A safe release should update assets and the manifest together, then ensure old cached versions are eventually removed.

It also must not be used to cache transactional data that must remain current. For example, checkout totals, account balances, and authorization decisions should normally be validated against the server. Client-side cache storage is not a replacement for server-side authorization.

Relationship to Other Caching Layers

A cache manifest is specifically a browser-side offline mechanism. It is different from:

  • HTTP caching, controlled by headers such as Cache-Control, ETag, and max-age.

  • CDN and reverse-proxy caching, where responses are stored at an edge or gateway.

  • Web App Manifest, usually manifest.webmanifest, which contains installation metadata such as icons and display mode—not cache rules.

  • Service Worker caching, which provides programmatic control over requests, cache names, update lifecycles, and offline responses.

Modern Status and Trade-Offs

The HTML Application Cache was removed in Chromium 95 and is not a good choice for new applications. The modern replacement is the Service Worker API with Cache Storage: the service worker installs named caches, decides which requests to cache, and can explicitly delete old cache versions during activation.

Service workers offer better invalidation and runtime caching, but they introduce lifecycle management, HTTPS requirements, and more opportunities for stale responses or excessive storage. Neither mechanism reduces the cost of the initial download, and browsers may evict local data under storage pressure.

Use a cache manifest only when maintaining an older HTML5 application. For new systems, use HTTP caching for static assets, a CDN for shared content, and a service worker when genuine offline behavior or controlled runtime caching is required. A common interview mistake is treating a cache manifest as a general performance cache without defining invalidation, sensitive-data handling, and what happens when the network fails.

19

How do you maintain consistency between cache and primary storage?

Cache consistency is usually maintained by making the primary database the source of truth, committing writes there, and invalidating or updating the derived cache. Most systems accept eventual consistency and bound stale exposure with TTLs, idempotent retries, and per-key locking; strict read-after-write requires stronger coordination.

Read full answer

Applies toall caching architectures; examples use Redis 7, Memcached 1.6, PostgreSQL 16, Python 3.12, psycopg 3.2, and redis-py 5.2

Consistency Model

  • Eventual consistency: A cache entry may briefly be stale but converges after invalidation, refresh, or expiry.

  • Read-after-write consistency: A read immediately after a successful write observes that write. This may require reading the primary, using a known write version, or updating the cache before acknowledging the write.

  • The cache should not normally be authoritative. Authorization, balances, and other critical state should use a strongly consistent source.

Write Strategies

Strategy Behaviour Trade-off
Cache-aside Application loads data after a miss Common default; requires explicit invalidation
Read-through Cache loads missing data Simplifies reads but still needs consistent invalidation
Write-through Cache and primary are updated synchronously Reduces misses but can diverge on partial failure
Write-back Cache is updated first and persisted later Fast writes, but data can be lost
Write-around Write bypasses and does not populate the cache Suitable for data unlikely to be read soon; existing entries still need invalidation

No strategy atomically updates two independent systems. A process can update one and fail before updating the other.

Invalidation That Survives Failure

A cache-aside write path is:

  1. Commit the primary-store update.

  2. Invalidate the affected cache key.

  3. Reload the value on the next cache miss.

The main race is a stale fill: a reader fetches old data, a writer commits and invalidates, then the reader writes the old value into the cache. Prevent it by having the reader acquire a per-key lock before reading, recheck the cache, perform the primary read, and set the value before releasing the lock. The invalidator must take the same lock around deletion. Locking only the cache write does not close the race.

  • TTL: Bounds entry age if invalidation fails, but is not a replacement for invalidation.

  • Transactional outbox: Store the primary change and invalidation event in one database transaction. A retrying worker deletes the cache key with exponential backoff and jitter. At-least-once delivery is safe because deletion is idempotent.

  • Early expiry with jitter: Refresh popular entries slightly before expiry to avoid synchronized expiry.

  • Request coalescing: Combine concurrent misses or use a lock to prevent a cache stampede.

  • Versioned entries: Store a version with the payload and compare it with an authoritative, cheaply available version token. A write response may supply the version for a subsequent read; querying the primary version on every hit would remove the cache benefit.

Layering, Eviction, and Failure Modes

  • Browser, CDN, and reverse proxy: Use Cache-Control, ETag, max-age, and Cache-Control: s-maxage. CDNs may purge URLs, while some also support surrogate-key purging. These layers can remain stale independently.

  • Distributed caches: Redis supports strings, hashes, sets, sorted sets, and streams, with optional persistence and replication. Memcached 1.6 is multithreaded, stores opaque byte values, and has no built-in persistence or replication.

  • Eviction: Redis 7 defaults to noeviction; LRU-like behavior requires policies such as allkeys-lru or volatile-lru, and Redis also provides LFU policies. Other caches may use LRU, LFU, or FIFO. TTL controls freshness, while eviction manages memory pressure.

  • Hot keys: One popular key can saturate a shard. Replicate deliberately, partition the key, or use a small local cache.

  • Cache penetration: Cache negative results briefly or use a Bloom filter for repeated nonexistent identifiers.

  • Sensitive data: Do not store per-user or authorization-sensitive data in a shared cache without access isolation and authorization.

  • Database caches: Database-managed query or buffer caches are outside application invalidation. For example, PostgreSQL’s buffer pool caches pages rather than logical application rows.

The hit ratio is hits / (hits + misses). Cache only when expected database savings exceed memory, latency, invalidation, and operational costs; rapidly changing, rarely read, cheap-to-recompute, or unsafe-to-invalidate data is often unsuitable.

Code Example

The outbox records invalidation atomically with the primary update:

CREATE TABLE products (
    id bigint PRIMARY KEY,
    name text NOT NULL,
    version bigint NOT NULL DEFAULT 1
);

CREATE TABLE cache_invalidation_outbox (
    event_id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    entity_id bigint NOT NULL,
    cache_key text NOT NULL,
    created_at timestamptz NOT NULL DEFAULT now(),
    processed_at timestamptz
);

The writer commits the product change and event together:

import os

import psycopg
from redis import Redis

DB_URL = os.environ["DATABASE_URL"]
cache = Redis.from_url(os.environ["REDIS_URL"], decode_responses=True)


def update_product(product_id: int, name: str) -> None:
    with psycopg.connect(DB_URL) as conn:
        with conn.cursor() as cur:
            cur.execute(
                """
                UPDATE products
                SET name = %s::text, version = version + 1
                WHERE id = %s::bigint
                RETURNING id
                """,
                (name, product_id),
            )
            if cur.fetchone() is None:
                raise LookupError(f"Product {product_id} does not exist")

            cur.execute(
                """
                INSERT INTO cache_invalidation_outbox (entity_id, cache_key)
                VALUES (%s::bigint, %s::text)
                """,
                (product_id, f"product:{product_id}"),
            )

A worker invalidates Redis before marking each event processed:

def process_invalidations() -> None:
    with psycopg.connect(DB_URL) as conn:
        with conn.cursor() as cur:
            cur.execute(
                """
                SELECT event_id, cache_key
                FROM cache_invalidation_outbox
                WHERE processed_at IS NULL
                ORDER BY event_id
                LIMIT 100
                FOR UPDATE SKIP LOCKED
                """
            )

            for event_id, cache_key in cur.fetchall():
                with cache.lock(
                    f"lock:{cache_key}", timeout=10, blocking_timeout=5
                ):
                    cache.delete(cache_key)

                cur.execute(
                    """
                    UPDATE cache_invalidation_outbox
                    SET processed_at = now()
                    WHERE event_id = %s::bigint
                    """,
                    (event_id,),
                )

If Redis fails, the exception rolls back the PostgreSQL transaction, leaving the event pending for retry.

Common Pitfalls

The common mistake is adding a cache without specifying its consistency target, invalidation path, TTL, stampede protection, and failure recovery. “The cache is eventually consistent” is an architectural decision with measurable stale-data risk, not an implementation detail.

20

How do cache tags improve selective invalidation?

Cache tags are logical labels associated with cached entries so related items can be invalidated as a group. A tag-to-entry index lets an application remove only entries registered under the affected tag rather than flushing or scanning the entire cache; the trade-off is extra index memory and coordination between data changes and invalidation.

Read full answer

Applies toall versions for the concept; Redis 7.x standalone with redis-py 5.x for the example

How Cache Tagging Works

An application associates each cache key with one or more logical tags, either in cache metadata or in a reverse index such as tag:product:123 -> {cache keys}. When product 123 changes, the application invalidates that tag’s members while preserving unrelated entries.

Tags are usually an application-level convention, not a universal cache feature. Redis SCAN with MATCH can enumerate keyspace entries, but it is not a native tag abstraction. Memcached has no native tags, so applications must maintain and operate their own indexes.

Benefits and Example

  • Selective invalidation: Removes entries registered under one tag without deleting unrelated cached data.

  • Group invalidation: Invalidates every cached representation associated with an entity, tenant, region, or schema version.

  • Better observability: Tag-level counts can identify high-volume entities or endpoints.

  • Higher hit ratio: Unrelated hot entries remain available after a targeted invalidation.

  • Simpler change handling: A write event can target all affected cache keys through one logical name.

For example, a product summary and recommendation response can both carry product:123. Updating that product invalidates both entries. A category page tagged only with category:electronics remains cached; if it embeds the changed product, it must also carry an appropriate version or product tag, accepting the broader invalidation cost.

Invalidation Risks and Mitigations

  • Failed invalidation: If the database write succeeds but invalidation fails, stale data remains until its TTL expires. Use observable, idempotent invalidation with bounded retries, exponential backoff, and jitter, or publish reliable change events through an outbox.

  • Stale repopulation: An old request can write an old value after invalidation. Version tokens, compare-and-set logic, or request coalescing can prevent this race.

  • Missing or broad tags: Missing tags leave stale entries, while broad tags unnecessarily reduce the hit ratio. Define tag ownership and cardinality explicitly.

  • Orphaned indexes and memory overhead: Expired entries can leave stale index references. Expire indexes and remove reverse references.

  • Stampedes: When one popular key expires, use request coalescing, a bounded lock, or early refresh with randomized jitter.

TTL expiry remains a safety net, not a replacement for a correct invalidation plan.

Strategies and Engine Choices

Tags are orthogonal to the cache-fill strategy:

  • Cache-aside: Read from cache, then load and tag the value from the database on a miss.

  • Write-through: Write synchronously to cache and the backing store, then maintain the entry’s tags.

  • Write-back: Write to cache first and persist later; tagging helps targeted eviction, but the stale-data window is larger.

  • Write-around: Write directly to the backing store and bypass cache for that write, commonly for write-heavy or one-off data.

TTL expiry, LRU, LFU, and FIFO eviction solve capacity and lifetime problems; tags solve group invalidation and do not replace them.

Area Redis Memcached
Data structures Strings, hashes, lists, sets, sorted sets, streams, and others Opaque key-value strings with flags and expiry
Persistence Optional RDB snapshots and AOF No built-in persistence
Replication Built-in asynchronous primary-replica replication; cluster sharding is also available No built-in replication; clients normally shard across nodes
Execution Command execution is mainly single-threaded, with optional I/O and background threads Multithreaded server

Code Example

This example assigns one tag per entry for simplicity and uses a Redis set as the tag index:

from redis import Redis
from redis.exceptions import WatchError

client = Redis(host="localhost", port=6379, decode_responses=True)


def cache_value(key: str, value: str, tag: str, ttl: int = 300) -> None:
    tag_key = f"tag:{tag}"
    pipe = client.pipeline(transaction=True)
    pipe.setex(key, ttl, value)
    pipe.sadd(tag_key, key)
    pipe.expire(tag_key, ttl + 60)
    pipe.execute()


def invalidate_tag(tag: str) -> int:
    tag_key = f"tag:{tag}"

    while True:
        try:
            with client.pipeline() as pipe:
                pipe.watch(tag_key)
                keys = pipe.smembers(tag_key)
                pipe.multi()
                if keys:
                    pipe.delete(*keys)
                pipe.delete(tag_key)
                pipe.execute()
                return len(keys)
        except WatchError:
            continue  # A concurrent tag update changed the snapshot; retry.


try:
    cache_value("product:123:summary", '{"name":"Desk lamp"}', "product:123")
    cache_value("product:123:recommendations", '["product:456"]', "product:123")
    invalidate_tag("product:123")
finally:
    client.close()

The watched transaction prevents a concurrent tag update from being lost during invalidation, but production cache-aside code must still prevent a stale request from repopulating an invalidated value.

When Not to Use It

Tags add little value for a small, short-lived cache where flushing everything is inexpensive, or when entries have nearly identical expiry and invalidation policies. They are also unnecessary when the backing store can meet the required latency directly.

Do not place sensitive per-user responses in a shared cache unless the key, authorization boundary, and invalidation scope prevent cross-user access. The common interview mistake is adding tags without defining who updates them, how invalidation failures are handled, and how stale requests are prevented.

21

How does distributed caching compare with local caching?

Distributed caching stores reusable data in a shared backend that multiple application instances can access. Compared with local caching, it enables cross-instance reuse but adds network latency, invalidation complexity, and another failure dependency.

Read full answer

Applies toall versions of the caching concepts; Redis 8.x and Memcached 1.6.x for the tool comparison

How It Works

A typical request path is:

Client → Load balancer → API service → Distributed cache → Database

On a miss, the service reads the database and populates the cache. Later requests can reuse the value. For example, a news API can cache article metadata by article ID so several API instances share one entry instead of maintaining separate local copies.

Distributed vs Local Caching

Aspect Local cache Distributed cache
Scope One process or host Multiple hosts and services
Lookup Direct memory access Network request
Typical latency Tens to hundreds of nanoseconds Roughly 0.2–2 ms within a datacenter
Capacity Limited per instance Shared pool; horizontally scalable
Consistency Private to one instance; may still be stale Shared entries; replication and invalidation can still produce staleness
Failure impact Usually one instance Can affect the application when required for every request
Operational cost Low Deployment, monitoring, replication, and failover
Best fit Immutable, instance-specific, or very hot data Frequently reused data across instances or services

A distributed cache helps when instances share expensive results or per-instance memory would be wasteful. A local cache is faster and can provide graceful degradation: when the distributed cache is unavailable, an instance may serve a sufficiently fresh local entry or bypass the cache.

Strategies and Invalidation

  • Cache-aside: The application checks the cache, loads from the database on a miss, and then populates the cache. The database remains authoritative.

  • Read-through: A cache client loads missing data from a configured backing store. This simplifies application code but couples the client to the data source.

  • Write-through: Writes synchronously update both cache and backing store, providing consistent reads at the cost of slower writes and more complicated failure handling.

  • Write-back: The cache is updated first and persisted later. This reduces database write pressure but can lose data if the cache fails before persistence.

  • Write-around: Writes bypass the cache, preventing them from evicting frequently read entries; a later read populates the cache.

After a successful database write, the application should update or delete the affected cache key, or publish a change event for a consumer to handle. Because invalidation messages can be delayed or lost, an appropriate TTL should remain a safety net.

Failure Modes and Eviction

  • Stale data: A cached value outlives a database update. Use explicit invalidation, versioned keys, and TTLs based on the acceptable staleness window.

  • Cache stampede or thundering herd: Concurrent misses query the database together. Use request coalescing, a short-lived distributed lock, stale-while-revalidate, or early recomputation with jitter.

  • Cache penetration: Requests repeatedly seek nonexistent values. Cache negative results briefly or use a Bloom filter for a large key space.

  • Hot keys: One key receives excessive traffic. Use a local cache in front of it, partition its value, or distribute reads across replicas.

  • Eviction pressure: When capacity is full, LRU favors recently used entries, LFU favors frequently used entries, and FIFO removes entries in insertion order. TTL is expiration rather than an eviction policy.

  • Cache loss: Persistence and replication reduce some failure risks, but a cache should not be the only durable copy of critical data.

Redis vs Memcached

Capability Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and others Mostly opaque key-value bytes
Persistence Optional RDB snapshots or AOF append-only logging None; data is in memory
Replication Built-in asynchronous primary-replica replication No built-in server replication; managed services may add failover
Concurrency Main command execution is single-threaded; I/O and background threads handle other work Multi-threaded server
Typical use Caching, sessions, queues, counters, locks, and pub/sub Simple, large-scale object caching

Redis is the more versatile option. Memcached remains effective when the requirement is straightforward distributed caching with minimal server-side features.

Best Practices

  • Measure hit ratio, miss latency, evictions, memory use, and backend load; a high hit ratio alone does not prove the cache is worthwhile.

  • Derive TTLs from change frequency and business tolerance for staleness, not from habit.

  • Avoid caching sensitive per-user data in a shared cache unless authorization, key isolation, encryption, and expiry are properly controlled.

  • Apply a bounded timeout to cache calls and decide whether the application should bypass the cache, serve stale data, or fail.

  • Use cache-aside for most services and combine explicit invalidation with TTL expiry.

  • Do not add a distributed cache when local caching already covers the workload or when data is rarely reused.

Interviewers commonly look for a clear invalidation strategy and defined failure behavior. The most common mistake is adding a cache without deciding who updates it, how long entries live, and what happens when the cache is unavailable.

22

How should you partition a cache across distributed nodes?

Cache partitioning divides a cache’s keyspace across multiple nodes so no single node stores or serves every entry. It improves memory capacity and request throughput, but introduces routing, rebalancing, invalidation, and failure-management complexity.

Read full answer

Applies toall versions

How Cache Partitioning Works

A typical request path is:

Client → hash/router → cache partition → database

Each key is assigned to a partition using a deterministic function of its partition key. For example, an e-commerce cache might partition by tenant_id + product_id, while a user-profile cache might use user_id.

  • Key distribution: Choose keys so entries are spread evenly across nodes.

  • Routing: The client, cache library, or proxy calculates the owning node and sends the request there.

  • Replication: Each partition can have replicas for availability, although replicas do not solve a single hot key.

  • Capacity: Estimate key count × average value size, add memory overhead and headroom, then divide the result across nodes.

  • Isolation: One unhealthy or overloaded partition should not exhaust the entire cache.

A partition key should reflect access patterns. Using tenant_id enables tenant-level isolation, but one large tenant could dominate a partition. In that case, use a compound key such as tenant_id + shard_number.

Partitioning Strategies

Strategy Benefits Limitations
Fixed hash modulo Simple and fast Changing the node count can remap many keys and cause a cache-miss storm
Consistent hashing Moving a node remaps only part of the keyspace More routing complexity; virtual nodes improve balance
Range partitioning Useful for ordered or range queries Can create hot ranges and requires careful rebalancing
Rendezvous hashing Good balance and limited disruption when membership changes More client or router computation
Directory-based routing Flexible mapping and controlled migrations Requires a highly available routing directory

Redis Cluster and Memcached differ in their native partitioning and operational models:

Aspect Redis Cluster Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and other Redis types Flat string or byte values; no built-in collections
Persistence Optional RDB snapshots and AOF; durability depends on configuration No built-in persistence
Replication Asynchronous primary-replica replication and cluster failover No built-in replication; clients or proxies must provide it when needed
Threading Command execution is mainly single-threaded; optional I/O threads and background persistence work do not make command execution generally multi-threaded Multi-threaded, event-driven server with worker threads
Partitioning Uses 16,384 fixed hash slots and supports slot migration during resharding Has no native cluster protocol; clients or proxies normally shard keys

Consistency and Caching Strategies

Cache-aside is usually the best default for partitioned caches: the application reads the cache, loads a miss from the database, and populates the cache. This keeps the cache out of the critical path, but invalidation becomes an application responsibility.

Common alternatives have different operational costs:

Strategy Partition behavior Main trade-off
Cache-aside Each partition loads misses on demand Requires explicit invalidation
Read-through The cache library loads misses Convenient but hides the source-of-truth dependency
Write-through Writes update cache and database together Lower read latency, but cache failure can block writes
Write-back Writes update cache and persist later Faster writes, but data-loss risk after a crash
Write-around Writes bypass the cache Useful for rarely read data; subsequent reads still miss

Use TTLs as a safety net, not as the only invalidation mechanism. To invalidate safely, delete or update entries in the owning partition, publish invalidation events, and use versioned keys or tokens when stale reads are unacceptable. TTLs also control memory and recover entries after missed invalidations.

Hot Keys and Failure Handling

A hot key receives disproportionate traffic and can overload one partition. Adding replicas helps with reads, but all replicas still share the same write and invalidation pressure. Better mitigations include:

  • Splitting a hot entity into independently cached fields or time windows.

  • Precomputing popular results and serving them from a CDN or local cache.

  • Using request coalescing so only one worker loads an expired value.

  • Refreshing popular entries before expiry, with jitter to prevent synchronized stampedes.

  • Applying circuit breakers, bounded timeouts, and bulkheads around cache calls.

On node failure, fail over to a replica or bypass the cache and read the source of truth. Rate-limit fallback reads to prevent a cache outage from becoming a database outage.

Protect against cache penetration with short-lived negative caching and, where appropriate, a Bloom filter. Do not store sensitive per-user data in a shared cache unless entries are isolated and access-controlled.

Operational Practices and Trade-offs

  • Measure hit ratio by key class, not only globally: hits / (hits + misses).

  • Track partition skew, latency, evictions, expired entries, replication lag, and fallback traffic.

  • Use workload-based load testing; a balanced key count does not guarantee balanced traffic.

  • Distinguish eviction from expiry: Redis supports configurable LRU and LFU policies, while Memcached uses LRU only; FIFO is common in other caches. TTL expiry makes entries unavailable, but physical memory reclamation depends on the cache’s expiry and eviction behavior.

  • Keep rebalancing tools and ownership metadata available, but avoid large migrations without request draining or rate limits.

  • Avoid partitioning a small cache until measurements justify the additional network and operational overhead.

The common interview mistake is choosing a hash function without defining rebalancing, invalidation, and failure behavior. Interviewers usually look for whether you can explain why a particular key becomes hot and how the system degrades safely when one partition or the cache as a whole is unavailable.

23

What consistency models apply to distributed caching?

Consistency models in distributed caching define which stale reads clients may observe after an update, both among cache replicas and between the cache and its source of truth. Stronger guarantees reduce staleness but add coordination, latency, and failure-management cost.

Read full answer

Applies todistributed caching models across all versions; Redis 8.x and Memcached 1.6.x are implementation examples

Common Consistency Models

Model Guarantee and cost
Linearizable or strict consistency Operations appear atomic and instantaneous; a read started after a write completes observes that write. Multi-node implementations require quorum or consensus coordination and may reject operations during a partition.
Sequential consistency All clients observe one total order consistent with each client’s program order, although that order need not match real time. It avoids strict real-time ordering but can still expose older values.
Causal consistency All clients agree on the order of causally related operations, while concurrent operations may appear in different orders. Version or dependency metadata adds complexity.
Eventual consistency Replicas converge after updates stop and messages are eventually delivered, but there is no guaranteed convergence time. It provides fast writes and good availability at the cost of temporary staleness.
Bounded staleness Reads return data no older than a defined bound. Logical versions or leases can enforce this, so synchronized clocks are not required.
Client- and session-centric Monotonic reads prevent one client from observing an older value after a newer one; read-your-writes guarantees that a client sees its own update. These are not global replica-consistency levels.

These guarantees do not arise automatically from a cache product. The update, invalidation, replication, and retry protocols determine the actual behavior.

Source of Truth and Write Strategies

A typical cache-aside flow is:

  1. Read the cache.

  2. On a miss, read the database.

  3. Populate the cache.

  4. Commit the database write and invalidate or update the cache.

Other strategies have different operational costs:

  • Cache-aside: The application loads misses and controls fallback behavior. It is a general-purpose default but creates an invalidation problem.

  • Read-through: The cache loads missing entries from the backing store itself. This simplifies application code but requires cache integration.

  • Write-through: The cache and backing store are updated synchronously before success is acknowledged. Writes have higher latency, and partial failures require reconciliation.

  • Write-back: The cache is updated immediately and the backing store is persisted asynchronously. It supports high write rates but can lose acknowledged writes after a crash.

  • Write-around: A write bypasses the cache and goes directly to the backing store. It prevents cache pollution but usually leaves the next read as a miss.

Invalidation before the database commit can be undone by a concurrent reload of the old value; invalidation after the commit leaves a stale-read window. A transactional outbox can publish at-least-once invalidation events reliably, provided consumers are idempotent. Versioned values with atomic compare-and-set can prevent an older loader from overwriting a newer value. TTLs bound staleness but are not a complete invalidation strategy.

Availability and Failure Trade-offs

Linearizable multi-node caching generally requires quorum or consensus coordination. Under a network partition, CAP forces a trade-off per request: a CP implementation may refuse operations that cannot meet the required quorum, while an AP implementation may serve stale data. Under normal operation, PACELC adds the trade-off between latency and consistency.

When the database is authoritative, a cache outage can be handled by reading the database with a timeout, rate limiting, circuit breakers, and bulkheads. The database must be sized for this degraded path. Retries should use exponential backoff and jitter; blindly retrying can extend an outage.

Eviction, Hot Keys, and Thundering Herds

  • Eviction: Use TTL for freshness limits and LRU, LFU, or FIFO for capacity pressure. Distributed implementations may approximate these policies. Avoid one policy everywhere because access patterns differ.

  • Hot keys: Replicate read-heavy keys and add a small local cache. Sharding cannot distribute one unchanged key across multiple shards.

  • Cache stampede: Prevent many requests from loading an expired key simultaneously with request coalescing, carefully designed distributed locks, early refresh with randomized jitter, or bounded stale-while-revalidate.

  • Cache penetration: Repeated missing keys can overwhelm the origin. Cache negative results with a short TTL and consider a Bloom filter when the key set is known in advance.

Choosing a Consistency Level

Use strong consistency for payments, inventory reservations, permission revocations, and other decisions where stale data is unsafe. Eventual consistency fits product descriptions, articles, and public profiles; bounded staleness fits data with an explicit freshness limit. Checkout prices, for example, may be approximately cached for display but must be revalidated authoritatively at purchase.

Do not cache one-off reads, data with extreme freshness requirements, or anything the origin cannot serve during misses. Per-user sensitive data belongs in an isolated, access-controlled namespace with strict expiry, never in a generic shared-cache key.

Measure hit ratio, staleness, invalidation lag, and origin load together. A 95% hit ratio is not useful if it violates the required freshness guarantee.

Aspect Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and other structures Primarily opaque key-value entries
Persistence Optional RDB snapshots and AOF No built-in persistence
Replication Built-in replication, normally asynchronous No built-in replication; clients normally shard data
Threading Command execution remains mainly single-threaded per instance Multi-threaded request-serving model

A common interview mistake is adding a cache without specifying its source of truth and invalidation protocol. Interviewers usually want to hear which reads may be stale, how long staleness lasts, how failures degrade, and how stampedes are prevented.

24

How does cache replication work in distributed systems?

Cache replication maintains copies of cached data on multiple cache nodes so reads can be served closer to clients and data remains available after some failures. It reduces read latency only when clients or a proxy distribute reads across replicas; replication itself normally stores the same keys on every replica rather than dividing the keyspace.

Read full answer

Applies todistributed cache systems; Redis 8.x and Memcached 1.6.x implementation notes

Why Cache Replication Exists

Replication addresses two distinct problems:

  • Latency: an in-process memory lookup is roughly tens to hundreds of nanoseconds, while a same-datacenter network request commonly costs around 0.5 ms. Cross-region requests can take roughly 50–150 ms.

  • Failure tolerance: another node may serve cached data when the preferred node fails, provided the implementation has coordinated failover and the replica has received the required data.

Replication is distinct from sharding:

  • Sharding divides the keyspace among nodes; each key has one owner.

  • Replication stores the same data on additional nodes.

  • Production clusters commonly combine them: shards distribute capacity and load, while replicas provide redundancy.

Replication adds network traffic, memory use, operational complexity, and potentially stale reads. It is unnecessary for small workloads where a local cache is sufficient or where strict consistency makes replicated reads unacceptable.

How Replication Works

A typical replicated cache follows this lifecycle:

  1. One node is the primary and owns writes for its keys.

  2. A new replica connects to the primary and performs an initial synchronization, usually by bulk-transferring the dataset.

  3. After synchronization, the primary streams incremental changes to the replica. The replica maintains an offset or equivalent position so it can detect gaps.

  4. Clients read from the primary or, when staleness is acceptable, from a replica. A load balancer can use health and lag information to select a target.

  5. If the primary fails, a coordinated mechanism elects and promotes a sufficiently current replica, updates discovery information, and redirects writes. Without such coordination, replication alone does not provide automatic failover.

Replication may be engine-managed, proxy-managed, application-managed, or supplied by a messaging/change-data-capture pipeline. CDNs and reverse proxies also maintain geographically distributed copies, but these are usually independently populated caches rather than primary-replica replicas of one cache dataset.

Consistency and Failure Trade-Offs

Approach Benefits Costs and risks
Asynchronous replication Lower write latency and better tolerance of replica outages Replicas may be stale; unacknowledged writes can be lost during failover
Synchronous replication Smaller lost-write window Higher write latency; writes may be rejected when required replicas are unavailable
Application-level replication Flexible across regions and providers More code, ordering concerns, and operational responsibility

Replica reads should therefore have an explicit staleness policy. Common choices include primary-only reads, replica reads with a bounded lag, short-lived read-your-writes routing, and monotonic reads that prevent a client from moving to an older version.

In Redis, replication is asynchronous. Redis Sentinel and Redis Cluster provide coordinated promotion, while a plain primary-replica pair does not. In a cache-aside system, the database remains authoritative; a database update must trigger cache invalidation. A DEL sent to a Redis primary is itself replicated, but changes made only in the database require application logic, CDC, or another invalidation channel.

Cache Write Strategies

Strategy Behavior Typical fit
Cache-aside The application loads missing data and populates the cache General-purpose caches and read-heavy services
Read-through The cache calls a loader and populates itself on a miss Encapsulated cache access with standard read semantics
Write-through Cache and backing store are updated synchronously Read-after-write requirements where added write latency is acceptable
Write-back The cache updates asynchronously and persists later High write rates where some update loss is acceptable
Write-around A write bypasses the cache and goes directly to the store Write-once/read-many workloads where cached writes would be short-lived

Write-back reduces origin load but complicates durability and replication. Write-around can avoid polluting the cache with entries that will not be read soon, but the subsequent read may still miss.

Redis and Memcached

Capability Redis 8.x Memcached 1.6.x
Built-in replication Yes, asynchronous replication; Sentinel or Cluster coordinates promotion No
Data structures Strings, hashes, lists, sets, sorted sets, streams, and others Primarily byte-limited key-value items
Persistence Optional snapshots and append-only persistence Memory-only
Command execution Primarily single-threaded command execution, with background and optional I/O threads Multithreaded
Cluster behavior Hash-slot sharding with replicas No built-in cluster protocol; clients or external infrastructure partition keys
Typical fit Shared state, queues, rate limiting, sessions, and replicated caching Simple distributed caching with high request concurrency (items are limited to 1 MB by default)

Memcached replication must be implemented by application logic or external infrastructure. This keeps the server simple but makes consistency and recovery the application’s responsibility.

Best Practices and Failure Handling

  • Design invalidation first: define the owner, update path, and expiry or invalidation rule for every cached representation.

  • Use TTLs as a safety net: they bound stale-data lifetime, but simultaneous expiry can create a cache stampede.

  • Protect hot keys: use request coalescing, single-flight locking, jittered early refresh, and replicas for read distribution.

  • Prevent cache penetration: cache negative results briefly and consider a Bloom filter for large sets of nonexistent identifiers.

  • Choose eviction deliberately: LRU suits recency, LFU suits frequency, FIFO favors simplicity, and TTL expiration bounds lifetime.

  • Isolate sensitive data: avoid putting per-user or authentication data in a shared cache without explicit access controls, isolation, and transport protection.

  • Monitor replication lag, hit ratio, evictions, and origin load: a poor hit ratio can add latency and cost without reducing database traffic.

  • Test partitions and failover: verify stale-read limits, possible lost writes, promotion delay, and whether origin traffic remains bounded.

25

How do you maintain cache coherence in distributed systems?

Cache coherence means preventing cached data from remaining inconsistent with its authoritative source after a write. Distributed systems usually combine explicit invalidation, update propagation, version checks, and bounded TTLs rather than expecting instantaneous consistency across every cache replica.

Read full answer

Applies toall versions; PostgreSQL and Apache Kafka are illustrative only.

Core Strategies

Strategy How it works Trade-off
Cache invalidation Delete or expire cached entries after the source changes Simple, but invalidation messages can be delayed, duplicated, or lost
Write-through Update the cache and backing store synchronously Keeps entries fresh, but increases write latency and couples cache availability to writes
Write-back Update the cache asynchronously and persist later Fast writes, but creates stale-data and data-loss windows
TTL-based expiry Refresh entries after a time-to-live Provides bounded staleness, but consumes memory and still permits temporary staleness
Event-driven invalidation Publish change events through Kafka or similar infrastructure Scales well, but introduces eventual consistency and processing dependencies
Versioned keys Include a data version in the key, such as product:42:v17 Avoids version collisions, but requires a reliable version source and leaves obsolete keys until expiry

Recommended Write Path

A common design is cache-aside:

  1. Read from the cache.

  2. On a miss, read the authoritative database.

  3. Populate the cache with a TTL.

  4. After a successful database commit, delete the affected cache entry.

For stronger delivery guarantees, use the outbox pattern: store the domain change and an invalidation event in the same database transaction, then publish the event asynchronously. This avoids the failure where the database update succeeds but the process crashes before invalidating the cache.

For example, an e-commerce service can update a product in PostgreSQL and invalidate product:42. If deletion fails, a short TTL bounds the stale-data window.

Preventing Concurrent Read Problems

Invalidation alone does not prevent concurrent requests from loading the same expired entry.

  • Request coalescing: Ensure only one request loads a missing entry while others wait for or reuse its result.

  • Early expiry: Refresh popular entries shortly before their TTL ends.

  • Jittered expiration: Stagger expirations to avoid synchronized stampedes.

  • Locking: Use a narrowly scoped distributed lock when expensive work must run once; prefer local coalescing when possible.

  • Negative caching: Cache “not found” results briefly to reduce repeated misses.

  • Bloom filter: Reject likely nonexistent keys before querying the database, reducing cache penetration.

  • Hot-key replication: Use cache replicas or secondary nodes when supported, and coalesce requests for the hot key.

Controlling Consistency

Choose the required consistency level before selecting a strategy:

  • Strong consistency: Read the authoritative source or use a coordinated write-through path. Expect higher latency and lower availability during dependency failures.

  • Eventual consistency: Use asynchronous invalidation or short TTLs. This is often appropriate for product catalogs, news feeds, and recommendations.

  • Monotonic reads: Use versioned keys or revision checks to prevent an older response from overwriting a newer cached value.

Do not cache data where staleness causes unacceptable harm—such as authorization decisions, account balances, inventory reservations, or payment state—unless the design includes explicit transactional coordination.

Code Example

The following example assumes a PostgreSQL table items(id bigint primary key, payload jsonb) and uses Redis 7.4 with redis-py 5.x, PostgreSQL 16, and Python 3.12:

import json
import logging

import psycopg
from psycopg_pool import ConnectionPool
import redis

logger = logging.getLogger(__name__)


def get_item(
    item_id: int,
    pool: ConnectionPool,
    cache: redis.Redis,
) -> dict | None:
    key = f"item:{item_id}"
    cached = cache.get(key)
    if cached is not None:
        return json.loads(cached)

    with pool.connection() as conn:
        with conn.cursor() as cur:
            cur.execute(
                "SELECT payload::text FROM items WHERE id = %s::bigint",
                (item_id,),
            )
            row = cur.fetchone()

    if row is None:
        return None

    item = json.loads(row[0])
    cache.set(key, json.dumps(item), ex=60)
    return item


def update_item(
    item_id: int,
    payload: dict,
    pool: ConnectionPool,
    cache: redis.Redis,
) -> None:
    key = f"item:{item_id}"

    with pool.connection() as conn:
        with conn.cursor() as cur:
            cur.execute(
                """
                UPDATE items
                SET payload = %s::jsonb
                WHERE id = %s::bigint
                RETURNING id
                """,
                (json.dumps(payload), item_id),
            )
            if cur.fetchone() is None:
                raise ValueError("item does not exist")
        conn.commit()

    try:
        cache.delete(key)
    except redis.RedisError:
        # The TTL bounds staleness if the cache is temporarily unavailable.
        logger.exception("cache invalidation failed for %s", key)

Operational Trade-offs and Best Practices

Caching improves read latency and reduces database load, but adds invalidation logic, memory cost, monitoring, and failure modes. Different requests may observe different versions, making stale-cache incidents harder to diagnose than ordinary database latency.

Specify who invalidates each key, how quickly it may become stale, and what happens when an invalidation message is lost. Use TTLs as a safety net, monitor hit ratio and invalidation failures, and fall back to the authoritative data source when the cache is unavailable. Do not add a cache without an invalidation plan.

26

How to reduce network latency in distributed caching?

Reducing distributed-cache latency means shortening or eliminating network round trips between callers and cached data. The main trade-off is stronger locality and throughput versus stale data, additional memory, and more complex invalidation.

Read full answer

Applies toRedis 8.x with redis-py 6.x on Python 3.10+ and Memcached 1.6.x; other techniques are product-independent

The Latency Path

A distributed-cache hit is still a network call. Rough orders of magnitude are local memory at ~100 ns, SSD at ~100 µs, same-datacenter round trip at ~0.5 ms, and cross-region at ~50–150 ms.

A typical path is:

Client → CDN/reverse proxy → application → L1 cache → distributed cache → database

  • Edge caches: Browser, CDN, and reverse-proxy caches can remove origin round trips. HTTP Cache-Control: max-age controls freshness, s-maxage targets shared caches, and ETag supports conditional revalidation, although an ETag check can still require a request.

  • Application cache: An in-process or on-host L1 cache removes the distributed-cache hop for hot data.

  • Distributed cache: A shared Redis or Memcached tier removes database work, but the application still pays network latency.

  • Database caches: Buffer and query caches reduce origin work; they do not remove the application-to-cache round trip.

Techniques With the Largest Effect

  • Use proximity: Place cache nodes in the same availability zone as callers and use topology-aware routing. Route to an in-zone replica only when its staleness and failover behavior are acceptable.

  • Reduce call count: Combine related reads with batch commands or pipelines. In Redis Cluster, MGET requires all keys in the same hash slot; use hash tags or a cluster-aware pipeline for cross-slot keys.

  • Coalesce concurrent misses: When many requests miss the same hot key, let one request rebuild the value while the others wait for its result. This prevents a cache stampede and duplicate database traffic.

  • Reuse connections: Use a connection pool and keep-alive. Opening TCP connections and performing handshakes for every lookup can cost more than the cache operation.

  • Reduce payload size: Fetch only required fields, compress large values when CPU permits, and avoid serializing unused data.

  • Handle hot keys deliberately: A nearby node does not solve one key receiving most traffic. Combine request coalescing, rate limiting, key partitioning, or read replicas while accounting for stale reads.

Cache Strategy and Consistency

Strategy Behaviour Latency and consistency trade-off
Cache-aside Application loads the origin on a miss Flexible and read-friendly; a miss adds another round trip
Read-through Cache loads the origin itself Keeps origin access inside the cache abstraction
Write-through Cache and origin are updated synchronously Faster subsequent reads; writes are slower and partial failures need handling
Write-back Cache is updated first and persisted asynchronously Fast writes, but cache failure can lose recent writes
Write-around Write goes to the origin without populating the cache Avoids cache pollution; the next read may miss

The database or other system of record normally remains authoritative. Choose invalidation by explicit deletion, change events, versioned keys, or bounded TTLs with jitter. Eviction may use LRU, LFU, FIFO, or TTL according to access patterns.

Protect against penetration with short-lived negative caching and, at larger scale, a Bloom filter. Protect against stampedes with per-key locking or request coalescing, plus early recomputation with jittered TTLs.

Redis Versus Memcached

Concern Redis 8.x Memcached 1.6.x
Data model Strings, hashes, sets, sorted sets, streams, and others Simple key-to-value storage
Persistence Optional RDB and AOF persistence No built-in persistence
Replication Built-in replication and Redis Cluster No built-in replication or cluster protocol
Concurrency Mostly single-threaded command execution, with I/O and background threads Multi-threaded server
Operations More capabilities and memory overhead Simpler and often smaller per item

Choose Redis when richer data, persistence, or replication is required; choose Memcached for a simple shared object cache.

Code Example

import asyncio

from redis.asyncio import Redis


async def main() -> None:
    client = Redis.from_url(
        "redis://localhost:6379/0",
        decode_responses=True,
        socket_connect_timeout=1.0,
        socket_timeout=1.0,
    )

    try:
        # These tags also place the keys in one Redis Cluster slot.
        keys = ["{feed}:1", "{feed}:2", "{feed}:3"]

        for _ in range(3):
            # One command and round trip instead of three GET round trips.
            # The client reuses its pooled connection across iterations.
            values = await client.mget(keys)
            print(values)
    finally:
        await client.aclose()


asyncio.run(main())

Trade-Offs and Best Practices

  • Use caching when repeated reads are expensive or common, as with product pages on an e-commerce site. Avoid it when the source lookup is already cheaper than the extra cache round trip.

  • Do not share sensitive per-user data without isolation and appropriate access controls. Do not use L1 caching for strict read-after-write requirements or rapidly changing values.

  • Plan failure behavior: Fall back to the database, bound waits with timeouts, and use circuit breakers when the cache is unavailable.

  • Measure the right metric: Track cache-call count, payload size, hit ratio, and p50/p99 latency separately for application-to-cache and application-to-database traffic. Proximity helps RTT; batching and coalescing help call count.

27

What are the challenges of a distributed cache?

A distributed cache shares reusable data across multiple nodes to scale beyond one machine’s memory and, with replication and failover, tolerate node loss. Its main challenges are consistency and invalidation, hot keys, stampedes, network failures, memory pressure, and operational complexity.

Read full answer

Applies toall distributed-cache versions; Redis 8.x and Memcached 1.6.x for the tool comparison

Why Distribution Adds Problems

A local memory read is roughly 100 ns, a same-datacenter round trip roughly 0.5 ms, and a cross-region round trip roughly 50–150 ms. These are rough orders of magnitude. Distributed lookups also add serialization, routing, monitoring, rebalancing, and partial-failure handling.

During a network partition, CAP requires choosing between refusing operations that need current data and serving potentially stale data. Most caches deliberately provide eventual consistency; PACELC also captures the latency/consistency trade-off outside partitions.

Consistency and Invalidation

  • Ownership: For every cached entity, identify the authoritative writer and how that writer invalidates other copies. Versioned keys work only when the current version can be obtained reliably.

  • Invalidation: Explicit invalidation messages can be delayed or lost. TTLs bound staleness but do not provide immediate invalidation, so choose them according to the acceptable stale window.

  • Cache-aside: The application loads missing data from the backing store and then caches it. This is common, but misses and stampedes can overload the store.

  • Read-through: The cache loads missing data itself. This simplifies application logic but couples the cache layer to backing-store access.

  • Write-through: The cache and backing store are updated synchronously. Reads remain warm, but write latency and failure handling become more complex.

  • Write-back: The cache is updated first and the backing store later. Writes are fast, but failures create a data-loss and reconciliation window.

  • Write-around: Writes go to the backing store while the cache is invalidated or bypassed. This avoids caching write-once data, but subsequent reads can miss.

Load and Failure Challenges

  • Hot keys: A trending product or shared homepage feed can overload one shard. Replication or read replicas help, but consume extra memory and can return stale replicas.

  • Cache stampede: When a popular key expires, concurrent misses may overload the database. Use per-key locking, request coalescing, or early refresh with TTL jitter, while considering the coordination overhead of each mitigation.

  • Cache penetration: Repeated requests for nonexistent keys bypass the cache. Short-lived negative caching can hide newly created objects until expiry. A Bloom filter can reject most nonexistent lookups, but it has false positives and no false negatives.

  • Partial failure: Use timeouts, bounded retries with exponential backoff and jitter, circuit breakers, bulkheads, and health checks. Retried operations that cause side effects need idempotency keys or equivalent deduplication.

Memory, Security, and Operations

  • Eviction: Under memory pressure, policies include LRU, LFU, FIFO, and TTL-based expiry. The best policy depends on access patterns, and eviction can reduce the hit ratio.

  • Availability versus durability: Replication and failover target availability; persistence affects recovery after restart. Neither makes the cache the source of truth.

  • Sensitive data: Do not cache raw credentials or authorization tokens. Per-user or tenant data on a shared cache requires strict key scoping, access control, encryption in transit, and encryption at rest when persistence is enabled.

Layers and Tool Choices

Caching can occur at several layers: browser and private caches use directives such as Cache-Control and max-age; CDN and reverse-proxy caches can use s-maxage and ETag; application-local caches avoid network calls; distributed caches share data across instances; and database query or buffer caches reduce database I/O.

Aspect Redis 8.x Memcached 1.6.x
Data structures Strings, hashes, lists, sets, sorted sets, streams, and probabilistic structures Primarily opaque byte values with counters and expiry
Persistence Optional RDB snapshots and AOF No built-in disk persistence
Replication Built-in replication with Sentinel, Cluster, or managed failover No built-in replication or failover
Threading Command execution is largely single-threaded; I/O can use additional threads Uses worker threads to serve clients concurrently

Practical Guidance

  • Measure hit ratio by endpoint and workload. Do not chase 100%: long TTLs can increase hit ratio while worsening staleness.

  • Define the acceptable stale window before selecting TTLs, and decide whether stale reads are preferable to cache failure.

  • Treat a cache miss as an expected event, not an exception; protect the backing store with coalescing, jittered expiry, and bounded concurrency.

  • Do not add a distributed cache when traffic is low, a local cache is sufficient, strict consistency is mandatory, or the team cannot operate replication, monitoring, and failover.

  • A common interview mistake is adding caching without an invalidation owner and failure policy.

28

How does a write-through cache work in distributed systems?

A write-through cache synchronously updates both the cache and its backing store before acknowledging the write. This reduces stale reads after successful writes, but the two updates are not inherently atomic; the extra network hop and failure handling increase latency and complexity.

Read full answer

Applies toall versions

How It Works

The typical request flow is:

Client → Application service → Cache → Database

For a write operation:

  1. The service submits the updated value to the cache or write-through layer.

  2. The layer updates the cache and synchronously forwards the write to the backing store. Some implementations update the cache first; others update the backing store first.

  3. The layer acknowledges success only after both required writes complete successfully.

  4. Subsequent reads can be served from the cache.

For example, an e-commerce product service might update a product’s price through a write-through cache. The cache stores the new price and writes it to the primary database before returning success. A later lookup can then be served from the cache.

Some systems coordinate this flow in the application rather than inside the cache. In that case, the application performs both writes and explicitly defines their ordering, retry behavior, and failure semantics.

Distributed Consistency and Failure Handling

A write-through cache does not make the cache and database one atomic data store. A partial failure can leave them inconsistent:

  • Database write succeeds, cache update fails: The database contains the new value while the cache may contain the old value. This can occur when the database is updated first or when the two writes are attempted independently.

  • Cache update succeeds, database write fails: The cache contains the new value while the database retains the old value. This can occur when the cache is updated first.

The exact outcome depends on the implementation’s write ordering and whether a failed step stops the operation. Common mitigations include:

  • Treat the database as the source of truth, then retry the database write or invalidate/update the cache after a failure.

  • Use idempotent updates so retries do not duplicate side effects.

  • Use a transactional outbox for reliable cache-update or invalidation events and later reconciliation.

  • Route writes for the same key through one ordered owner to prevent out-of-order updates.

  • Apply a short TTL as a bounded fallback, not as a replacement for correct invalidation.

  • Allow reads to fall back to the database when the cache is unavailable.

Cross-region writes can cost roughly 50–150 ms per round trip, while a same-datacenter round trip may cost around 0.5 ms. Sequential cache and database operations can therefore add material latency.

Comparison With Other Cache Strategies

Strategy Write path Read path Main trade-off
Write-through Synchronously updates cache and backing store before acknowledgment Usually from cache Lower stale-read risk, but higher latency and failure coupling
Write-back Updates cache and acknowledges before the delayed backing-store write From cache; writes are delayed Faster writes, but data loss is possible if the cache fails
Cache-aside Writes to the database and invalidates or updates the cache separately Cache, falling back to the database Simple and flexible, but requires an invalidation plan
Write-around Writes to the database and invalidates any existing cache entry Cache, falling back to the database Avoids caching write-only values, but causes extra misses
Read-through Not a write policy; on a miss, the cache loads from the database Cache, loading misses from the database Convenient, but requires loader logic and can be combined with write-through

A successful write-through acknowledgment means the backing-store write completed at its configured durability level. This is stronger than write-back’s acknowledged-but-not-yet-persisted write, but cache-aside is often simpler when the cache is strictly an optimization.

Hit Ratio and Eviction

The benefit depends on the hit ratio and the cost saved by avoiding the database:

Benefit ≈ cache hits × avoided database cost − cache operating cost

Caching read-heavy, relatively stable data is most effective. Expensive database queries, repeated reference data, and frequently accessed product metadata are typical candidates.

Common eviction and expiration policies include:

  • LRU: Evicts the least recently used entry; effective for recency-heavy workloads.

  • LFU: Evicts the least frequently used entry; useful when popularity is stable.

  • FIFO: Evicts the oldest entry; simple but less adaptive.

  • TTL: Expires entries after a fixed lifetime; it bounds stale-data lifetime but does not replace invalidation.

Distributed caches often combine TTL with an approximate LRU or LFU policy. For hot keys, serve reads from replicas, split the hot data where possible, or coalesce concurrent misses. Negative caching and a Bloom filter can reduce cache penetration. A cache stampede can be controlled with a per-key lock, request coalescing, or early recomputation with randomized jitter.

Best Practices and When Not to Use It

  • Keep one ordered write path for each data item.

  • Define whether acknowledgment means database completion, cache completion, or both, including the required durability level.

  • Monitor hit ratio, eviction rate, stale-read incidents, database load, and write latency.

  • Avoid placing per-user sensitive data in a shared cache unless keys, access controls, isolation, and expiration are enforced.

  • Do not cache data that changes on every request, is rarely read again, or requires immediate transactional consistency, such as payment finalization.

  • Prefer cache-aside when low write latency is more important than synchronous dual writes.

  • Use write-through when read-after-write consistency is important and the added latency and failure handling are acceptable.

A common interview mistake is claiming that write-through guarantees atomic consistency. It reduces stale reads, but partial failures still require explicit retries, idempotency, invalidation, or reconciliation.

29

How do distributed caches handle node failures?

A distributed cache is part of a system that detects unavailable nodes, routes requests to healthy replicas where possible, and falls back to the authoritative data store when necessary. Reliability depends on replication, failover, retry behavior, and whether cached data can be regenerated.

Read full answer

Applies toRedis 8.x and Memcached 1.6.x; the general principles also apply to other distributed caches

Failure Model

A cache node can fail in several ways:

  • Process or host failure: The node stops responding and is detected as unavailable.

  • Network partition: The node remains healthy locally but is unreachable by clients or other nodes.

  • Slow node: Requests time out even though the node has not crashed.

  • Data loss: A primary fails before replicated data reaches a surviving replica.

Failover mechanisms need timeouts and health checks to distinguish a failed node from temporary delay. A request should have a bounded timeout; waiting indefinitely for one cache node defeats the purpose of distributing traffic.

Replication and Failover

Capability Redis Memcached
Replication Supports replicas and Cluster replication No built-in replication
Automatic failover Available through Redis Sentinel or Redis Cluster No server-side failover
Client behavior Uses topology information and redirects requests to the owning node Marks a server unavailable; failover behavior depends on the client and failure mode
Consistency during failover May lose recently acknowledged writes because replication is asynchronous No replicated state to lose, but cached entries are absent after failure

With Redis Sentinel, Sentinel monitors the primary and replicas and promotes a replica when it detects a qualifying failure. Redis Cluster distributes data by slot and coordinates failover among cluster nodes.

With Memcached, a failed server is marked unavailable by the client. Its cached entries are lost, so affected keys become unavailable; if the client supports failing over, the application can treat them as cache misses. Otherwise, requests to those keys may time out until the server returns.

The key trade-off is availability versus consistency. During a partition, a system may continue serving stale data for availability, or reject writes and preserve consistency. For derived cache data, many systems prefer continued reads with explicitly bounded staleness.

Request Handling Under Failure

A robust flow is:

  1. The client sends a request to the selected cache node.

  2. If the node times out, the client checks whether the operation is safe to retry.

  3. The client retries only a healthy destination indicated by topology, such as a promoted primary or an available replica that can serve the key. It does not assume an arbitrary node owns that key; otherwise, it treats the result as a miss.

  4. If the cache is unavailable, the application bypasses it and reads from the authoritative store.

  5. The application applies bounded timeout, retry, and circuit-breaker policies.

Caching should generally be optional for correctness. If the database can serve the request, a total cache outage should cause extra database load rather than application downtime. However, bypassing a large cache can overload the database, so use a circuit breaker, request coalescing, and a fallback limit.

Retries must use exponential backoff with jitter. An immediate retry against every failed node can create a retry storm.

Invalidation and Stale Data

If an invalidation message is sent to a failed node, other nodes may retain stale data until its TTL expires. Reliable invalidation therefore needs:

  • Retries, delivery tracking, and idempotent processing for invalidation messages.

  • Version metadata or generation changes so readers and update handlers reject stale values.

  • A reasonably short TTL as a final safety bound.

  • Durable event streams when every invalidation must be delivered.

Redis Pub/Sub is useful for low-latency notifications, but messages are not durable and may be missed during failures. Redis Streams can be used only when its persistence, replication, retention, and consumer acknowledgement and pending-entry handling meet the required durability guarantees. An external durable event log is often more appropriate when missed invalidations are unacceptable.

Capacity and Recovery

Losing a node reduces both memory capacity and key coverage. After recovery, the node is empty or has only a subset of data, causing a temporary spike in cache misses. This can create a cache stampede, where many requests request the same missing key.

Mitigate it with:

  • Request coalescing so only one request loads a missing key.

  • Distributed locking with a bounded lease when loading cannot safely be repeated.

  • Early expiration with randomized jitter.

  • Rate-limited fallback to the origin.

  • Prewarming for predictable high-traffic datasets.

Eviction policies also matter after node loss. Redis supports approximate LRU and LFU variants under maxmemory; Memcached maintains LRU lists within each memory slab class. If the cache shrinks below working-set size, hit ratios and database load will deteriorate.

Practical Guidance

  • Use replication and automated failover when cache outages would otherwise cause unacceptable latency.

  • Treat cache data as disposable unless its durability has explicit business value.

  • Keep a bounded timeout for every cache operation.

  • Avoid retrying non-idempotent cache operations without protection against duplicate work.

  • Never share sensitive per-user data in a shared cache without encryption, authorization boundaries, and appropriate key isolation.

  • Monitor hit ratio, miss latency, evictions, replication lag, failover events, and origin-database load.

A common interview mistake is claiming that replication prevents failures without discussing acknowledged-write loss, stale invalidations, reduced capacity, and the stampede created when failed nodes return empty. Interviewers usually follow up with how the system behaves during a network partition and whether the cache remains consistent with the source of truth.

30

What is the difference between a shared cache and a distributed cache?

A shared cache is a single remote cache node accessed by many clients or services, while a distributed cache is a cluster of multiple cache nodes that partition and/or replicate data to act as one logical cache.

Read full answer

Applies toall versions

The Problem It Solves

As you scale from a single application instance to a fleet, in-process caches become isolated and duplicate work. Moving cached data to a remote store fixes cross-instance reuse, but you must choose between putting everyone on one node (shared cache) or spreading across many nodes (distributed cache) with different scaling and failure properties.

Key Differences

Aspect Shared Cache Distributed Cache
Topology One remote cache node used by all clients and services. A cluster of nodes forming a single logical cache, typically with key partitioning (often via consistent hashing).
Scalability Primarily vertical. Limited by that node’s memory, CPU, and network. Primarily horizontal. You can add nodes to grow total memory and aggregate throughput.
Fault Tolerance A single point of failure unless you add explicit high availability to that lone node. Can tolerate loss of some nodes depending on replication and partitioning, but that comes with consistency trade-offs.
Consistency Easier to reason about: there is only one location for those cached copies. During a network partition, you must trade off consistency and availability (the CAP trade-off). In practice, PACELC is often more realistic to reason about than “pick any two”.
Latency Often predictable. A same-datacenter round trip to that single node is roughly ~0.5 ms (order of magnitude). Can be in a similar ballpark, but routing by key, sharding, or replication can add hops and make tail latency less predictable.
Operational Complexity Lower. Easier to monitor, debug, and reason about. Higher. Requires node discovery, rebalancing, failure handling, and cluster-aware operations.

Trade-offs

  • Cost: A shared cache is usually cheaper upfront, but hitting its capacity ceiling can force a costly redesign. A distributed cache spreads cost across nodes, however operational effort grows with cluster size.

  • Invalidation: Both are prone to stale data. With a shared cache, updates go to one place. With a distributed cache, a key update must reach the node(s) that own it (or follow the cluster’s replication rules), which makes invalidation and cache stampede protection harder.

  • Safety of data: Never put per-user sensitive data on a shared or distributed cache unless you have strict namespacing, keying by tenant/user scope, and tight access control. These are often multi-tenant across services and are not a secure place to store raw secrets by default.

When To Use Which

  • Use a shared cache if you have a small fleet of stateless instances, a manageable working set that fits on one node, read-heavy traffic, and limited operational bandwidth for clustering.

  • Use a distributed cache if your working set exceeds any single machine, you need to scale memory/throughput horizontally, you need to tolerate partial node failures, or multiple services must share one logical cache at scale.

Common Mistake To Avoid

Don’t equate “shared” with “highly available”. A single remote cache node is still a single point of failure unless replication/high availability is explicitly configured for it. Also, don’t assume cross-instance reuse alone justifies going distributed too early—only do it when the scale or fault-tolerance need outweighs the added complexity.

31

What is a CDN and how does it use caching?

Content Delivery Network (CDN) is a geographically distributed set of proxy servers (edge points of presence) that cache and serve static or cacheable content closer to the client. Its relation to caching is direct: a CDN places a shared HTTP cache at the edge, so requests that would otherwise travel all the way to the origin can be answered from nearby cache instead.

Read full answer

Applies toall versions

The Problem It Solves

  • Origin overload: Without a CDN, every request for images, CSS, JS, or other cacheable assets hits the origin directly. For an e-commerce site during a sale, this can create a massive spike on the origin with very little need to recompute those assets.

  • High network distance: A user far from the origin can see cross-region round trips of roughly 50-150 ms, compared to a same-datacenter round trip of roughly 0.5 ms or a local edge response much closer to that. These are rough orders of magnitude, but they show why distance matters.

  • Wasted bandwidth: Re-delivering the same unchanging file to thousands of clients pulls the same bytes from the origin each time, increasing load and cost without adding value.

How A CDN Uses Caching

  • Edge cache: The CDN’s edge servers act as a cache layer between the client and the origin. On a cache hit, the edge returns the stored response with no call to the origin. On a cache miss, the edge fetches from the origin, stores a copy according to the caching policy, then returns it to the client.

  • Cache key: Most CDN caches use the request’s URL as the primary key, but can also factor in headers like Accept-Encoding or use Vary. The key determines what is considered “the same” object.

  • HTTP cache directives: The origin controls edge behavior with headers like Cache-Control. For shared caches, s-maxage tells the CDN how long to treat a response as fresh without revalidating. max-age is mainly for private/browser caches. ETag and Last-Modified allow conditional revalidation (If-None-Match/If-Modified-Since) to save bandwidth when the object hasn’t changed.

  • Request flow (text): Client → DNS/Nearest CDN Edge → (cache hit) → Client or Client → Nearest CDN Edge → (cache miss) → Origin → Edge (store by policy) → Client.

Caching Behavior: Freshness, Invalidation, And Safety

  • TTL-driven freshness: The CDN serves from cache while the edge copy is within its TTL (s-maxage/Expires). This trades absolute immediacy for lower latency and lower origin load.

  • Revalidation: When stale, many CDNs will do a conditional request to the origin to check for changes. A 304 Not Modified lets the edge keep its copy and just update freshness, avoiding re-downloading the full body.

  • Invalidation/Purge: If the origin changes content before TTL expires, you cannot rely on the clock alone. CDNs support purging by URL, by path patterns, or by cache tags/versioned asset names. Versioning (e.g. app.abc123.js) is often the simplest and safest way to force a new cache entry.

  • Stale-while-revalidate / stale-if-error (where supported): Some CDNs can serve a slightly stale response while fetching a fresh one in the background, or serve stale on origin failure. These can improve availability, but they intentionally increase staleness under those conditions.

  • What should NOT be cached on a shared CDN: Never put per-user sensitive data, authentication tokens, CSRF-bearing responses, personalized HTML for a logged-in dashboard, or any response with Set-Cookie that would leak across users. Endpoints with POST, PUT, PATCH, DELETE are not safely cacheable by default. If a response must never be shared, use Cache-Control: private, no-store so shared caches (including CDNs) must not store it.

Trade-offs

Factor Gain Cost/Risk
Latency Much lower for cacheable content (closer to users). No benefit for uncacheable, private, or purely real-time requests.
Origin Load & Availability Fewer requests to origin, better headroom during traffic spikes. If the CDN has issues or is misconfigured, you need to know your origin fallback path.
Bandwidth/Cost Can reduce egress from origin. CDN has its own pricing; aggressive caching of the wrong things can hide bugs or create surprise costs.
Freshness vs Consistency You can accept small staleness for static assets. For rapidly changing public content, a long TTL means users see old data until invalidated/revalidated. This is a consistency trade-off, not a free fix.
Operational Complexity Scales reads globally. Invalidation, cache keys, Vary, auth-splitting, and debugging “am I seeing the edge or origin?” adds complexity.

When To Use It (And When Not To)

  • Use it when: Serving cacheable, public, read-heavy content to a geographically spread audience (e.g. product images on an e-commerce storefront, public news assets). The closer the content is to “write once, read many times”, the better the fit.

  • Do NOT rely on it when: You need strict, immediate consistency for per-user data, you are serving protected API responses to arbitrary users on a public shared cache, or every request is dynamic with no safe cache key. Also don’t use it as a replacement for your application cache for highly personalized, short-lived computed results.

Interviewer’s tip: The mistake candidates usually make is adding a CDN without an invalidation plan. Don’t just ask “should we cache?”, ask “how do we change this safely?” Prefer versioned asset URLs over trying to purge the entire world on every deploy.

32

What is edge caching and when should systems use it?

Edge caching stores HTTP responses in a CDN or reverse proxy close to users so repeated requests can be served without reaching the origin server. It is most effective for public, read-heavy content with high reuse and an acceptable staleness window.

Read full answer

Applies toHTTP/1.1, HTTP/2, and HTTP/3 with CDN or reverse-proxy edge caches

How Edge Caching Works

Without an edge cache, every request may travel to the origin, consuming compute, database connections, and network bandwidth. A typical flow is:

Client → CDN hostname resolves to CDN addresses → anycast, BGP, or GeoDNS routes to a nearby PoP → edge cache → origin on a miss or revalidation

On a cache miss, the edge fetches the representation from the origin, stores it according to the response’s freshness and cache policies, and returns it. On a fresh hit, the edge returns the cached representation without contacting the origin. A stale hit may be revalidated with the origin using an ETag or Last-Modified validator.

Because the user and origin may be in different regions, an edge hit can avoid a cross-region round trip of roughly 50–150 ms. It also reduces origin load and improves scalability during traffic spikes.

When to Use It

  • Static assets: JavaScript, CSS, images, fonts, and downloadable files.

  • Public read-heavy pages: Product categories, news articles, documentation, and marketing pages.

  • API responses: Public catalog, weather, or reference data with a predictable freshness requirement.

  • Media and large files: Video segments and images benefit from nearby delivery and reduced origin egress.

  • Traffic shielding: Protect the origin during launches, sales, or viral events.

Do not publicly cache personalized dashboards, payment responses, account details, or other user-sensitive data. Such responses should normally use Cache-Control: private, no-store. Edge caching is a poor fit when every response is unique, reads are rare, or the application requires immediate strong consistency.

Strategies and Hit Ratio

Strategy Behavior Edge-cache fit
Cache-aside The cache fetches missing data from the origin Common model for cacheable CDN content
Read-through The cache automatically loads missing data More common in application-level caches
Write-through Cache and origin are updated together Complex for distributed CDN PoPs
Write-back Writes are deferred to the origin Unsuitable for authoritative mutable data
Write-around Selected operations bypass the cache Useful for updates or low-reuse operations

The hit ratio is hits / (hits + misses). An 80% hit ratio means 80% of eligible requests are cache hits; fresh hits can avoid the origin, while revalidating hits still contact it. Measure hit ratio by route, content type, region, and cache status.

Edge caches mainly use expiration policies and provider-managed capacity eviction. Applications commonly use LRU, LFU, or FIFO, but CDNs generally do not expose those algorithms as direct configuration options.

Invalidation and Consistency

Invalidation is the main design challenge. Common approaches are:

  • TTL: The representation expires after a defined period, bounding staleness.

  • Purge: Request deletion of specific objects or, where supported, URL prefixes from edge locations.

  • Versioned keys: Publish a new URL or cache key when content changes, reducing the chance of reusing stale content.

  • Event-driven invalidation: Publish content-change events that trigger CDN purges.

Purging is not necessarily instantaneous across every PoP. For mutable data, define an explicit staleness target and decide whether serving slightly stale content is acceptable.

The cache key must include request components that affect the representation, such as scheme, host, path, relevant query parameters, and header values identified by Vary. Excluding a parameter can cause one user or locale to receive another user’s response.

For HTTP caching, max-age controls freshness in browsers and caches, while s-maxage overrides it for shared caches such as CDNs. ETag supports conditional revalidation, while stale-while-revalidate and stale-if-error explicitly trade freshness for availability.

Failure Modes and Trade-offs

  • Cache stampede: Many clients request the same expired representation simultaneously. Use request coalescing, provider-side collapsing, locking, or early refresh with jitter.

  • Hot keys: A highly popular object can overload one cache node or region. Use provider replication, sharding, and traffic management.

  • Cache penetration: Repeated requests for nonexistent objects can reach the origin. Cache negative responses briefly or use a Bloom filter near the origin.

  • Origin failure: Configure origin-fetch timeouts and bounded retries with exponential backoff and jitter for transient failures. Use circuit breakers and concurrency limits to prevent cache-fill traffic from exhausting downstream resources.

  • Operational cost: Edge caching adds CDN fees, configuration, invalidation workflows, observability, and cross-location debugging.

Serve stale content only when the business explicitly permits it. A CDN reduces origin load, but it cannot make a single unavailable origin healthy unless usable stale data exists.

Best Practices and Interview Guidance

  • Set TTLs from the business freshness requirement, not merely to maximize hits.

  • Instrument hit ratio, origin load, purge latency, stale responses, and cache errors.

  • Use no-store for sensitive data; do not rely on removing Authorization from the cache key.

  • Test behavior when the CDN, origin, or purge API is unavailable.

  • Include regional and per-route metrics because a global average can hide cold or unhealthy PoPs.

Interviewers usually look for an explicit invalidation strategy, cache-key design, hit-ratio reasoning, and failure handling. The common mistake is adding a CDN without defining acceptable staleness or how stale and expired content is handled.

33

What is cache warming and when should you use it?

Cache warming, also called cache priming, is the practice of preloading a cache with data expected to be requested soon. It reduces cold-start latency and can prevent many simultaneous requests from reaching the authoritative data store.

Read full answer

Applies toall cache technologies and versions

Why Cache Warming Exists

A cold or empty cache has no useful entries. Without warming, the first requests for popular data must:

  1. Miss the cache.

  2. Query the database or call another backend service.

  3. Populate the cache for subsequent requests.

If thousands of requests arrive simultaneously, they may all miss before the first entry exists. This creates a cache stampede, or thundering herd, and can overload the backend. Cache warming proactively stores known or predicted high-demand entries before peak traffic arrives.

Typical latency improves from tens or hundreds of milliseconds for an origin lookup to roughly microseconds or sub-milliseconds for a cache hit, although actual values depend on the workload and deployment.

Common Warming Approaches

  • Bulk preload: Load a known set of frequently requested records, such as product details, into a distributed cache before a release begins.

  • Replay popular reads: Capture recent or historical read keys and fetch them through the normal cache-population path.

  • Targeted warming: Preload only specific keys associated with an event, campaign, geographic region, or traffic shift.

  • Staggered warming: Warm entries gradually instead of flooding the database with a large simultaneous preload.

  • Refresh before expiry: Refresh popular entries shortly before their TTL expires. Adding jitter prevents many entries from expiring together.

Warming should populate the same cache using the normal cache-population path. This preserves the expected representation, serialization, and consistency checks. It does not replace authorization: only warm shareable data in a shared cache, or explicitly isolate and authorize per-user entries.

When It Is Used

Cache warming is most useful when demand is predictable and backend capacity is constrained:

  • Application deployment: Preload critical configuration and hot records in the shared cache before new instances receive traffic; warm an instance-local cache separately on each instance.

  • Cold cache or missing data: Restore expected entries after a restart that left the cache empty. A Redis restart does not necessarily create a cold cache if persisted data is restored.

  • Traffic event: Prepare for a product launch, flash sale, scheduled broadcast, or other expected traffic spike.

  • Regional failover: Warm the destination region before shifting users to it.

  • Autoscaling: Warm a shared cache once for newly added instances, or warm each instance separately when using an instance-local cache.

  • CDN or reverse-proxy activation: Prime selected URLs or objects after configuration changes or cache purges.

For example, an e-commerce service might preload popular product pages, category listings, and pricing data before a sale. A news service might warm articles attached to an expected breaking-news event.

Relationship to Caching Strategies

Strategy How warming applies
Cache-aside Preload entries using the same “read cache, then read database” path used during normal operation.
Read-through The application reads only through the cache, which loads missing entries from the database. Warming invokes this path in advance.
Write-through Writes update the cache and database as part of the write path, so separate warming is often unnecessary for newly written data.
Write-back Cache writes are flushed later. Do not warm an entry before pending writes have been synchronized with the authoritative store.
Write-around Writes intentionally bypass the cache, so newly written data is generally not a warming candidate until its normal cache policy permits caching it.

Warming complements these strategies; it does not replace TTLs, invalidation, or a consistency model.

Consistency and Failure Risks

  • Stale data: Preloaded entries may already be outdated. Use short TTLs where freshness matters, and verify that warming occurs after the authoritative update.

  • Invalidation races: If an entry changes while it is being warmed, the old value may overwrite the new one. Coordinate warming with versioning or invalidation events.

  • Bootstrap overload: Warming itself can generate a database spike. Rate-limit it, use replicas where appropriate, and spread work over time.

  • Partial success: A warming process may fail halfway through. The system should continue serving traffic and treat missing entries as ordinary cache misses.

  • Memory pressure: Preloading too many low-value entries can evict useful data. Select entries by popularity and business importance rather than warming everything.

Practical Guidance

A strong warming design should define the target keys, source of truth, maximum preload rate, TTL, failure behavior, and monitoring. Track cache-hit ratio, origin load, warming duration, eviction rate, and the number of simultaneous misses for a single key.

Do not warm per-user sensitive data in a shared cache unless keys and access controls are carefully isolated. Avoid warming rapidly changing data, entries with a very short useful lifetime, or datasets whose cold-start penalty is small and backend capacity is sufficient. The common interview mistake is treating warming as a substitute for cache design: it helps only when the warm set is accurate, invalidation is defined, and the backend can survive misses.

34

How does database query result caching work?

Query result caching stores the results of database queries so repeated requests can be served without executing the same query again. It reduces database load and response latency, but introduces staleness, invalidation complexity, and risk during cache failures.

Read full answer

Applies toall database systems and versions; Redis 8.x and Memcached 1.6.x for the product comparison

How It Works

A cache can exist inside the database engine, in application memory, or in a shared service such as Redis or Memcached.

A typical cache-aside request flow is:

  1. The application constructs a cache key from the query and its inputs.

  2. It checks the cache.

  3. On a hit, it returns the cached result.

  4. On a miss, it executes the database query.

  5. It stores the result in the cache with a TTL and returns it to the caller.

A safe cache key includes:

  • The normalized SQL text or query identifier

  • Parameter values and relevant data types

  • Database, schema, or tenant identity

  • Locale, currency, or other session settings that affect results

  • A version marker when records can be updated independently

For example, caching WHERE id = ? requires including the bound id; otherwise, different users could receive the same result.

Do not confuse query result caching with a database buffer pool. A buffer pool caches database pages or blocks to reduce physical I/O. A result cache stores complete query outputs and bypasses query execution on a hit.

Cache Write Strategies

Strategy Behavior Best fit Main cost
Cache-aside Application loads from the cache and populates it after a miss Read-heavy applications with separate ownership Every cache miss adds cache and database latency
Read-through Cache calls the database through a configured loader Applications wanting cache behavior encapsulated in one component Tighter coupling to the loader
Write-through Writes update the cache and database together Data that must be immediately visible in both stores Higher write latency and possible partial failures
Write-back Cache is updated first and persisted asynchronously High-write workloads where temporary loss is acceptable Data-loss risk and difficult consistency
Write-around Writes go to the database without updating the cache Data likely to be read once or not immediately A later read still misses and must repopulate the cache

For most web services, cache-aside is the simplest default because it keeps the cache optional and lets the database remain the source of truth.

Invalidation and Consistency

A TTL is a safety bound, not a complete invalidation strategy. If a product price changes, a short TTL may still leave incorrect prices visible for that period.

A stronger design invalidates or refreshes affected entries when data changes:

  • The writer updates the database and publishes an invalidation event.

  • The event removes or refreshes the relevant cache key.

  • A TTL handles missed events and eventually restores consistency.

An transactional outbox can reliably publish invalidation events when the database and message broker are separate systems. If immediate consistency is required, the write path may wait for the database update and cache invalidation, accepting additional latency and partial-failure complexity.

Common eviction policies include:

  • LRU: Removes least recently used entries; effective for changing workloads

  • LFU: Removes least frequently used entries; useful when access frequency is stable

  • FIFO: Removes oldest entries; simple but not usage-aware

  • TTL expiry: Removes entries after their freshness deadline

Concurrency and Failure Modes

  • Cache stampede: Many requests miss the same popular key and query the database simultaneously. Prevent it with per-key locking, request coalescing, or early recomputation with randomized jitter.

  • Hot key: One key receives disproportionate traffic. Mitigate it with local caching, key replication, or carefully designed sharding.

  • Cache penetration: Requests repeatedly ask for nonexistent records. Cache negative results briefly and consider a Bloom filter for a large key space.

  • Cache outage: The cache should fail open to the database only if the database can absorb the traffic. Use rate limits, concurrency limits, and circuit breakers to prevent an outage from becoming a database outage.

  • Stale data: stale-while-revalidate can serve old content while refreshing it, but is appropriate only when temporary staleness is acceptable.

A 95% hit ratio can materially reduce latency. For example, with a roughly 1 ms database query and 0.5 ms cache lookup, the average path becomes about 0.55 ms, although actual results depend on network, database, and serialization costs.

Redis or Memcached

Capability Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, streams, and others Primarily key-value entries
Persistence Optional snapshot and append-only persistence No persistence
Replication Built-in replication and clustering Replication available
Concurrency Main command execution is single-threaded, with I/O threading Designed for multithreaded request serving
Typical role Shared cache, counters, locks, queues, and streaming Large distributed memory cache

Choose Redis when richer data structures or persistence are useful; choose Memcached for a simple, memory-oriented distributed cache. Neither removes the need for TTLs, invalidation, and origin-load protection.

When to Cache

Cache queries that are frequently repeated, expensive to compute, and safe to serve briefly from an older value. Good candidates include product catalogs, news feeds, reference data, and rendered public responses.

Avoid caching highly sensitive per-user data on a shared cache unless the key includes the user or authorization scope and the response cannot expose another user’s data. Also avoid caching results when every request requires current financial, inventory, or authorization information unless the consistency mechanism is explicit.

Interviewers typically look for a complete design: cache keys, TTLs, eviction, invalidation, stampede protection, hit-ratio reasoning, and a safe fallback when the cache is unavailable.

35

What is object caching in object-oriented programming?

Object caching stores an object instance or its computed state so later requests can reuse it instead of creating it or retrieving its data again. It can reduce CPU work, allocations, I/O, and latency, but cached mutable objects introduce invalidation, memory, and concurrency risks.

Read full answer

Applies toall object-oriented languages and runtimes

The Problem It Solves

Constructing an object is not always cheap. An object may require database queries, network calls, file reads, parsing, or expensive calculations. For example, an e-commerce service might create a ProductCatalog object whose initialization loads thousands of products from PostgreSQL.

Without caching, every request repeats that work. Object caching retains the result so subsequent requests avoid reconstruction. This is most valuable when:

  • Initialization is expensive or performs remote I/O.

  • The same object is requested frequently.

  • The underlying data changes relatively infrequently.

  • Temporary staleness is acceptable.

Caching an object does not inherently make its data durable. The cache is an optimization, not normally the source of truth.

Common Caching Patterns

  • Lazy cache-aside: Look in the cache first; if the object is absent, load it from the database or service, store it, and return it. This is the most common pattern because the cache is populated only when needed.

  • Read-through: The cache handles misses by loading data through a supplied loader. Application code interacts only with the cache abstraction.

  • Write-through: Updates are written to both the cache and the underlying store. Reads become simple, but writes cost more, and partial failures require careful consistency handling.

  • Write-back: Updates modify the cache first and are persisted later. This reduces write traffic but creates a window where the cache and database differ, so it requires bounded persistence delays and failure recovery.

  • Write-around: Write changes to the source of truth without immediately updating or populating the cache. A later read reloads the value, which can suit write-heavy data whose updated entries are rarely read, but it increases cache misses.

  • Eager initialization: The application creates and caches an object at startup or refreshes it periodically. This moves initialization off the request path but consumes resources even when the object is never used and requires startup-failure handling.

For object-oriented code, a lazy cache is often implemented with thread-safe initialization or a single-flight mechanism. The cached object should normally be immutable, immutable after construction, or defensively copied when exposed to callers.

Advantages

  • Lower latency: A cache lookup avoids reconstruction and potentially a database or network round trip.

  • Reduced CPU and allocation pressure: Reusing an object avoids repeated parsing, calculation, and object creation.

  • Reduced database load: Frequently read records can be served without repeatedly querying the source of truth.

  • Improved throughput: The same worker can handle more requests when expensive work is avoided.

  • Predictable request latency: Preloaded objects keep slow first-use initialization off the request path, although initialization failures then affect startup instead.

The benefit depends on the hit ratio:

hit ratio = cache hits / total cache requests

A cache with a 90% hit ratio removes about 90% of the load it intercepts, assuming hits are substantially cheaper than misses. A low-hit-ratio cache may add lookup overhead and memory usage without meaningful benefit.

Trade-offs and Correctness

  • Staleness: A cached object may contain outdated data. Use a TTL to bound staleness or invalidate the entry when the source changes.

  • Memory pressure: Large object graphs can exhaust heap capacity. Distributed caches such as Redis or Memcached have independent memory limits and eviction policies.

  • Eviction: Common policies include LRU, LFU, FIFO, and TTL expiration. LRU favors recent access, while LFU favors frequently accessed entries.

  • Mutable shared state: If several threads share a mutable object, they may observe concurrent modifications. Prefer immutable snapshots or strict synchronization.

  • Cache stampede: Many threads may load the same missing object simultaneously. Use per-key locking, request coalescing, or single-flight loading; stale-while-revalidate can reduce latency further.

  • Cache penetration: Requests for nonexistent IDs can repeatedly bypass the cache and reach the database. Validate identifiers, cache short-lived negative results, or use a Bloom filter when the key universe is known.

  • Hot keys: One frequently requested object may overload a cache node or network link. Replication, key sharding, or request coalescing can distribute that load.

  • Sensitive data: Do not place per-user or confidential objects in a shared cache unless access is isolated and entries are encrypted where necessary.

When to Use It

Use object caching for shared, frequently accessed, reasonably stable data such as reference tables, feature flags, compiled templates, or configuration snapshots. Avoid it for rapidly changing records, single-use large objects, strongly consistent transactional state, or data whose correctness depends on every request seeing the latest write.

The common interview mistake is adding a cache without defining ownership, TTL, eviction, invalidation, and stampede behavior. A strong answer should also state whether the application cache, Redis, or CDN is appropriate and identify the database as the source of truth.

36

How does caching impact microservices architecture?

Caching reduces latency, database load, and cross-service calls by storing reusable data closer to its consumers. In microservices, it improves scalability and can support graceful degradation, but it introduces invalidation complexity, stale-data risk, and additional failure modes.

Read full answer

Applies toversion-independent microservices architecture principles; current stable releases of Redis, Memcached, PostgreSQL, MySQL, Nginx, and Envoy

Architectural Impact

A typical request flow is:

Client → CDN → Load balancer → API service → Redis cache → Database or downstream service

Caching can exist at several layers:

  • Browser cache: Stores responses on the client; Cache-Control, ETag, and max-age control reuse.

  • CDN: Serves cacheable HTTP responses from edge locations, reducing origin load and latency.

  • Reverse-proxy cache: Nginx or Envoy caches eligible responses before they reach a service.

  • Application cache: Uses private process memory, avoiding distributed-cache network latency but giving each instance its own copy.

  • Distributed cache: Redis or Memcached shares cached data across service instances.

  • Database caches: Buffer pools cache hot pages; PostgreSQL can reuse query plans, while MySQL 8.0+ has no former server-wide query cache.

The benefit depends on the hit ratio. With a 1 ms hit path and a 6 ms miss path comprising a 1 ms lookup plus a 5 ms database read, a 95% hit ratio gives 0.95×1 + 0.05×6 = 1.25 ms, versus roughly 5 ms for database-only reads. A frequently missing cache adds latency and another failure dependency.

Cache Strategies

Strategy Behavior Appropriate use Main cost
Cache-aside Application reads cache and loads data on a miss General-purpose service caching Application implements population and invalidation
Read-through Cache loads missing data itself Single cache abstraction Couples the service to the cache library
Write-through Write updates cache and backing store synchronously Read-heavy data requiring cache coherence Higher write latency and possible partial failure
Write-back Cache is updated first and persisted asynchronously Workloads tolerating eventual persistence Data-loss window before persistence
Write-around Write bypasses the cache; later reads populate it Large or rarely read data Existing stale entries still need invalidation

Cache-aside is a common default because services retain control over database access. Define which service owns each entry; avoid an ungoverned shared cache across unrelated bounded contexts.

Consistency and Invalidation

Invalidation must be designed rather than left to implementation details.

  • TTL: Bounds staleness and provides a fallback when invalidation is lost. Set it from business tolerance, not one global default; immutable data may safely have no TTL.

  • Event-driven invalidation: Publish an event after committing a change, and have consumers update or delete related entries.

  • Versioning: A version prevents stale use only when readers compare it with an authoritative version or keys are switched atomically; embedding a version alone is insufficient.

  • Transactional outbox: Atomically persist a database change and invalidation event, then publish the event asynchronously.

Immediate deletion after a commit still has a race: a concurrent reader can reload old data and write it back after deletion. Version checks, coordinated invalidation, or a short TTL can contain this problem.

During a partition, CAP forces a consistency-versus-availability choice: serve stale results with relaxed consistency or fail requests to preserve linearizability. PACELC additionally identifies the normal-operation trade-off between consistency and latency.

Failure Modes and Protections

  • Cache stampede: Many requests miss one hot key. Use request coalescing, a carefully implemented short-lived lock, or early refresh with jitter.

  • Cache penetration: Repeated nonexistent lookups consume capacity. Cache negative results briefly or use a Bloom filter where appropriate.

  • Hot keys: Replicate read-heavy keys where supported, split them across shards, or maintain a bounded local cache.

  • Cache outage: Bypass the cache if the source can handle the load; use rate limits and a circuit breaker to prevent cascading failures.

  • Eviction pressure: Redis supports configurable LRU/LFU eviction, while Memcached uses LRU. Both expire entries according to TTL; TTL expiration is not itself an eviction policy.

  • Stale data: Monitor hit ratio, eviction rate, data age, and invalidation lag.

Redis and Memcached

Capability Redis Memcached
Data model Strings, hashes, sets, sorted sets, streams, and others Simple key-value values
Persistence Optional RDB snapshots and AOF No built-in persistence
Replication Built-in replication and Redis Cluster support No built-in replication protocol
Threading Core command execution is largely single-threaded, with optional extra I/O threads Multiple worker threads serve requests
Best fit Structured data, counters, queues, pub/sub, and shared state Simple high-speed object or page caching
Main limitation More operational and memory complexity Fewer features and no durable shared state

Choose Redis when richer operations or shared coordination are valuable; choose Memcached when simple in-memory caching is sufficient. Neither is automatically the better choice.

Best Practices

  • Estimate hit ratio and expected savings before rollout, then instrument the actual hit ratio and validate the benefit.

  • Do not put sensitive per-user data in a shared cache unless keys, authorization, encryption, and isolation are handled explicitly.

  • Avoid caching continuously changing data or records requiring strict transactional consistency, such as an in-flight payment authorization.

  • Set TTLs where staleness is possible, monitor cache failures, and test behavior with an empty or unavailable cache.

  • Treat the cache as an optimization, not the business system’s source of truth.

Interviewers commonly look for an explicit invalidation and failure plan. The common mistake is discussing only the cache hit path while ignoring stale data, stampedes, and cache outages.

37

How does caching work with serverless computing architectures?

Serverless caching stores reusable data outside a function’s execution lifecycle so repeated requests can avoid computation, database access, or even function invocation. It commonly uses a shared Redis-compatible cache, while CDN and HTTP caches can serve responses before a function runs.

Read full answer

Applies toall versions of serverless computing platforms (AWS Lambda, Azure Functions, and Google Cloud Functions)

Why Serverless Changes Caching

Serverless functions are ephemeral and horizontally scaled, so an in-process map is not shared reliably across concurrent instances and disappears when an instance is recycled. It is useful only for temporary, non-critical data.

A shared cache such as Redis provides cross-instance visibility but adds network latency, cost, and another failure dependency. A CDN or HTTP cache is often better for public content because it can answer before the function is invoked.

Where to Place the Cache

Layer Suitable content Main trade-off
Browser or HTTP cache Static assets and private user content Device-specific freshness and limited visibility
CDN Public HTML, images, APIs, and videos Cache-key and purge complexity
Reverse proxy Responses from an internal service Shared-state correctness and invalidation
Platform-managed cache Managed API or framework responses Provider-specific behavior
Distributed cache Data shared across function instances Network latency and operational cost
In-process cache Temporary instance-local data No sharing or durable state
Database query or buffer cache Frequently accessed results or pages Managed inside the database, but limited to database workloads

Browser caching is controlled by Cache-Control; max-age defines freshness and ETag enables conditional validation. Shared CDNs and proxies use s-maxage, which takes precedence over max-age; sensitive or user-specific responses should use private or no-store, with Vary used when the response depends on request headers.

Redis supports strings, hashes, lists, sets, sorted sets, and streams, with optional RDB or AOF persistence, replication and clustering, and primarily single-threaded command execution. Memcached provides simpler string-like items, no built-in persistence or native replication, and a multi-threaded server. Choose Redis when richer operations or persistence are useful; choose Memcached for a simple, high-throughput shared cache.

Common Cache Strategies

  • Cache-aside: The function reads the cache and loads the database on a miss. This is the usual default because the cache can be bypassed during an outage and only reusable data is cached.

  • Read-through: The cache loads missing data itself. This simplifies application code but couples the cache implementation to the database.

  • Write-through: Writes synchronously update both cache and database. Reads can be fast, but write latency and availability requirements increase; consistency also depends on write ordering and failure handling.

  • Write-back: The cache is updated first and the database later. This reduces database load but creates stale-read and data-loss or recovery risks.

  • Write-around: Writes bypass the cache. This prevents low-reuse data from evicting hotter entries.

A typical cache-aside flow is:

Client → CDN → Serverless function → Redis → Database

Invalidation, Eviction, and Stampedes

  • TTL expiration: Bounds staleness but requires choosing an appropriate lifetime.

  • Event-driven deletion: A write publishes an invalidation event that removes affected keys.

  • Versioned keys: Writers change the data version, and readers move to a new namespace.

  • Early refresh: A value is refreshed shortly before expiry, ideally with randomized jitter.

TTL expiry and capacity eviction are different: TTL defines when an item expires, while LRU, LFU, and FIFO determine which items are removed under memory pressure. Shared Redis can select LRU, LFU, or TTL-based eviction policies; Memcached uses LRU.

A hot key can expire while many function instances request it, causing a cache stampede. Use request coalescing, a short distributed lease lock with a timeout, stale-while-revalidate behavior, or early refresh with jitter. Monitor key frequency and shard or replicate read-heavy hot keys when appropriate.

Cache penetration can be reduced by briefly caching negative results or using a Bloom filter. Do not place personal or authorization-sensitive data in a shared cache unless keys, isolation, and authorization prevent cross-user disclosure.

Performance, Cost, and Failure

If a database query takes about 50 ms and a nearby cache read takes about 1 ms, a 90% hit ratio gives an approximate read time of 0.9 × 1 ms + 0.1 × 50 ms = 5.9 ms, excluding other overhead. Actual gains depend on payload size, network distance, serialization, and database behavior.

Front-door caching can reduce function invocations, charges, and cold starts; an internal Redis cache only reduces database work. Caching adds TTL management, invalidation logic, observability requirements, and infrastructure cost. If the cache fails, use timeouts, rate limiting, and a circuit breaker, then fall back to the database where correctness permits rather than allowing a cascading outage.

When Not to Cache

Do not cache data that is rarely reused, changes constantly, or must be strongly consistent, such as an authoritative account balance after a transfer. Avoid sensitive shared-cache entries without explicit isolation and authorization. Use caching when expected reuse and a measured hit ratio justify the added latency, consistency risk, and operational work.

38

What are cache compression techniques and their trade-offs?

Cache compression is the act of shrinking cached values before writing them to a cache and decompressing them on read, done to fit more entries in limited memory at the cost of extra CPU work.

Read full answer

Applies toall versions

The Problem It Solves

Before reaching for compression, the issue is simple: cache capacity is almost always the bottleneck before raw I/O. If values are large (e.g. serialized JSON for a product catalog in an e-commerce system), evicting them earlier drives down the hit ratio. Compression tries to buy effective capacity, but only by spending CPU cycles on every store that compresses and every hit that decompresses. Whether that trade-off pays off depends entirely on the workload.

Common Techniques

  • LZ-based block compression: Uses dictionary/repetition scanning to shrink structured text or serialized objects. Best for repetitive values (lists, rendered fragments), but gains drop sharply on high-entropy data.

  • Dictionary compression (shared): Builds a small shared dictionary across many entries (useful when many objects share keys/fields). Effective if the value set is homogeneous, but requires dictionary versioning and coordination across nodes in a distributed cache to avoid encoder/decoder mismatches on upgrades.

  • Delta compression: Stores the difference from a base object instead of the full copy. Great for near-identical versions (e.g. a feed item with minor updates), but reconstruction is required, random/partial reads become more complex, and cache key/base tracking adds overhead.

  • Selective compression: Only compress values above a chosen byte threshold. This avoids wasting CPU on tiny objects where compression gain is negligible or negative.

  • Value-aware compression: Chooses or tunes compression based on the serialization format (e.g. compact binary vs raw JSON). Gains depend heavily on entropy of the data, not just its size.

Trade-offs

Factor Gain Cost
Memory More entries can fit by bytes stored, which can raise the hit ratio under memory pressure. Savings vary wildly by entropy. Already-compressed media, encrypted blobs, or high-entropy values often compress little to nothing. Eviction must be charged against the compressed bytes actually used, not the original size.
Latency Can avoid slower fetches (to DB/CDN/origin) if the improved hit ratio outweighs compression/decompression time. Every compressed hit pays decompression and every compressed write pays compression. This hits p99/p999 tail latency far more than average. The CPU cost is highly workload- and encoder-dependent (must be measured per payload).
Complexity Lets you keep larger objects warm with the same node size. You must track “compressed vs raw”, version your compressed format, handle corruption safely, and keep compression changes aligned with rolling upgrades. Invalidation discipline stays mandatory.
Scalability Can save network egress between app and distributed cache if you transfer compressed payloads end-to-end. Cross-node consistency is unchanged, but shared dictionaries, format versions, or delta bases require explicit coordination and make rolling upgrades riskier.
CPU pressure None by default. Moves pressure from memory/network to CPU. On CPU-bound nodes this is often worse than just evicting. Favor fast, bounded encoders over maximal-ratio ones when CPU is tight.

Best Practices

  • Compress by size, not by default: Only compress above a threshold and test with real payloads. What works for one workload may hurt another.

  • Skip low-value data: Do not compress already-compressed files, encrypted/high-entropy blobs, or values where gain is negligible. Compression provides no confidentiality and is not a security control.

  • Protect sensitive data by placement, not by compression: Do not put per-user sensitive data on shared caches even if compressed. The rule is placement/invalidation and access boundaries, not shrinking the bytes.

  • Account for access pattern: Read-heavy caches with large values can justify compression more often than write-heavy caches or ultra-low-latency hot sets.

  • Version and isolate formats: Tag compressed entries with a compression format/version to safely roll encoders/decoders. Never let a mismatch corrupt reads.

  • Preserve cache semantics: Compression must never change cache keys, TTLs, eviction policy (LRU/LFU/FIFO), or your invalidation plan. Cache-aside, read-through, write-through, write-back, and write-around still need the exact same invalidation discipline. Charge eviction against compressed bytes.

When NOT To Use It

Do not use cache compression if: objects are almost all small (where threshold-based compression gains nothing), your node is already CPU-saturated, you need deterministic sub-microsecond hits for a tiny hot set, you cannot version encoders/decoders for rolling upgrades, or the measured hit-ratio gain cannot offset the added p99/p999 latency for this workload.

Mistake to watch for: “compress everything to save memory” without measuring hit ratio and p99/p999 latency together against actual payloads. The correct question isn’t “how much smaller?” it’s “does effective capacity (by compressed bytes with correct eviction) improve enough to offset decompression cost for this workload?”

39

How does machine learning influence caching decisions?

Predictive caching uses machine learning to forecast demand, decide what to cache, and adapt cache policy. It can improve hit ratios and reduce origin load, but it supplements rather than replaces TTLs, invalidation, consistency controls, and graceful fallback.

Read full answer

Applies toall versions

Why Machine Learning Is Used

Rule-based policies such as LRU, LFU, FIFO, and TTL work well for predictable workloads but can react slowly when demand changes. Machine learning can improve several decisions:

  • Prefetching: Warm objects, CDN locations, or cache shards likely to be requested soon.

  • Admission: Cache an object when its expected reuse justifies the storage and lookup cost.

  • TTL selection: Give volatile data a shorter lifetime and stable data a longer one, within business consistency limits.

  • Eviction: Remove entries unlikely to be reused instead of using only recency or frequency.

  • Traffic classification: Apply different policies to anonymous, premium, and likely one-time-user traffic without weakening authorization.

For example, an e-commerce service can predict demand after a campaign starts and prewarm CDN entries. A news service can predict which articles will become popular and fetch them before requests arrive.

Strategy Fit

Machine learning can complement each standard caching strategy:

Strategy How It Works Useful ML Enhancement
Cache-aside Application checks the cache, then loads from the database on a miss Predict hot keys and prewarm them
Read-through Cache loads missing data from the database Proactively warm entries likely to be requested
Write-through Write updates the cache and database in the same write path Prepare related entries for likely reads
Write-back Write updates the cache before the database Schedule flushes, while accepting possible data loss
Write-around Write bypasses the cache Avoid polluting cache capacity with write-heavy, rarely read data

Cache-aside is usually a practical default for web applications. Write-through simplifies the read path but makes writes slower. Write-back can reduce write latency but requires durable recovery procedures. Write-around fits write-heavy data that is rarely read.

Where ML Predictions Apply

  • Browser and HTTP caches: Clients apply server-provided Cache-Control, ETag, max-age, and s-maxage directives. The server can tailor these directives and trigger CDN prefetching.

  • CDN and reverse proxy: Predictive warming, adaptive TTLs, and geographic prefetching can reduce origin requests.

  • Application cache: Models can estimate reuse probability and avoid caching one-off queries.

  • Distributed cache: An orchestration layer can control admission, TTLs, and hot-key placement, although Redis and Memcached do not natively expose arbitrary ML eviction policies.

  • Database caches: Buffer pools remain database-managed; workload-aware memory allocation is possible, but transactional correctness must not be weakened.

Redis and Memcached differ in important operational ways:

Capability Redis Memcached
Data structures Strings, hashes, lists, sets, sorted sets, and streams Primarily opaque key-value bytes
Persistence Optional RDB snapshots and append-only-file persistence No built-in persistence
Replication Built-in primary-replica replication No built-in replication
Threading Command execution is primarily single-threaded Multithreaded server

Choose Redis when richer data types, persistence, or replication are required. Choose Memcached for a simple, highly concurrent key-value workload where those features are unnecessary.

Consistency and Failure Modes

Predictions can be wrong, so the system must tolerate stale or incorrectly prefetched data. Use TTLs, explicit invalidation events, and versioned keys where appropriate.

  • Cache stampede: Concurrent misses on an expiring key can overload the database. Use request coalescing, a short-lived lock, stale-while-revalidate, or early refresh with jitter.

  • Cache penetration: Repeated requests for nonexistent objects cause repeated cache misses and origin lookups. Cache negative results briefly and consider a Bloom filter.

  • Hot keys: One key can overload a shard or instance. Replicate or distribute reads for that key, or add a local cache; ordinary sharding alone does not split a single key.

  • Prediction errors: Useless prefetches can evict useful entries. Limit prefetch capacity and retain a simple eviction baseline.

  • Model failure: Fall back to normal cache-aside behavior rather than blocking requests.

  • Stale data: Select TTLs according to business tolerance, not only hit-rate goals. Payments, authorization, and inventory reservations generally require validation against the authoritative system.

  • Sensitive data: Do not place per-user sensitive data in a shared cache unless keys, authorization, encryption, and isolation prevent cross-user access.

Trade-offs and Best Practices

Machine learning adds model-serving latency, training infrastructure, monitoring effort, and operational complexity. Incorrect decisions can reduce hit ratios, increase database traffic, or evict valuable data. Do not add it when ordinary TTL and eviction policies already meet the service-level objective; it is also a poor fit for low-traffic services or data requiring immediate consistency.

Start with measurable baselines:

  • Track hit ratio = hits / (hits + misses) by layer and traffic class, while weighting misses by their latency and origin cost.

  • Measure origin QPS, database load, eviction rate, stale-read rate, and latency percentiles.

  • Compare the ML policy with a simple fallback during normal and peak traffic.

  • Bound prefetching so it cannot exhaust cache capacity or overload origins.

  • Monitor model drift, feedback quality, and policy regressions.

The key interview point is that machine learning can optimize what is cached and when, but a trustworthy design still needs explicit invalidation, bounded staleness, stampede protection, sensitive-data isolation, and graceful degradation.

40

How does caching improve IoT applications?

Caching stores frequently requested data closer to consumers, reducing latency, bandwidth use, and backend load. In IoT it is especially useful for constrained devices and intermittent links, but adds staleness, invalidation, and cache-failure risks.

Read full answer

Applies toall versions of IoT platforms and supporting HTTP, MQTT, and CoAP protocols

Why IoT Systems Need Caching

IoT combines large device fleets with slow networks, limited hardware, and frequent state changes. Devices often need configuration quickly, while backends receive high-volume telemetry.

  • Lower latency: Edge or application caches may respond in roughly 1–10 ms versus roughly 50–150 ms for a cross-region database round trip.

  • Reduced bandwidth: Local state and reference data avoid repeated upstream downloads.

  • Backend protection: A 95% hit ratio can remove about 95% of origin reads, assuming each miss results in one origin read.

  • Offline resilience: Gateways can serve recent state during upstream outages.

  • Higher throughput: Distributed caches absorb repeated reads and scale horizontally.

These gains cost temporary inconsistency, extra infrastructure, and operational complexity. Never make a cache the sole authority for safety-critical or unrecoverable data.

For example, in a smart factory, an edge gateway can cache production schedules, equipment metadata, and recent sensor readings. Local controllers avoid cloud round trips and continue routine schedules during a WAN outage; safety interlocks remain outside the cache.

Where Caching Fits

Device → Local or edge cache → Gateway → Regional cache → API service → Database or time-series store

  • Device cache: Stores small configuration snapshots and measurements within RAM and flash limits.

  • Edge gateway cache: Serves nearby devices, supports disconnected operation, and reduces cloud traffic.

  • Application cache: Holds device metadata, latest state, authorization decisions, and reference data.

  • Distributed cache: Shares hot entries across API instances using Redis or Memcached.

  • Browser and CDN cache: Supports management dashboards, device catalogs, and firmware metadata through Cache-Control, max-age, s-maxage, and ETag.

  • Database buffer cache: PostgreSQL uses shared buffers backed by the operating system’s page cache; it does not have a general query-result cache.

Raw, high-volume telemetry is often better sent to a message broker or time-series database. Cache the latest value, rollups, or frequently queried windows rather than every event.

Cache Strategies

Strategy Behaviour Appropriate IoT use
Cache-aside Application loads from the origin on a cache miss General metadata and configuration
Read-through Cache loads missing values automatically Shared cache abstractions
Write-through Cache and origin are updated synchronously Settings requiring immediate visibility
Write-back Cache is updated first and persisted later Local counters, with crash-related data-loss risk
Write-around Writes bypass the cache; a later read populates it Write-heavy, read-later workloads

Cache-aside is a common default, but it still requires explicit TTLs, invalidation, and failure handling. Write-through increases write latency and requires both stores to be available. Write-back improves throughput but can lose data after a crash.

Invalidation and Consistency

Every cached value needs an explicit expiry or invalidation policy:

  • Use a TTL to bound staleness even when events also invalidate entries.

  • Publish device-state change events for affected keys.

  • Use an authoritative monotonic version in the key or value to reject older updates; do not rely on skewed device clocks for ordering.

  • Use short TTLs for rapidly changing state and longer TTLs for immutable reference data.

  • State that consistency is eventual because a partitioned edge node may temporarily report old state.

Write-through alone does not solve concurrent updates or cross-region replication. Critical commands need version validation and idempotency rather than relying on cache ordering.

Failure Modes and Protection

  • Cache stampede: Many requests can miss one hot key simultaneously. Use request coalescing, a short distributed lock, or early recomputation with randomized jitter.

  • Cache penetration: Repeated requests for nonexistent device IDs can exhaust the cache. Cache negative results briefly and consider a Bloom filter when IDs are predictable.

  • Hot keys: One entry can overload a shard. Replicate read-heavy entries or partition them at the application layer.

  • Eviction: LRU favors recent access, LFU favors frequent access, FIFO is simple, and TTL expires entries by deadline. TTL can be combined with LRU or LFU.

  • Cache outage: Fall back to the authoritative store with rate limits and circuit breakers, and shed noncritical reads if the origin cannot handle the traffic.

Tool Choice and Best Practices

Characteristic Redis Memcached
Data model Core types include strings, hashes, lists, sets, sorted sets, and streams Opaque key-value items
Persistence Optional RDB snapshots and append-only file None
Replication Built-in primary-replica replication None
Command execution Mostly single-threaded command execution Multi-threaded server
Typical fit Rich data types, persistence, replication, sessions, streams, and pub/sub Simple high-throughput shared caching

Choose Redis when its data structures, persistence, replication, or pub/sub capabilities are required. Choose Memcached for straightforward, high-throughput key-value caching where durability and replication are unnecessary.

Do not put private per-user data or unrecoverable command state in a shared cache. If secrets must be cached, encrypt them and enforce access controls. Avoid caching short-lived device authentication tokens unless their lifetime and revocation semantics are controlled.

Interviewers look for a defined consistency model, an invalidation plan, and a cache-failure fallback. The common mistake is adding Redis without specifying what is cached, when entries expire, or what happens when the cache is unavailable.

42

How do you configure HTTP caching in Nginx and Apache?

HTTP caching in nginx and Apache combines response cache directives with optional server-side caches. Browser/CDN caching, reverse-proxy caching, and origin protection are separate layers, so each needs explicit rules for freshness, reuse, and invalidation.

Read full answer

Applies tonginx 1.26+ and Apache HTTP Server 2.4.10+

Cache Scope and Directives

Caching reduces repeated origin work, lowering latency and protecting databases or downstream services. The response must also identify who may store it:

  • Cache-Control: public: Allows browser and shared caches such as a CDN to store the response.

  • Cache-Control: private: Allows browser storage but prohibits shared caches.

  • max-age=300: Allows reuse for 300 seconds.

  • s-maxage=300: Sets freshness for shared caches; browsers ignore it.

  • no-cache: The response may be stored, but must be revalidated before reuse.

  • no-store: The response must not be stored.

  • must-revalidate: Once stale, the response cannot be reused without contacting the origin.

  • ETag or Last-Modified: Enables conditional requests and 304 Not Modified responses.

Do not place personalized or sensitive responses in a shared cache. Partitioning by user or tenant must include every input that changes authorization or representation, such as relevant cookies, headers, and query parameters.

Cache Strategies and Invalidation

Choose the write strategy according to the consistency requirement:

Strategy How it works Best fit Main trade-off
Cache-aside Application reads the cache and loads from the database on a miss General application caching A possible stale-data window
Read-through The cache loads missing data itself Cache-centric systems Greater coupling to the data source
Write-through Update the cache and database synchronously Data that must be immediately readable Higher write latency and partial-failure handling
Write-back Update the cache immediately and persist later High-write workloads with controlled staleness Data loss if persistence fails
Write-around Write to the database while bypassing or invalidating the cache Write-heavy or rarely read data Lower hit ratio

Use TTLs as a staleness bound and explicit invalidation or versioned keys for faster updates. A transactional outbox can make event-driven invalidation reliable. A successful database update followed by a failed cache deletion leaves stale data; deleting the cache after a failed database update normally causes only an unnecessary miss.

Code Example

The nginx configuration enables a reverse-proxy cache for a public API and a separate browser policy for versioned static assets:

worker_processes auto;

events {
    worker_connections 1024;
}

http {
    # The parent directory must exist and be writable by nginx.
    proxy_cache_path /var/cache/nginx
        levels=1:2
        keys_zone=api_cache:10m
        max_size=1g
        inactive=60m
        use_temp_path=off;

    upstream app_backend {
        server 127.0.0.1:8081;
    }

    server {
        listen 8080;
        server_name _;

        location /assets/ {
            # Files should be content-hashed or versioned.
            root /var/www/html;
            add_header Cache-Control "public, max-age=31536000, immutable" always;
        }

        location /api/ {
            proxy_pass http://app_backend;

            proxy_cache api_cache;
            proxy_cache_key "$scheme$host$proxy_host$request_uri";
            proxy_cache_valid 200 5m;
            proxy_cache_valid 404 1m;

            # Coalesce concurrent requests for the same expired key.
            proxy_cache_lock on;
            proxy_cache_lock_timeout 5s;

            # These cookie names are illustrative.
            proxy_cache_bypass $http_authorization $cookie_session;
            proxy_no_cache $http_authorization $cookie_session;

            proxy_cache_background_update on;
            proxy_cache_use_stale error timeout updating
                http_500 http_502 http_503 http_504;

            add_header X-Cache-Status $upstream_cache_status always;
        }
    }
}

proxy_cache_valid supplies fallback freshness when the origin does not send suitable caching headers; it does not safely override an explicit no-store or private policy.

The following is an Apache server-config fragment (LoadModule is not allowed inside a virtual host). The required modules must be installed, and the cache directory must be writable by Apache:

LoadModule headers_module modules/mod_headers.so
LoadModule cache_module modules/mod_cache.so
LoadModule cache_disk_module modules/mod_cache_disk.so

CacheRoot /var/cache/apache

CacheEnable disk /public-api/
CacheDefaultExpire 5 minutes
CacheMaxExpire 1 hour
CacheMaxFileSize 1000000

# Responses marked Cache-Control: private or no-store are not stored by default
# (CacheStorePrivate Off, CacheStoreNoStore Off). Clients' own no-cache request
# headers are still honoured because CacheIgnoreCacheControl defaults to Off.

# Apply only to an endpoint whose responses are always public.
<Location "/public-api/">
    Header always setifempty Cache-Control "public, max-age=60"
</Location>

setifempty preserves a Cache-Control supplied by the application and only provides a fallback when none exists.

Failure Modes and Operating Trade-offs

  • Cache stampede: Many requests populate one expired key. Use request coalescing, proxy_cache_lock, or early recomputation with randomized jitter.

  • Cache penetration: Repeated missing keys consume origin resources. Cache negative results briefly or use a Bloom filter.

  • Hot keys: One popular key can overload a cache node. Replicate hot data or use a small local cache in front of the shared cache.

  • Cache outage: Cache-aside systems normally fall back to the origin with rate limiting. Authentication or authorization services may instead need to fail closed.

  • Eviction: Combine TTL expiry with a maximum size. Redis and Memcached commonly use LRU-like eviction; LFU better favors strongly skewed workloads, while FIFO is simpler but ignores reuse frequency.

  • Hit ratio: A 90% hit ratio removes roughly 90% of equivalent origin reads, but it does not guarantee an equivalent latency improvement.

Common Pitfalls

  • Adding a cache without defining cache ownership, expiry, and invalidation.

  • Using a cache key that omits an authorization or representation dimension.

  • Setting a long TTL without an invalidation strategy.

  • Assuming every application generates a strong ETag.

  • Serving stale data when a business operation requires an immediate, consistent read.

For payments, inventory reservations, and other transactional decisions, prefer strong consistency and treat caching as an optimization with explicit consistency boundaries.

43

How do HTTP headers control web caching?

HTTP cache headers tell browsers, CDNs, and reverse proxies whether a response may be stored, how long it remains fresh, and whether it must be validated against the origin. They reduce latency and origin load, but they introduce stale-data, privacy, and invalidation risks when caching rules are poorly designed.

Read full answer

Applies toHTTP/1.1, HTTP/2, and HTTP/3 using RFC 9110 and RFC 9111 semantics

How HTTP Headers Drive Caching

A cache uses response headers, plus a few request headers, to make three main decisions:

  1. May it store the response? Directives such as no-store and private restrict storage.

  2. Is the response fresh? max-age, s-maxage, and Expires determine freshness.

  3. What must be included in the cache key? Vary identifies request headers that affect the representation.

Without explicit freshness information, caches may apply heuristics based on response status and Last-Modified. Such behavior is less predictable than configuring an explicit caching policy.

Important Response Headers

Header Purpose
Cache-Control Primary policy mechanism using directives such as public, private, no-cache, no-store, max-age, and s-maxage.
ETag Opaque validator for a representation; clients use If-None-Match to check whether it changed.
Last-Modified Timestamp-based validator, commonly paired with If-Modified-Since.
Expires Absolute expiry mechanism; Cache-Control takes precedence.
Vary Request headers that influence the selected response, such as Accept-Encoding or Accept-Language.
Age Estimated time since the response was generated; caches normally add it.
Date Message origination date, used when calculating a response’s age.

Common Cache-Control directives include:

  • no-store: The cache must not store any part of the response.

  • no-cache: The response may be stored, but it must be successfully revalidated before reuse.

  • private: A shared cache must not store it; a browser may store it unless another directive forbids storage.

  • public: A shared cache may store it, including when the request contains Authorization; the application must still ensure the representation is safe to share.

  • must-revalidate: A connected cache must not serve stale content after expiry; RFC 9111 permits stale data when disconnected from the origin.

  • stale-while-revalidate: The cache may serve stale content while revalidating asynchronously, unless the request forbids it with no-cache.

  • stale-if-error: The cache may serve stale content when the origin fails, unless the request forbids it with no-cache.

Freshness and Revalidation

A response is fresh when its calculated age has not exceeded its freshness lifetime. A response-level max-age=60 normally permits reuse without contacting the origin for up to 60 seconds, subject to age calculations and request overrides.

When a stored response becomes stale, the cache can revalidate it:

  • If-None-Match sends the previous ETag. If the representation has not changed, the origin returns 304 Not Modified, and the cache reuses its stored body.

  • If-Modified-Since provides a weaker timestamp-based alternative.

Revalidation saves response-body transfer and may avoid generating a full representation, but it still consumes an origin request and validator-check work. It therefore may not reduce latency substantially.

Browser, Shared-Cache, and Request Controls

A browser cache is private to one user and may store that user’s personalized responses. A CDN or reverse-proxy cache is shared, so privacy and authorization rules are critical.

For shared caches:

  • s-maxage takes precedence over max-age and Expires.

  • private prevents shared storage; public permits it but does not make user-specific data safe.

  • Vary must include every request header that changes the representation, otherwise one user could receive another user’s content.

Request-side Cache-Control has different meanings:

  • no-cache: Revalidate before reuse.

  • min-fresh=N: Reject a response that would become stale within the next N seconds.

  • max-stale=N: Accept a response that is no more than N seconds stale, unless the response prohibits stale reuse.

  • only-if-cached: Return a suitable stored response or 504 Gateway Timeout; do not contact the origin.

Code Example

The request explicitly requests validation. The response policy otherwise permits browser reuse for 60 seconds and shared-cache reuse for five minutes. Because this request contains no-cache, the cache must obtain successful validation and cannot use stale-while-revalidate for this request.

GET /products/42 HTTP/1.1
Host: shop.example
Accept-Encoding: gzip
Cache-Control: no-cache
If-None-Match: "product-42-v7"

HTTP/1.1 304 Not Modified
Date: Thu, 02 Jan 2025 00:00:00 GMT
ETag: "product-42-v7"
Cache-Control: public, max-age=60, s-maxage=300, stale-while-revalidate=60
Vary: Accept-Encoding
Age: 12

The 304 response has no body; the cache reuses the stored body. Vary: Accept-Encoding separates compressed and uncompressed variants, while content negotiation remains responsible for selecting an encoding the client supports.

Best Practices and Common Pitfalls

  • Set explicit policies: Prefer Cache-Control over Expires or heuristic freshness.

  • Choose by audience: Use private for user-specific responses and s-maxage for shared caches.

  • Plan invalidation: Changing an ETag only affects later revalidation. Use a short TTL, an explicit CDN purge, or a versioned URL when cached copies must be bypassed immediately.

  • Protect hot keys: Use request coalescing, locking, or early expiry with jitter to prevent a stampede.

  • Prevent penetration: Cache negative lookups briefly or use a Bloom filter when invalid identifiers create repeated origin load.

  • Measure hit ratio: Estimate savings as hits multiplied by avoided origin work, then subtract cache, storage, and invalidation costs. Avoid caching frequently changing or low-hit-ratio data.

  • Protect sensitive data: Do not place per-user or confidential responses in a shared cache without an explicit, reviewed policy.

  • Test end to end: Inspect actual Age, cache-status, and validator behavior at the browser, CDN, and reverse proxy.

The common interview mistake is adding Cache-Control without defining cache ownership, invalidation, privacy, failure behavior, and the expected hit ratio.

44

How do caching approaches differ across programming languages?

Caching approaches differ across programming languages mostly because of runtime behavior (memory/GC, concurrency model, and object identity/serialization), not because of built-in “cache” keywords. Most production caching is either instance-local (in-process) or shared via a language-agnostic external store; the trade-off is between raw speed and cross-instance consistency.

Read full answer

Applies tolanguage runtimes in general (language-agnostic comparison)

The Problem It Solves

Before picking an approach, the problem is the same: avoid repeated expensive work. For a concrete example, a news feed may need to serve the same pre-computed list to many requests. With rough orders of magnitude, a memory read is ~100 ns, an SSD read ~100 µs, a same-datacenter round trip is ~0.5 ms, and a cross-region round trip is ~50-150 ms (all rough orders of magnitude). The closer you get to ~100 ns, the harder it becomes to keep that copy correct across a load-balanced fleet.

How Language Runtimes Drive the Difference

  • Memory ownership & GC: Runtimes with automatic garbage collection (e.g. Java, C#, Go, Python) can easily hold object graphs in-process, but heap growth, GC pauses and finalization affect when entries can be safely evicted. In languages closer to manual memory (e.g. C), you can pin structures and avoid GC pressure, but leaks and lifetime errors become your responsibility.

  • Concurrency model: A GIL-bound runtime (e.g. CPython) changes how threads contend compared to a thread-per-core or multi-threaded runtime (e.g. Java, Go). Async/await event loops shift the race from “many OS threads” to “many concurrent tasks”. Preventing a double fetch (cache stampede) may require locks, read-write locks, concurrent maps, or single-flight/request coalescing depending on those primitives.

  • Object identity, hashing & serialization: Keys/values must be hashed and compared correctly. Equality by reference vs by value, and the cost of serialization when moving data to a distributed cache, vary by language. Version skew between processes is a cross-language concern, but surfaces differently depending on your serialization choices.

  • Ecosystem: Only tiny memoization exists in many standard libraries. Production-grade caching is usually a vetted library or an external service, so the “language difference” is often which concurrency/collections/serialization tools you reach for, not a mandated API.

In-Process vs External: Trade-Offs

Aspect In-process (per instance) External (e.g. Redis, Memcached)
Scope Per process. Lost on restart, not shared across instances. Shared across instances. Survives process restarts depending on persistence/configuration.
Latency Closest to ~100 ns, no network hop. Adds a network hop (~0.5 ms same-datacenter rough order of magnitude) in exchange for sharing.
Consistency Easier to reason about for that single process, but quickly diverges across a scaled-out fleet (stronger local read-after-write, weaker global consistency). Prone to cache–database divergence; must plan invalidation/TTL. Can use atomic operations, but still eventual in practice without strict coordination.
Concurrency safety Must use the runtime’s synchronization (e.g. locks, read-write locks, concurrent collections, or single-flight). Cost depends heavily on the concurrency model. Rely on the store’s commands and client-side logic. Pushes some atomicity to the server, but network races (e.g. stampede between instances) remain.
Eviction Often policy-driven by the library/heap (size-based LRU/LFU, TTL). Can be impacted by GC pressure. Driven by maxmemory + eviction policy and key TTLs set by your application.
Cost/ops Low infra, but horizontal scaling forces duplication or invalidation complexity. Extra service to run/monitor, but avoids per-instance duplication and keeps shared hot data consistent enough for read-heavy paths.

Redis vs Memcached (Honest Comparison)

Feature Redis Memcached
Data structures Richer (strings, hashes, lists, sets, sorted sets, etc.) useful for more than raw key-value. Primarily simple key-value strings.
Persistence Supports persistence options (depending on configuration) and replication features. Generally in-memory only with no built-in durable persistence by design.
Replication/HA Has replication and high-availability oriented features; trade-offs depend on topology. Simpler replication/HA approaches vary; less built-in than Redis in most common setups.
Threading Primarily single-threaded for command execution with I/O/event loops, relying on atomic commands for many operations. Often multithreaded, which can help with certain workload shapes but trades off some built-in atomic high-level operations.
Fit Better when you need data structures, TTLs, atomic ops, or optional persistence. Better for very simple, throwaway key-value caching where richness and persistence aren’t required.

Practical Guardrails (Cross-Language)

  • Scope shared caches: Never cache per-user sensitive data (tokens, PII) on a globally shared cache namespace without strict key scoping and a review of exposure. The risk is the same in Python, Java, Go, or C#.

  • Protect against workload failures: Cache stampede/thundering herd, hot keys, and cache penetration are workload problems, not language bugs. Use request coalescing/single-flight per key, locking with care, jittered early expiry, short TTLs for negative results, or a bloom filter where appropriate. The exact primitives differ by runtime.

  • Be explicit about consistency: Don’t rely on in-process maps to be “eventually consistent” across a fleet. For strict read-after-write or instant revocation (e.g. authz decisions), be cautious with caching or add an explicit invalidation/versioning plan.

  • Hit-ratio reasoning: A high hit ratio can hide cold-path latency or hot-key skew. Treat it as workload-dependent (e.g. a news feed vs a payment service).

  • Don’t paper over access patterns: Do not add a cache purely to cover an unindexed database query without fixing that query/access pattern first. Fixing the source of truth is often cheaper and safer than masking it.

When to Use, and When Not To

  • Use in-process caching when data is instance-local, mostly read-only for the process lifetime, or for a tight hot path with known single-process bounds. Accept that it won’t share across instances.

  • Use a distributed/external cache when you scale horizontally (multiple instances) and need shared reads (e.g. short->long lookups in a URL shortener). Budget for invalidation, network latency (~0.5 ms same-datacenter rough order of magnitude), and operational effort.

  • Do NOT use caching where the source of truth must be strongly consistent with zero tolerance for stale reads, for highly mutable per-request authorization that must reflect revocations instantly, or to obscure correctness issues. Nothing is free: every layer of caching adds complexity and the risk of divergence.

Common Pitfalls & Interview Focus

  • Mistake to avoid: Adding a cache without an invalidation plan, or assuming functools.lru_cache (or a concurrent dictionary) will behave identically across a load-balanced fleet.

  • What interviewers look for: The distinction between “language feature” and “system component”. Be able to say cross-language differences are mostly runtime/concurrency/GC/serialization, while external caches normalize most of that, and state the trade-offs.

  • Quick follow-up: For a hot key, how would your cache-aside path behave differently between a GIL-bound runtime and a multi-threaded runtime? Focus on stampede protection and synchronization, not which cache store is “faster” in that language.

45

How is caching used in mobile application development?

Caching in mobile applications stores reusable data close to the user so repeated requests can avoid network or database work. It reduces latency and data usage and can support offline experiences, but introduces staleness, invalidation complexity, storage overhead, and security risks.

Read full answer

Applies tomobile applications on currently supported iOS and Android versions

Where Caching Fits

A cacheable mobile request may follow this path:

Mobile app → local HTTP cache → CDN/reverse proxy → API service → distributed cache → database

  • Memory cache: Holds hot objects in RAM; it is fastest but volatile and size-limited.

  • Disk cache: Stores images, files, or API responses across restarts; it needs size and cleanup management.

  • HTTP cache: Uses Cache-Control, ETag, max-age, and s-maxage; max-age defines freshness, while s-maxage overrides it for shared caches.

  • CDN cache: Serves cacheable static assets or API responses from an edge, reducing origin traffic.

  • Reverse-proxy cache: A proxy such as Nginx can cache eligible HTTP responses before requests reach the API service.

  • Application cache: Stores domain objects such as news feeds, product catalogs, or search results.

  • Offline cache: Persists essential data for read-only use or queued work during connectivity loss.

  • Database cache: Buffer pools cache database pages, while optional result caches reuse repeated work and reduce database I/O.

An in-memory lookup may take roughly microseconds, while a mobile network request commonly takes tens or hundreds of milliseconds, depending on signal and geography.

Cache Strategies

Strategy Behavior Best fit Main trade-off
Cache-aside The caller reads the cache; on a miss it reads from the API or origin and fills the cache Read-heavy data Cache and origin can diverge
Read-through The cache fetches missing data from the origin for the caller Uniform service interface Extra dependency or network hop
Write-through Cache and authoritative store are updated synchronously Read-after-write data Higher latency; atomicity needs care
Write-back The cache is updated first and persistence happens asynchronously Buffered or offline writes Possible loss before durable persistence
Write-around A write bypasses the cache; a later read repopulates it Frequently changing or rarely read data The next read is slower

On mobile, cache-aside is usually safest because a client miss fetches from the API. Use write-back for offline edits only with a durable local outbox; payments must be durably recorded before success is acknowledged.

Invalidation and Consistency

Choose an explicit consistency model:

  • Short TTL: Entries expire after a defined interval, providing simple eventual consistency.

  • Versioned keys: Include a version such as feed:v42; publishing a new version makes old entries unreachable without immediately deleting them.

  • Event-driven invalidation: Publish an event when authoritative data changes, but account for messaging delays or lost events.

  • Conditional refresh: Send If-None-Match with an ETag; the server can return 304 Not Modified when the copy is current.

  • Read-after-write: Update or invalidate the affected entry after a successful write when immediate visibility is required.

TTL should be a safety net, not the only invalidation mechanism; a five-minute price TTL can violate a stricter pricing rule.

Eviction, Stale Data, and Failure Modes

  • LRU evicts the least recently used entry.

  • LFU favors frequently requested entries, but can retain obsolete data without invalidation.

  • FIFO evicts in arrival order without considering popularity.

  • TTL expiry removes entries based on age regardless of usage.

Also plan for these failure modes:

  • Cache stampede: Many clients request the same expired entry. Use request coalescing, a short-lived lock, jittered early refresh, or stale-while-revalidate.

  • Cache penetration: Invalid or malicious keys repeatedly miss. Cache negative results briefly and consider a Bloom filter for a large key space.

  • Hot keys: Add local caching or deliberately distribute reads across replicas; one cache shard may otherwise become a bottleneck.

  • Stale data: Decide whether staleness is acceptable. Do not treat cached authorization or payment state as authoritative.

  • Cache outage: Fall back to the origin or a limited response. Use timeouts and a circuit breaker because a cache outage can otherwise overload the origin.

Security and Server-Side Caches

Do not place sensitive per-user data in shared caches unless keys, values, access controls, and tenant boundaries are enforced. Store credentials in iOS Keychain or Android Keystore rather than ordinary cache files, and clear user-specific caches during logout.

Redis supports richer data structures, persistence, replication, and Pub/Sub messaging for invalidation. Pub/Sub is lossy, so retain TTL or version checks. Memcached is a simpler, multithreaded, memory-oriented key-value cache with no built-in persistence or replication.

Practical Guidance

  • Measure the hit ratio, hits / (hits + misses), alongside tail latency, origin load, storage use, and stale-data incidents.

  • Cache immutable assets aggressively and personalized or authorization-sensitive data conservatively.

  • Set size limits, TTLs, and an eviction policy before deploying a mobile cache.

  • Make cache operations fail safely: use short timeouts, a circuit breaker, and an authoritative-service fallback.

  • Avoid caching without an invalidation plan—the common mistake is treating cache-aside as a performance switch rather than a consistency system.

46

Which tools monitor and analyze cache performance?

Cache performance monitoring is the practice of using layer-specific metrics, logs, traces, and diagnostics to measure hit rate, latency, capacity, evictions, and failures in a cache so you can detect correctness and overload issues.

Read full answer

Applies toall versions

Tools by Caching Layer

Layer Tools to monitor/analyze Key signals to watch
Browser Browser DevTools (Network panel) Cache-Control, ETag, Age, origin vs memory/disk cache, request timing
CDN CDN logs and provider analytics Cache-Control, s-maxage, Age, x-cache/vendor Cache-Status equivalents, edge hit ratio by path/region
Reverse proxy Nginx access logs ($upstream_cache_status), Varnish (varnishstat, varnishlog, varnishncsa) Cache status (HIT/MISS/BYPASS/EXPIRED), request rate, latency, and cache size usage
Application OpenTelemetry traces, structured logs, Micrometer/custom counters Hit/miss per cache region/key namespace, lookup latency, timeouts, fallbacks, and correlation to origin latency
Redis redis-cli INFO (memory, stats, commandstats), SLOWLOG, LATENCY commands; Prometheus redis_exporter + Grafana Hit/miss (via keyspace_hits/keyspace_misses), memory/evictions, connected clients, command latency, slow commands
Memcached stats, stats items, stats slabs, stats settings; Prometheus memcached_exporter Get/hit/miss/evictions, bytes used/maxbytes, curr_items, cmd_get/cmd_set, connection counts
Database buffer cache (for comparison) PostgreSQL pg_stat_database (e.g. blks_hit/blks_read) and pg_stat_statements; equivalent vendor views Buffer hit ratio for reads, but treat as different from application-cache hits

Metrics to Monitor

  • Hit ratio: hits / (hits + misses). Interpret against volume (a high ratio can still hide expensive misses at scale).

  • Latency: Track p50/p95/p99 for hits, misses, writes, and timeouts separately.

  • Capacity and evictions: Memory used vs max, item count, evictions, and expiration rate.

  • Traffic and errors: Operations per second, timeouts, connection failures, rejected connections, and fallback usage.

  • Application impact: Correlate with origin latency and load (a cache can mask or shift pressure).

Best Practices and Common Pitfalls

  • Alert on symptoms: Page on sustained errors, origin overload, or unacceptable p99 latency—not a single dip in hit ratio.

  • Avoid high-cardinality labels: Don’t label metrics with raw keys, user IDs, or full URLs in time-series stores.

  • Time out and degrade: Bounded cache timeouts, circuit breakers, and bulkheads prevent cache failure from becoming an origin outage.

  • Protect shared caches: Don’t put per-user sensitive or authorization-deciding data in a shared cache without strict namespacing, access control, and short TTLs.

  • Keep the source of truth: Treat cache data as disposable. If monitoring shows invalidation is untrackable or savings don’t justify complexity, don’t cache it.

The most common interview mistake is adding a cache without monitoring hit ratio, evictions, latency, and failure modes alongside it.

47

How does caching integrate with cloud computing services?

Caching stores frequently requested data closer to its consumers, reducing origin traffic, network calls, latency, and cloud infrastructure cost. In a cloud architecture, it can operate at the browser, CDN, reverse-proxy, application, distributed-cache, and database layers, with trade-offs involving stale data, invalidation work, memory use, and availability.

Read full answer

Applies toHTTP caching semantics (RFC 9111), Redis 8.x, Memcached 1.6.x, and version-independent cloud cache patterns

Where Caching Fits

A common cloud request flow is:

Client → CDN → API gateway or reverse proxy → application → distributed cache → database

  • Browser cache: Stores resources on the device. Cache-Control: max-age sets freshness, while ETag enables revalidation. Use no-store for highly sensitive responses.

  • CDN cache: Caches static or cacheable API responses at edge locations. s-maxage governs shared caches; cache keys must distinguish response variants and tenants.

  • Reverse-proxy or application cache: Nginx, Envoy, or process-local memory can serve requests before the database. Local caches are fast but unshared and potentially inconsistent.

  • Distributed cache: Redis or Memcached shares entries across instances, adding network latency and an operational dependency.

  • Database cache: Buffer pools, supported query caches, and read replicas reduce read work; replicas add consistency lag and operational cost.

Cache Strategies

Strategy Behavior Best fit Main cost
Cache-aside Application checks the cache; on a miss, it reads the database and fills the cache General-purpose caching Duplicate reads on concurrent misses
Read-through Cache loads missing data from the database for the application Uniform application access code Cache becomes a runtime dependency
Write-through Cache and database are updated synchronously Immediate read-after-write visibility Slower and less available writes
Write-back Cache is updated first and persistence happens asynchronously Counters and telemetry Data loss and reconciliation risk
Write-around Write goes to the database; a later read fills the cache Read-once data Little cache benefit

Cache-aside is generally the safest default because the cache can be bypassed and is not the authoritative store.

Consistency and Invalidation

Cache-aside normally permits eventual consistency unless the application coordinates updates and invalidation explicitly.

  • TTL expiry: Bounds staleness and provides a simple safety net when invalidation fails.

  • Event-driven invalidation: A domain event deletes or updates related entries. Retries, durable delivery, and monitoring reduce the risk of missed invalidations.

  • Versioned keys: A new version can make old entries unreachable after all readers and writers switch to it; old versions should still expire.

  • Delete after commit: Invalidating before the database commits can allow a concurrent reader to reload and cache the old value.

  • Population race: A reader can repopulate stale data after invalidation. Short TTLs, locking, or compare-and-delete operations can reduce this risk.

Failure Modes and Workload Problems

  • Cache stampede: Many requests miss one popular key and overload the origin. Use request coalescing, per-key locking, early expiry with jitter, and origin rate limits.

  • Cache penetration: Repeated requests for nonexistent data bypass the cache. Cache negative results briefly and consider a Bloom filter at larger scale.

  • Hot keys: Split a logical value into partitioned subkeys, replicate it across independent cache instances with explicit routing, or add a local cache.

  • Cache failure: Use short timeouts, degrade to the database only when safe, and protect the database with concurrency limits, circuit breakers, and bulkheads.

  • Eviction: Redis supports approximate LRU and LFU policies; Memcached uses LRU. FIFO is simpler but has weaker locality awareness. TTL controls freshness, not necessarily eviction timing.

Hit ratio is hits / (hits + misses). With a 90% hit ratio, tenfold traffic leaves origin load approximately unchanged; with a 50% ratio, it becomes roughly five times the original load.

Redis vs Memcached

Concern Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, and streams Opaque key-value entries
Persistence Optional RDB snapshots and append-only-file persistence Primarily memory-resident; optional experimental disk-backed extstore exists
Replication Native asynchronous replication and Redis Cluster hash-slot partitioning No native replication or partitioning; clients distribute keys, while some managed services add failover
Concurrency Pipelining and atomic operations; command execution is primarily single-threaded Multithreaded request processing
Typical fit Caching plus counters, locks, queues, sessions, and rate limiting Simple, memory-heavy shared object caching

Use Redis when richer data structures, persistence, or atomic operations are useful; use Memcached for straightforward distributed caching with a minimal feature set.

Production Guidance and Trade-offs

  • Do not place per-user sensitive data in a shared cache unless keys, authorization scope, and response boundaries are explicitly controlled.

  • Set TTLs according to acceptable staleness rather than using one duration for every resource.

  • Avoid large, rarely reused objects and rapidly changing transactional state. For an e-commerce site, product pages may be cached, but cart and checkout state should remain authoritative in the database.

  • Include tenant, locale, authorization scope, and representation version in cache keys when required.

  • Measure hit ratio, eviction rate, origin load, stale-read rate, object size, and cache latency; a high hit ratio can still hide expensive entries.

  • Design cache timeouts, fallback behavior, invalidation ownership, and stampede protection before deployment.

Interviewers usually look for a request-flow design, an explicit invalidation policy, and a cache-failure strategy. The common mistake is adding a cache without defining who removes stale entries or what happens when the cache is unavailable.

48

How does caching operate in content management systems (CMS)?

Caching in content management systems (CMS) is the practice of storing pre-built HTML pages, page fragments, asset responses, or database query results across multiple layers so that repeated requests can be served without re-running the full CMS rendering or querying stack.

Read full answer

Applies toall versions

The Problem It Solves

  • Expensive rendering: Many CMS platforms build pages by pulling from a database, running templates, applying modules/plugins, and assembling blocks. Doing this per request is costly.

  • Scale mismatch: Editorial traffic can spike far beyond what a single CMS application server can generate on-the-fly.

  • Latency: Without caching, users may wait through multiple layers for content that rarely changes between publishes.

Layers Where It Applies

Layer What is cached Typical control Notes
Browser/HTTP Static assets, sometimes cacheable responses Cache-Control, ETag, max-age Respects client-side reuse. Only safe for content not dependent on private session state.
CDN Full public pages, media, and static routes s-maxage, edge rules, path-based purge Great for globally read-heavy content. Can serve from edge but must be purged on publish.
Reverse-proxy cache Full page HTML for anonymous users Vary headers, cache keys by host/path/query Sits between CDN/origin and app. Must respect authentication and cookies. Never share authenticated responses.
Application cache Rendered fragments, menus, navigation, or block outputs In-memory keys, tags by content type/node Avoids re-rendering shared pieces when only part of a page changed. Keep local to the instance with care for cross-instance consistency.
Distributed cache CMS query results, API responses, taxonomy lists, or computed objects Keys by content IDs, languages, sites, or contexts Used to share across multiple app instances via a shared key-value store.
Database query/buffer cache Reused result sets internally DB-internal Helpful, but should not be relied on to replace front-end CMS caching.

Distributed Cache: Redis vs Memcached

Aspect Redis Memcached
Data structures Rich: strings, hashes, lists, sets, sorted sets, streams and more. Useful for menus, counters or complex CMS objects. Primarily simple string key-value pairs. Sufficient for flat, serialized objects only.
Persistence Supports optional persistence (RDB snapshots and/or AOF). Can survive a restart if configured, but not a guarantee of full durability by default. No built-in persistence. Data is lost on restart or process crash.
Replication Supports replication (primary-replica) and high-availability patterns depending on setup. Can help with read scaling. Basic replication not its core strength; often scaled by sharding client-side with no built-in durable replication model.
Threading Primarily single-threaded for the command execution path (with some threaded I/O in modern builds), but still predictable. Must account for blocking commands. Multi-threaded by design, which can be helpful for very simple, high-throughput get/set workloads on certain hardware.
Honest note More flexible, but adds operational complexity (persistence, eviction policy tuning). Simpler and lighter for pure caching, but far less forgiving if you need more than raw key-value or any form of recovery.

Strategies and Eviction

  • Cache-aside (lazy load): On miss, read from database, render as needed, write to cache. Fits read-heavy, mostly static CMS pages. Risk: first request after expiry/purge is slower.

  • Read-through: Cache sits in front and fetches on miss. Less application logic, but the cache must know how to load CMS entities.

  • Write-through: On content save/publish, write to cache and database together. Keeps cache fresh at write time, but adds write latency.

  • Write-back: Write to cache first, flush to database later. Rarely appropriate for published CMS content due to durability concerns. Only consider in very narrow, non-critical cases.

  • Write-around: Write directly to database, invalidate related cache keys. Common for editorial publishing where reading the just-written draft is less critical than avoiding polluting cache.

  • Eviction: LRU is common for general objects, LFU can help for “evergreen” pages, FIFO is blunt, and TTL should be used alongside explicit purge.

  • Hit-ratio reasoning: Aim to measure by route type (e.g. homepage vs search). A high hit-ratio on anonymous public pages is worth it, but forcing one for personalized pages is usually a mistake. Treat hit-ratio and any latency numbers as rough orders of magnitude for planning only (e.g. memory read ~100 ns, SSD read ~100 µs, same-datacenter round trip ~0.5 ms).

Invalidation, Consistency and Protection

  • Real-world example: Take a news feed homepage. A CDN and reverse-proxy cache can serve the anonymous homepage from the edge for seconds/minutes, while the application uses tag-based keys for its “top stories” block. On publish of a breaking story, an event-driven purge must invalidate only the keys tagged to that article and its lists, not every route on the site.

  • Tag-based invalidation: Tie cached entries to content entities (nodes, blocks, menus, taxonomies). Publishing one item should only purge keys with its tag, not the entire site.

  • Event-driven purges: Trigger CDN/reverse-proxy/application purges on create/update/delete/publish/unpublish. Relying only on TTL will leave stale content visible for its full max-age/s-maxage.

  • Vary correctly: Use Vary by Accept-Language, site, or other safe dimensions. Never let a shared public cache serve one user’s authenticated view to another.

  • Cache penetration: For known “not found” routes (e.g. old URLs), cache the negative response with a short TTL and consider a bloom filter to stop hammering the DB on garbage requests.

  • Cache stampede/thundering herd: When many requests miss at the same time (after a mass purge), use locking, request coalescing, or early expiry with jitter to spread the rebuilds.

  • Hot keys: A viral article can become a hot key. Spread by key design and consider local in-memory copies with care, or ensure your distributed cache can handle the burst without creating a single point of failure.

  • Consistency: Public, anonymous pages can tolerate eventual consistency between publish and visibility if purges are reliable. Authenticated, personalized, or gated surfaces should favor correctness over maximum cacheability, since stale personalized data is far harder to excuse than a slightly delayed public post.

Trade-offs and What Not To Cache

  • Trade-offs: Lower origin load and latency vs invalidation complexity, risk of stale data, extra operational effort, money (CDN/distributed cache), and more failure modes to reason about.

  • Do not cache on shared caches: Per-user sensitive data, cart/checkout flows, account dashboards, personalized feeds, previews for unpublished content, or anything gated by authentication/authorization.

  • When not to use it: If content changes on nearly every request with no safe cache key, if your team cannot enforce a purge plan, or if most of your traffic is fully personalized, adding layers of cache will create more bugs than it fixes. Treat “how will we purge this on publish?” as a requirement from day one, equal to “how will we read it?”.

  • Key rule: Caching solves read scale, invalidation is the hard part for a CMS.

49

What are the key caching considerations in serverless architecture?

Serverless caching reuses data closer to requesters to reduce latency and origin load, but ephemeral function memory, cold starts, regional distance, concurrency, and connection setup shape the design. Use HTTP/CDN caching where appropriate and a shared external cache when data must survive invocations.

Read full answer

Applies toall current serverless platform versions (provider-neutral)

Serverless Constraints and Cache Placement

  • Cold starts: A function instance can disappear, so instance-local state is not durable; use an external cache for reusable data.

  • Local memory: It avoids network calls and is useful for small reference data, but each concurrent instance has a separate copy and it cannot be authoritative.

  • Placement: Put the cache near the service. A same-datacenter call may take about 0.5 ms, versus roughly 50–150 ms cross-region.

  • Connections: Initialize supported cache clients outside the handler and reuse them; per-request TLS setup can erase the benefit.

  • Concurrency and cost: A hot key can stampede the origin. Caching adds infrastructure and egress cost, stale-data risk, and operational complexity.

Layer Appropriate use Important limitation
Browser Static and user-specific responses Cache-Control, max-age, and ETag govern reuse
CDN or edge Public pages, images, and APIs Never share user data; s-maxage governs shared caches
Reverse proxy Repeated backend HTTP responses Useful mainly for stateless HTTP traffic
Function memory Small reference data Ephemeral and duplicated across instances
Distributed cache Shared hot data Use regional instances; handle eviction, invalidation, and outages
Database Repeated queries and buffer pages Controlled by the database; benefits depend on reuse

For a product-catalog API, cache the shared catalog at the CDN or in Redis, but make price and inventory freshness explicit.

Read and Write Strategies

Strategy Behavior Serverless fit
Cache-aside Function checks the cache, loads the origin on a miss, and caches the result Default; provides explicit fallback and write behavior
Read-through Cache loads missing data from the origin Convenient when supported; origin failures must still surface or be deliberately handled
Write-through Cache and origin update synchronously Simpler reads; higher write latency and availability coupling
Write-back Cache updates immediately, origin later Fast, but creates data-loss windows
Write-around Write to the origin without updating the cache Existing cached values still require invalidation

With cache-aside, bound miss work and coalesce concurrent misses. During an outage, use rate limits, circuit breakers, and bulkheads before falling back to the database.

Freshness, Eviction, and Failure Modes

  • TTL and eviction: Give every shared entry a TTL as a safety net. TTL expiry is time-based; LRU and LFU evict by recency or frequency, while FIFO evicts the oldest insertion. Actual defaults depend on the cache implementation.

  • Invalidation: TTL does not provide immediate correctness. Delete or version affected keys after writes and define the acceptable stale-data window. For event-driven invalidation, atomically persist an outbox event with the database change, then process it with retries and idempotent consumers.

  • Stampede: Use a short lock, request coalescing, or early refresh with randomized jitter. Serving a bounded-stale value can protect the origin.

  • Penetration: Cache negative results briefly; consider a Bloom filter when the valid key space is predictable.

  • Hot keys: Shard or split hot keys, serve reads from replicas where supported, or add a local cache.

  • Sensitive data: Do not put user-sensitive data in a shared cache without tenant and authorization scoping plus appropriate access controls. Use private for responses shared caches must not store.

Redis and Memcached

Concern Redis Memcached
Data model Strings, hashes, lists, sets, sorted sets, and streams Primarily opaque key-value items with byte values
Persistence Optional RDB snapshots and append-only-file persistence No built-in persistence
Replication Built-in replication and Redis Cluster No built-in replication; scaling is client-driven
Concurrency Command execution is mainly single-threaded, with optional I/O threading Multi-threaded server architecture

Redis fits richer operations, persistence, replication, or coordination. Memcached is simpler for serialized values. Neither removes the need for TTLs, invalidation, or overload protection.

Best Practices and Interview Pitfalls

  • Measure hit ratio from representative traffic using hits / (hits + misses). A high ratio matters only when the origin is materially slower or more expensive.

  • Version and bound keys; namespace them by tenant and authorization scope.

  • Monitor hit ratio, misses, evictions, origin load, latency, errors, and stale responses.

  • Avoid caching one-off reads, write-heavy data, immediately consistent data, or workloads where cache cost exceeds source cost.

  • Use cache-aside with TTL, explicit invalidation, stampede protection, and bounded fallback.

A common interview mistake is introducing Redis without defining invalidation, stale-data tolerance, stampede protection, and outage behavior. Treat the cache as a consistency and reliability component, not merely a latency optimization.

50

What caching strategies apply to big data processing and analytics?

Caching strategies for big data processing and analytics are the deliberate patterns for storing intermediate, aggregated, or frequently queried results closer to the query or compute layer to avoid re-running expensive scans, joins, shuffles, or full-table aggregations.

Read full answer

Applies toall versions

The Problem It Solves

Big data and analytics workloads rarely read a few rows. A single dashboard can trigger multi-stage jobs that scan partitions of a large dataset. Network, disk, or object storage access adds up fast. These numbers are rough orders of magnitude: memory read ~100 ns, SSD read ~100 µs, same-datacenter round trip ~0.5 ms, cross-region ~50-150 ms. Caching exists to cut repeat work, but it only pays off when the same computation is requested often enough to justify the trade-off of extra memory, complexity, and the risk of stale data.

Core Strategies For Read/Write Paths

Strategy How It Works When To Use Trade-Offs
Cache-aside On read: check cache first. On miss, read from the data source, write to cache, return. Writes go directly to the source of truth and do not auto-update cache. The most common fit for analytics dashboards, ad-hoc reports, or read-heavy views where data changes at known intervals. Simple, but cache can go stale until TTL expires or explicit invalidation. Prone to cache stampede/thundering herd on a mass miss.
Read-through The cache sits between the caller and the source. The cache itself is responsible for loading missing entries from the source. Good when you want to centralize “cache on miss” logic away from every service/job. Less control over load order, and the cache layer must know how to fetch from the correct source of truth.
Write-through Every write goes to the cache and to the source of truth in the same operation before returning success. Useful when reads must see the latest written values with minimal lag (e.g. a recently updated report filter result tied to a controlled write path). Higher write latency (two writes). Every writer pays this cost, even if those results are never read again.
Write-back (write-behind) Writes go only to the cache first, and are flushed to the source of truth asynchronously in batches. Only consider in very specific, high-ingest analytics pipelines where eventual persistence is acceptable and you can tolerate loss on cache failure. Great for write burst absorption, but risks data loss and makes strict consistency extremely hard to guarantee. Rarely safe as a default for “source of truth” analytics data.
Write-around Writes go straight to the source of truth, bypassing the cache entirely. The cache is only populated on subsequent reads. Best when newly written data is unlikely to be read immediately (e.g. batch ingested facts that will only be queried hours later by scheduled reports). Avoids polluting cache with one-time writes, but the first read after a write will be a miss and be slower.

Invalidation, Eviction And Protection

  • Invalidation: Treat invalidation as mandatory. Key by the exact query scope (e.g. dataset, partition, time bucket, filter set). Prefer explicit invalidation on known refreshes over blindly relying on TTL alone.

  • Eviction policies: LRU (least recently used) fits bursty dashboard traffic. LFU (least frequently used) can help for “always-on” top reports. FIFO is usually too naive for skewed analytics access. TTL is your baseline to cap staleness regardless of policy.

  • Cache stampede / thundering herd: On expiry of a very popular key, many concurrent jobs hit the source at once. Mitigate with request coalescing, a single-flight lock for that key, or early expiry with jitter to spread re-computes.

  • Hot keys: A single KPI (e.g. “today’s revenue”) can hit one shard relentlessly. Shard-aware keys, read replicas of the cache, or splitting by time window can relieve pressure.

  • Cache penetration: If queries for non-existent partitions/filters are hammered, they create endless misses. Use short-TTL negative caching for known “empty” results and consider a Bloom filter to reject impossible keys before hitting cache or storage.

  • Hit-ratio reasoning: Don’t chase 100% hit ratio. Judge by “cost avoided” (expensive full scans) vs “memory + staleness cost”. A 70-90% hit ratio on your top 10 report keys is often far more valuable than caching every possible permutation.

Big Data & Analytics Specific Guidance

  • Intermediate results: In multi-stage processing, caching partial aggregates (e.g. per-partition rollups) can reuse work across jobs. Only do this if the inputs for that intermediate are immutable for the run window.

  • Query/result caching: Cache final, materialized query results for fixed time ranges (e.g. “last 7 days” at hour granularity). Key must include every dimension that affects the answer.

  • Distributed cache choice: Redis is stronger if you need richer data structures, key expiration control, replication, optional persistence, and more flexible eviction. Memcached is simpler and can be very fast for plain string blobs, but it has no built-in persistence/replication and a more limited feature set. Pick based on your need for durability/semantics, not just raw speed.

  • What not to cache: Avoid caching per-user sensitive exports, raw PII, or unredacted result sets on a shared CDN/reverse-proxy/distributed cache. Never cache one-off, highly-filtered ad-hoc dumps with huge cardinality. Avoid caching entire raw fact tables; cache the answer (aggregation) instead.

  • Consistency: Between cache and the source of truth, aim for eventual consistency in most batch/warehouse analytics. Only push closer to read-your-writes if the business truly cannot tolerate staleness, and be explicit about the TTL/invalidation window.

Trade-Offs & When To Skip Caching

  • Cost: Memory for a distributed cache is not free. Cross-region replication multiplies it.

  • Staleness: Wrong numbers erode trust faster than slightly higher latency for a regulated report.

  • Complexity: Every caching layer adds a failure mode (miss storms, bad keys, partial invalidation).

  • Skip it: For one-off exploratory queries against fresh data, for strictly real-time “exact now” reconciliations, or when the working set is nearly as large as cache and keys have near-zero reuse. Workload, not hype, must justify it.

What Interviewers Look For & Common Mistake

Look for: Defining the problem first, naming cache-aside/write-through/write-back/write-around/read-through, stating eviction + invalidation + stampede protection, calling out hot keys/penetration, and choosing granularity by “what is reused”.

Common mistake: Adding a cache without an invalidation plan. For big data, caching the wrong granularity (entire table vs aggregated key) or dumping per-user sensitive data onto a shared cache is far more dangerous than not caching at all.

Sign in to keep reading

You've opened 2 full answers. Sign in to read all 50 Caching answers, mark what you know and pick up where you left off.

Report a mistake

What's wrong?

All System Design subjects