Design a thread-safe in-memory cache.

This is like building a tiny locked pantry: the food is close, but only one person should restock and rearrange the shelves at a time.

Question: Design a thread-safe in-memory cache.

Answer: A thread-safe cache stores key/value pairs in RAM and makes sure multiple threads can read and write without corrupting data. In Java, the simplest correct design is to guard the cache with a lock and use a map that supports fast lookups, then add an eviction rule such as TTL (time to live) or LRU (least recently used). The main goals are correctness first, then performance, then memory control.

Interview-Ready Answer: I’d design it as an in-memory map protected by synchronization, because thread-safety means every get, put, and evict operation must be atomic and visible to all threads. For a simple and correct version, I’d use a lock plus a LinkedHashMap for LRU order and a TTL field for expiry, so reads stay O(1) on average and eviction is predictable. If traffic grew, I’d move to finer-grained locking or a concurrent structure to reduce contention.

What thread-safe really means

Thread-safe means two things: atomicity (an operation happens as one unit, not half-finished) and visibility (when one thread writes a value, another thread can actually see the latest value). A cache breaks if one thread updates the map while another thread reads a partially updated state, or if eviction runs while reads are still using stale metadata.

One practical design

1. Store each entry as value + expiry time, so the cache can answer both “is it present?” and “is it stale?” quickly.
2. Protect the whole data structure with one lock for the first version. This is the easiest way to make every operation atomic.
3. Use a LinkedHashMap in access-order mode when you want LRU behavior, because the oldest accessed entry becomes the eviction candidate.
4. On get, lock, check the key, remove it if expired, and otherwise return the value. Because access-order is enabled, the hit also refreshes recency.
5. On put, lock, insert the item, then evict expired entries and trim the oldest item if the cache is above capacity.
6. Optionally run a background cleanup task every 10–60 seconds in real systems to remove expired items even when traffic is low.

Why this works under the hood

With one lock, you avoid race conditions, which are bugs caused by timing. No two threads can change the map at the same time, so the internal order stays valid. In Java 8+, this simple design is often good enough for small or medium caches, especially when correctness matters more than extreme throughput.

Comparison of common designs

Performance and edge cases

Average get and put are O(1). Eviction from a linked order structure is also O(1) for one entry, but a cleanup scan is O(n), where n is the number of cached items. Space is O(n). A realistic setup might be 10,000 to 100,000 entries with a TTL of 30 seconds to 5 minutes, but once reads become very hot, a single lock can become a bottleneck. One important gotcha: do not allow null values if you want get to clearly mean “missing” versus “present but empty.” Another gotcha is cache stampede: if many threads miss at once and all call the database, the cache stops helping. The fix is a loader that stores one in-flight computation per key.

When and why to use it

Use this design when you need fast repeated reads, limited memory, and predictable correctness. It is a great fit for reference data, feature flags, rate-limit state, or short-lived computed results. If you need cross-service sharing, persistence, or millions of ops per second, a JVM-local cache is usually not enough on its own.

Real-World Example: Imagine a checkout service in an e-commerce app. It caches product prices and tax rules so every page load does not hit the database. A thread-safe cache matters because dozens of requests may ask for the same price at once, and one background thread may refresh expired data while user threads are reading.

If the cache is not synchronized, you can get symptoms like random wrong totals, duplicate loads, or rare ConcurrentModificationException failures. In logs, you might see one thread writing a new price version while another thread still serves the old one. From the user’s view, the cart total flickers or changes at checkout. If eviction is missing, heap usage grows, garbage collection pauses get longer, and the service slowly becomes unstable under load.

Follow-up & Tricky Questions:

• How would you prevent a cache stampede? Store one in-flight load per key, usually with FutureTask or CompletableFuture, so 100 threads do not all hit the database for the same missing item.
• How would you make it scale better? Use finer-grained locking, striping, or a concurrent map plus a separate eviction structure. That lowers contention when many threads read different keys.
• How do you choose between TTL and LRU? TTL removes old data by age, while LRU removes cold data by usage. Many production caches use both: TTL for freshness, LRU for memory safety.
• How would you test it? Run a multithreaded stress test that hammers get, put, and eviction together, then assert that no invalid states appear and that expired entries disappear.
• What metrics would you expose? Hit rate, miss rate, eviction count, load time, and current size. Those numbers tell you whether the cache is actually helping.
• Is ConcurrentHashMap enough by itself? No. It helps with safe concurrent access, but it does not automatically give you LRU, TTL cleanup, or atomic multi-step operations across structures.
• Can the cache store null values? Better not. If null is allowed, it becomes ambiguous whether the key is missing or the cached value is actually null.
• Does size() stay exact under concurrency? In this locked design, yes. In a fully concurrent design, size can be approximate or more expensive to compute.

Tricky / gotcha questions:

• Why not just synchronize a HashMap and call it done? You can for a first version, but you must still think about eviction, expiry, and stampede control. A synchronized map without those rules is thread-safe but not a complete cache.
• Why is a get sometimes a write? If the cache uses LRU order, a read changes recency metadata. That means a seemingly harmless get may need the same lock as a write.
• What happens if cleanup runs while a thread reads? With the lock in this design, they cannot overlap and corrupt state. Without proper coordination, one thread could remove an entry while another thread is using stale metadata.

Common Mistakes

• Using a plain HashMap with no lock. Fix: guard all shared state with synchronization or use a correct concurrent design.
• Assuming ConcurrentHashMap solves everything. Fix: it handles concurrent access, but eviction order and atomic load logic still need design.
• Allowing null values. Fix: reject null so a miss is not confused with a stored null.
• Ignoring eviction. Fix: add TTL, LRU, or both; otherwise the cache can grow until the JVM hurts.

Memory Hook

Think of the cache as a locked pantry with a lazy Susan: the lock keeps people from bumping into each other, the lazy Susan keeps the newest-used item near the front, and expired food gets thrown away before it smells up the room.

Cheat Sheet

• Thread-safe means atomic and visible.
• Simplest correct design: map + lock.
• Use TTL for freshness and LRU for space control.
• Average get/put should be O(1).
• Cleanup scans are O(n), so do them periodically, not constantly.
• Do not cache null unless you have a very explicit policy.

Practice Tasks

• Start with a synchronized HashMap cache that supports put and get.
• Add TTL expiration and a periodic cleanup method.
• Upgrade it to LRU eviction and write a small multithreaded stress test.