Production engineering workshop · 50 MIN
Java · Coalesce concurrent cache loads
One in-flight operation per key prevents duplicate expensive work.
Use ConcurrentHashMap.putIfAbsent to elect one loader for a key. The winning future is published before the operation begins, so concurrent callers can join it. Always remove the same future on completion or executor rejection. Return a derived future so an individual caller cancelling its view does not cancel the shared operation. This is request coalescing, not a durable cache or distributed lock.
First reproduce the normal path. Then force a failure at each boundary and inspect what remains true.
Read the example
import java.util.concurrent.*;
import java.util.function.Supplier;
public final class SingleFlight<K,V> {
private final ConcurrentHashMap<K,CompletableFuture<V>> active = new ConcurrentHashMap<>();
private final Executor executor;
public SingleFlight(Executor executor) { this.executor = executor; }
public CompletableFuture<V> get(K key, Supplier<V> loader) {
var own = new CompletableFuture<V>();
var existing = active.putIfAbsent(key, own);
var shared = existing == null ? own : existing;
if (existing == null) {
try {
executor.execute(() -> {
try { own.complete(loader.get()); }
catch (Throwable error) { own.completeExceptionally(error); }
finally { active.remove(key, own); }
});
} catch (RuntimeException rejected) {
active.remove(key, own);
own.completeExceptionally(rejected);
}
}
return shared.thenApply(value -> value);
}
}Check the expected output
Review the behavior in the stated project environment.
Your challenge
Create a Java project with this SingleFlight class. Use a bounded executor and a counter in the loader. Fire many same-key requests behind a latch, release the loader, and assert one invocation and identical results. Then fail one load and confirm the next request can retry.
Solution cost: Expected constant map operations per call, plus loader cost. time · O(k + c) for active keys and waiting caller futures. space
Common trap
A future retained after failure becomes a permanent failed cache entry. This local map does not coalesce across multiple application instances.
Study the project implementation
import java.util.concurrent.*;
import java.util.function.Supplier;
public final class SingleFlight<K,V> {
private final ConcurrentHashMap<K,CompletableFuture<V>> active = new ConcurrentHashMap<>();
private final Executor executor;
public SingleFlight(Executor executor) { this.executor = executor; }
public CompletableFuture<V> get(K key, Supplier<V> loader) {
var own = new CompletableFuture<V>();
var existing = active.putIfAbsent(key, own);
var shared = existing == null ? own : existing;
if (existing == null) {
try {
executor.execute(() -> {
try { own.complete(loader.get()); }
catch (Throwable error) { own.completeExceptionally(error); }
finally { active.remove(key, own); }
});
} catch (RuntimeException rejected) {
active.remove(key, own);
own.completeExceptionally(rejected);
}
}
return shared.thenApply(value -> value);
}
}Further reading: Official documentation