Bulkhead

A ship's hull is divided into watertight compartments. Breach one and it floods; the ship stays up because the bulkheads stop the water reaching the rest. The naval architects who standardised this were not trying to prevent hull breaches — they were accepting that breaches happen and deciding what a breach would be allowed to cost.

That is the whole idea, and it transfers exactly. Your service will have a slow dependency. The question a bulkhead answers is not "how do we stop that" but "when it happens, does it take down this one feature or the entire process?"

The problem

A service has one thread pool, one connection pool, one concurrency limit. Every request draws from it, whatever it is doing.

Now a single downstream dependency — a recommendation API, say — degrades from 30 ms to 8 seconds. Recommendations are a nice-to-have: the product page renders perfectly well without them, and the code has a tidy fallback that returns an empty list.

The fallback never runs.

Requests calling recommendations occupy their threads for 8 seconds instead of 30 ms. At 100 recommendation calls per second, that is 800 threads' worth of demand against a pool of, say, 200. The pool saturates in a couple of seconds. And a saturated pool does not discriminate: checkout requests, health checks, and login requests now queue behind recommendation calls, because they need a thread from the same pool and there are none.

The load balancer's health check times out. The instance is marked unhealthy and removed. Its traffic shifts to siblings, which are calling the same degraded recommendation API and fall over in turn. A non-critical feature has taken down the entire service, and then the entire fleet.

The failure is not the slow dependency. The failure is that a shared resource pool converts a local problem into a global one. Nothing in the code expressed that recommendations were less important than checkout; at the moment it mattered, the runtime had no way to know.

This is the same family of problem the Circuit Breaker addresses, and the two are constantly confused. A breaker watches a dependency's health and eventually stops calling it. But a breaker needs failures to observe, and it needs a threshold's worth of them. During the seconds or minutes before it trips — and forever, if the dependency is slow but not failing — the calls are still in flight, still holding threads. A bulkhead caps that exposure structurally, from the first request, without needing to detect anything.

When to use it

• One process serves work of genuinely different importance. Checkout and recommendations. Interactive requests and batch exports. Paying customers and free-tier traffic. If everything in the process is equally critical, partitioning buys you less.
• A dependency can be slow without being down. This is the case breakers handle worst and bulkheads handle best. A dependency returning 200s in 8 seconds looks healthy to every failure-counting heuristic while consuming your capacity completely.
• The resource being exhausted is bounded and shared. Threads, database connections, HTTP client connections, in-flight request slots, memory. If saturating it stops unrelated work, it wants compartments.
• You can state what should survive. The design question a bulkhead forces is "when we are out of capacity, what still works?" If you cannot answer that, partitioning arbitrarily will not help.

The strongest fit is a service with several downstream dependencies of unequal criticality — which describes most services in a microservices topology, and quite a few monoliths.

When NOT to use it

• You have one dependency. Partitioning a pool into one compartment is just a pool. The isolation only pays when there is something to isolate from.
• Total capacity is the actual constraint. Bulkheads do not add capacity; they reserve it. Partitioning a saturated system makes every partition smaller and the system slower, because the checkout compartment cannot borrow the idle recommendation threads. If you are simply under-provisioned, provision.
• The work is uniform and equally critical. Splitting a homogeneous workload gives you N ways to be locally saturated while capacity sits idle next door, and worse tail latency than one pool would.
• You cannot size the partitions. This is the real reason bulkheads fail in practice, and it deserves to be said plainly: a badly sized bulkhead is a self-inflicted outage. A compartment sized at 10 for a dependency that needs 40 rejects traffic the system had ample capacity to serve, and it does so quietly, under normal conditions, forever. Isolation is easy; sizing is the engineering.
• A separate process or service would be honest. At some point "isolate this workload" stops meaning a pool and starts meaning a deployment. If a workload needs its own scaling profile and its own failure domain, give it its own process rather than an elaborate partition inside a shared one.

Architecture

Two implementations, and the choice between them matters more than it first appears.

flowchart TD
    A[Incoming requests] --> B{Which dependency?}
    B -->|checkout| C[Compartment: payments<br/>limit 60]
    B -->|catalog| D[Compartment: inventory<br/>limit 40]
    B -->|extras| E[Compartment: recommendations<br/>limit 10]
    C --> F[Payments API]
    D --> G[Inventory API]
    E --> H[Recommendations API]
    E -.->|full → reject fast| I[Empty list fallback]
    style E stroke-dasharray: 4 4

When recommendations degrades, its compartment fills, and the eleventh concurrent call is rejected immediately rather than queued. The fallback runs. Payments and inventory never notice, because their capacity was never available to recommendations in the first place.

Semaphore bulkhead — a counter. The calling thread checks out a permit, makes the call on its own thread, returns the permit. No extra threads, no context switch, negligible memory. The limitation is that the caller's thread is still blocked for the duration of the call, so this protects the downstream-specific resource but not the caller's own thread pool. In an async runtime — Node, or anything built on an event loop — this is the natural and usually the only sensible choice, because there is no thread to hand off to.

Thread-pool bulkhead — a genuinely separate pool per dependency. The calling thread hands work off and waits on a future, so the pool that saturates is the dependency's, not the caller's. Stronger isolation, and it is what makes a per-dependency timeout enforceable from outside the call. It costs a context switch per call and roughly a megabyte of stack per thread, and it makes thread-local context (request ids, tracing spans, security context) something you must propagate explicitly rather than something that just works.

sequenceDiagram
    participant R1 as Checkout req
    participant R2 as Extras req
    participant BP as payments (60)
    participant BR as recommendations (10)
    R2->>BR: acquire (10/10 taken)
    BR--xR2: rejected — fast
    Note over R2: fallback: empty list, 200 OK
    R1->>BP: acquire (12/60)
    BP-->>R1: proceed
    Note over R1: checkout unaffected

The rejection is the feature. A bulkhead that queues instead of rejecting has only moved the saturation somewhere less visible — an unbounded queue is a memory leak with extra steps, and it converts a capacity problem into a latency problem that ends in an OOM kill.

Code walkthrough

A semaphore bulkhead is small enough to read in one sitting, which is worth doing before reaching for a library.

export class Bulkhead {
  private inFlight = 0;

  constructor(
    private readonly name: string,
    private readonly limit: number,
  ) {}

  async run<T>(work: () => Promise<T>): Promise<T> {
    if (this.inFlight >= this.limit) {
      throw new BulkheadFull(this.name, this.limit);
    }

    this.inFlight += 1;
    try {
      return await work();
    } finally {
      this.inFlight -= 1;
    }
  }

  get saturation(): number {
    return this.inFlight / this.limit;
  }
}

Three things in that are load-bearing:

The release is in finally. Miss this and every thrown error permanently leaks a permit. The compartment shrinks with each failure until it reaches zero and rejects everything — and it does so after the incident that caused the errors has been resolved, which makes it maximally confusing to diagnose.

Rejection is immediate, not queued. BulkheadFull is thrown before work() is called. There is no waiting list. A caller that wants bounded waiting should compose that explicitly rather than get it by accident.

saturation is exposed, because a bulkhead you cannot observe cannot be sized, and sizing is the entire difficulty.

At the call site the compartment sits outside the fallback and inside nothing else:

export async function recommendationsFor(sku: Sku, deps: Deps): Promise<Recommendation[]> {
  try {
    return await deps.extrasBulkhead.run(() => deps.recommendations.fetch(sku));
  } catch (error) {
    if (error instanceof BulkheadFull || error instanceof TimeoutError) {
      deps.log.info('recommendations degraded, serving without', { sku, reason: error.name });
      return [];
    }
    throw error;
  }
}

Note that BulkheadFull and a timeout collapse to the same fallback. From the product's point of view they are the same event — we are not showing recommendations this time — and treating them differently produces two code paths that must stay in sync for no benefit.

Layering with the other resilience patterns has one correct order, and getting it wrong is common:

const guarded = () =>
  breaker.run(() =>
    bulkhead.run(() =>
      withRetry(() => withTimeout(client.fetch(sku), 300), retryPolicy),
    ),
  );

Outermost is the circuit breaker: when it is open, nothing else runs at all, and it must not spend a bulkhead permit to discover that. Then the bulkhead, which caps concurrency. Then retry, so that re-attempts are made inside the permit the call already holds — this is the subtle one. Put retry outside the bulkhead and each re-attempt competes for a fresh permit against a compartment that is by now full, so retries fail on saturation rather than on the dependency. Innermost is the per-attempt timeout, which is what bounds how long a permit can be held; without it a hung call holds its permit indefinitely and the compartment never recovers.

Performance characteristics

A semaphore bulkhead is a counter increment on the request path — genuinely free relative to the network call it guards. A thread-pool bulkhead adds a handoff of a few microseconds and about a megabyte of stack per thread, which is why 3-8 compartments is a sensible ceiling rather than one per downstream call site.

The number that decides whether a bulkhead helps or hurts is the limit, and Little's Law gives you the starting point: concurrency = arrival rate × latency. A dependency taking 50 ms at 200 requests per second needs 10 concurrent slots to keep up. Size the compartment there and you reject traffic the moment latency doubles — which is exactly when you want some headroom, not none. Size it at 10× and it will not protect anything until the process is already in trouble. In practice: measure p99 latency and peak arrival rate, compute the concurrency that implies, then set the limit at roughly 2× that for critical paths and close to 1× for the optional ones you are willing to shed first.

Then watch saturation, not rejections. Rejections tell you a compartment is already full; saturation trending upward tells you it is about to be, with enough warning to matter. A compartment that never exceeds 30% saturation is not protecting anything and its capacity would serve the system better elsewhere. A compartment sitting at 90% under normal load is one traffic spike from shedding real traffic, and it is mis-sized in the other direction.

The failure mode nobody plans for is the permit leak described above — the finally bug — because it presents as a gradual, unexplained loss of capacity long after the triggering incident, and it looks nothing like the dependency problem the bulkhead was installed to handle.

Live demo

A runnable bulkhead — a saturating dependency, a compartment shedding it, and the sibling traffic sailing past unaffected — is on the roadmap here. Until then the class above is the whole mechanism; the compartment is genuinely twenty lines, and everything difficult about this pattern is in the limit you give it.

Related patterns

Circuit Breaker composes with the bulkhead and does not replace it, though the two are frequently treated as alternatives. A breaker responds to a dependency that is failing; a bulkhead caps the cost of one that is merely slow, which is the case a breaker is worst at because slow successes never trip a failure threshold. Run the breaker outside the bulkhead so an open circuit costs no permit, and note that the bulkhead is what keeps you alive during the window before the breaker has seen enough failures to trip.

Retry must live inside the compartment, not outside it. Retrying outside means each attempt contends for a new permit against a compartment that is already full, so the retry fails on saturation and tells you nothing about the dependency. Inside, the re-attempts reuse the permit the call already holds, and the compartment's limit doubles as a genuine cap on total retry-amplified concurrency — which is the only place a retry budget can actually be enforced.

Load Balancing is the contrast worth drawing. A load balancer distributes work across instances so no one instance is overloaded; a bulkhead partitions work within one instance so no one workload consumes it. They operate on different axes and neither substitutes for the other: perfect load balancing still lets a slow dependency saturate every instance simultaneously, because every instance is running the same undivided pool.

API Gateway is where a coarser version of this often lives. Per-route and per-tenant concurrency limits at the edge are bulkheads applied to ingress rather than egress, and they are the right place to isolate tenants from each other. They do not help with the problem here — one caller's slow downstream dependency — because the gateway cannot see which internal dependency a request will end up waiting on.