Service Discovery

In a static deployment you hard-code where things live. The database is at db.internal:5432, the pricing service is behind pricing.svc:8080, and those addresses are baked into config that changes about as often as the servers do — which is to say, rarely. That world is comfortable and it is gone. In a modern deployment, instances are cattle: they scale out under load and back in when it passes, they get rescheduled onto different hosts by an orchestrator, they roll one-by-one during a deploy, and they die and get replaced without ceremony. The address of the pricing service is not a fact you can write down, because there is no the pricing service — there are seven of them right now, on ports and hosts that will be different in an hour.

Service Discovery is the pattern that resolves a stable name — "pricing" — into the current set of concrete network locations that answer to it. It is the phone book for a system whose subscribers move house constantly, and it has exactly one job: given a service name, return a healthy instance to call.

The problem

Consider a checkout service that needs to call pricing. In a fixed-address world it opens a connection to pricing.svc:8080 and moves on. Now put that system on an orchestrator that autoscales pricing between three and twenty instances and reschedules them across a fleet of hosts. The concrete questions checkout must answer on every call become genuinely hard:

1. Which instances exist right now? The set changed since the last request. Two instances scaled in a minute ago; one was rescheduled to a new host thirty seconds ago and has a new address.
2. Which of them are healthy? One instance is up but wedged — its process is running, it accepts TCP connections, but its database pool is exhausted and every request hangs. Sending traffic there is worse than not calling at all.
3. Which one should this call go to? Even among the healthy instances, you want to spread load — round-robin, least-connections, or locality-aware so you prefer an instance in the same zone.

Hard-coding addresses fails question 1 immediately. A load-balancer with a static backend pool fails question 1 more slowly — someone has to reconfigure it whenever the instance set changes, and in an autoscaling system that is constantly. DNS seems like the answer to question 1, and it partly is, but plain DNS fails question 2 badly: DNS records have TTLs measured in seconds-to-minutes, clients cache them aggressively and ignore TTLs, and a DNS A-record has no notion of "this instance is wedged, route around it." You end up sending traffic to an address that resolved fine but points at a dead process.

The root difficulty is that the set of healthy instances is a fast-moving, authoritative fact that must be shared across the whole system, and every caller needs a fresh-enough view of it to route correctly. That shared, fast-moving fact needs an owner. The owner is the service registry.

When to use it

Reach for explicit service discovery when the following hold:

• Instance locations are dynamic. You autoscale, you use an orchestrator that reschedules containers, or you do rolling deploys that continuously cycle instances. If the address of a service genuinely changes during normal operation, you need discovery. This is the defining condition.
• You have more than a handful of services calling each other. The cost of running and operating a registry is fixed overhead; it pays off when many services need to find many other services and hand-maintained config becomes a bottleneck and a source of stale-address incidents.
• Health, not just existence, must drive routing. When "the instance is registered" and "the instance can actually serve" can diverge — and in any real system they can — you need discovery coupled to health checking so callers route only to instances that pass a liveness and readiness bar.
• You want load-balancing decisions close to the caller. Client-side discovery lets each caller balance across the live instance set with full knowledge of the pool, which enables smarter policies (least-connections, zone-affinity) than a dumb external balancer.

The archetypal fit is a microservices topology on Kubernetes, Nomad, or a service mesh, where the platform is running discovery for you whether you named the pattern or not.

When NOT to use it

Discovery is infrastructure, and standing up a registry you do not need is a liability, not a feature. Skip it when:

• Your addresses are stable. A monolith talking to one managed database and one managed cache does not need a registry — the addresses are fixed, environment config resolves them, and adding Consul buys you a new distributed system to operate for no benefit.
• The platform already does it and you would be duplicating. On Kubernetes, pricing.namespace.svc.cluster.local already resolves through kube-proxy / the cluster's service abstraction to a healthy pod, with readiness gates and endpoint churn handled for you. Running a second, application-level registry on top usually adds a source of disagreement between two systems that both think they own routing. Use the platform's discovery; do not reinvent it beside itself.
• You have very few services and change them rarely. Three services that deploy monthly can live with load-balancer config or DNS. The operational cost of a highly-available registry — it must be more available than everything that depends on it — is only justified by real churn.
• You cannot make the registry more reliable than its clients. A registry is a shared dependency on the critical path of every inter-service call. If it goes down and clients have no cached fallback, it becomes the single point of failure for the entire system. If you cannot run it as a quorum with client-side caching, you have moved the fragility rather than removed it.

A useful test: if the honest answer to "where is service X right now?" is a set that changes on its own, you need discovery. If it is a constant, you do not.

Architecture

A registry sits at the center. Instances register themselves on startup and renew a lease with periodic heartbeats; when heartbeats stop, the registry evicts them. Callers resolve a service name to the current healthy instance set. The two structural variants differ in who does the resolving and balancing.

In client-side discovery, the caller queries the registry (through a cached local client), gets the instance list, and picks one itself. In server-side discovery, the caller talks to a fixed load balancer or gateway address, and that component queries the registry and forwards the call — the caller never sees the registry at all.

flowchart TD
    subgraph Registry
      R[(Service Registry)]
    end
    P1[pricing #1] -- register + heartbeat --> R
    P2[pricing #2] -- register + heartbeat --> R
    P3[pricing #3] -- register + heartbeat --> R
    C[Checkout caller] -- 1: resolve pricing --> R
    R -- 2: healthy instances --> C
    C -- 3: call chosen instance --> P2

The lifecycle of a single instance is a small state machine, and health checking is what drives the transitions that matter.

stateDiagram-v2
    [*] --> Registered: startup registers
    Registered --> Healthy: passes readiness probe
    Healthy --> Healthy: heartbeat renews lease
    Healthy --> Suspect: missed heartbeat / failed probe
    Suspect --> Healthy: probe recovers
    Suspect --> Evicted: TTL expires
    Evicted --> [*]
    Healthy --> Evicted: graceful deregister on shutdown

The essential property is that only instances in the Healthy state are ever returned to a caller. Registration proves an instance exists; health checking proves it can serve. Conflating the two is the classic discovery bug — you route to a registered-but-wedged instance and the caller hangs exactly as it would have without discovery.

Code walkthrough

Here is a correct, dependency-free client-side resolver in TypeScript. It caches the instance list to keep the hot path off the registry, balances with round-robin, and — crucially — falls back to a stale cache when the registry is unreachable, so the registry is not a hard single point of failure. Start with the domain model.

interface ServiceInstance {
  readonly id: string;
  readonly address: string; // host:port
}

interface Registry {
  healthyInstances(service: string): Promise<ServiceInstance[]>;
}

class NoHealthyInstanceError extends Error {
  constructor(service: string) {
    super(`no healthy instance for "${service}"`);
    this.name = "NoHealthyInstanceError";
  }
}

Registry is a port — an interface the resolver depends on, satisfied by a Consul/etcd/Kubernetes adapter or an in-memory fake in tests. healthyInstances returns only instances that pass health checks; the health bar lives on the registry side, so the resolver never has to route around a wedged box itself. NoHealthyInstanceError is distinct so callers can tell "nothing is up" apart from a downstream failure.

Next, the resolver. The cache is the whole point: resolving from memory is sub-millisecond, and we only pay a registry round-trip when the cache is cold or expired.

interface CacheEntry {
  instances: ServiceInstance[];
  expiresAt: number; // epoch ms
  cursor: number;    // round-robin position
}

class ServiceResolver {
  private readonly cache = new Map<string, CacheEntry>();

  constructor(
    private readonly registry: Registry,
    private readonly ttlMs = 10_000,
  ) {}

  async resolve(service: string): Promise<ServiceInstance> {
    const entry = await this.freshEntry(service);
    if (entry.instances.length === 0) throw new NoHealthyInstanceError(service);

    const instance = entry.instances[entry.cursor % entry.instances.length];
    entry.cursor++;
    return instance;
  }

resolve is the hot path callers hit on every request. It reads a fresh-enough cache entry, then advances a round-robin cursor across the healthy set. No network call happens here unless the cache needs refreshing, which is what keeps discovery off the critical-path latency budget.

The refresh logic is where the resilience lives. On a registry error we do not propagate the failure if we still hold a usable list — a brief registry outage should degrade freshness, not availability.

  private async freshEntry(service: string): Promise<CacheEntry> {
    const cached = this.cache.get(service);
    if (cached && Date.now() < cached.expiresAt) return cached;

    try {
      const instances = await this.registry.healthyInstances(service);
      const entry: CacheEntry = { instances, expiresAt: Date.now() + this.ttlMs, cursor: cached?.cursor ?? 0 };
      this.cache.set(service, entry);
      return entry;
    } catch (err) {
      // Registry unreachable: serve the last known instance set rather than
      // making the registry a hard dependency of every call. Stale beats down.
      if (cached) return cached;
      throw err;
    }
  }
}

The catch block is the single most important line in the file. A naive resolver treats a registry timeout as a routing failure and takes down every caller the instant the registry hiccups — turning the thing meant to improve availability into the system's most dangerous single point of failure. Serving a slightly-stale instance list is almost always the better trade: an address that was healthy ten seconds ago is very likely healthy now, and if it is not, that is precisely what a circuit breaker on the call is for. Usage is one line, and it composes with fail-fast on the instance you get back:

const resolver = new ServiceResolver(consulRegistry);

const instance = await resolver.resolve("pricing"); // <1 ms cached
const price = await breaker.execute(() => httpGet(`http://${instance.address}/price/${sku}`));

Two refinements matter in production. First, the instance you resolve can still be dead — the registry's health view lags reality by up to a heartbeat interval — so pair discovery with a per-call timeout and a circuit breaker, and on a connection failure retry resolve to get a different instance rather than hammering the dead one. Second, prefer watches over polling: registries like Consul and etcd can push instance-set changes to the client (a blocking query or a watch stream) so the cache updates on real events instead of on a fixed TTL, cutting both staleness and registry load. The TTL fallback shown here is the floor, not the ceiling.

Performance characteristics

The metrics in the panel above are indicative — sensible defaults and order-of-magnitude figures to build intuition, not measurements from a specific benchmark.

Registry lookup vs. failure-detection lag. These two numbers are in tension and they are the numbers that matter. A cached lookup is sub-millisecond — it never leaves the process — which is why client-side caching is non-negotiable on the hot path. Failure-detection lag is the opposite concern: the gap between an instance actually dying and the registry evicting it, which is the heartbeat TTL (or the probe interval times the failure threshold). Shorten it and you route around dead instances faster but pay more heartbeat/probe traffic and risk evicting a healthy-but-briefly-slow instance; lengthen it and the registry is calmer but callers keep getting handed corpses for longer. Three-to-thirty seconds is the usual band; tune it against how fast your instances actually fail and how expensive a wrongly-routed call is.

Heartbeat cost and the staleness trade. Every registered instance renews its lease on an interval, so the registry absorbs a steady trickle of heartbeat traffic proportional to fleet size divided by interval. A thousand instances on a thirty-second lease is about thirty-three renewals per second — trivial. Drop the interval to one second chasing faster failure detection and it is a thousand per second, which starts to matter at the registry. This is the same staleness-versus-load dial as the TTL, viewed from the write side.

Registry availability is the real design constraint. The registry sits on the critical path of every inter-service call, so it must be more available than the sum of everything that depends on it. That means running it as a quorum (three-to-five replicas for a consistent store like Consul or etcd, tolerating one or two failures) and giving every client a stale-serving cache so a registry blip degrades freshness rather than causing an outage. A discovery layer without client-side fallback has not removed a single point of failure — it has built a new one and put it on every request.

Live demo

An interactive discovery playground is coming soon. You will be able to spin instances up and down, kill one mid-flight and watch the resolver route around it after the health TTL expires, toggle the registry offline to see stale-cache fallback keep calls flowing, and compare client-side round-robin against a server-side balancer. Check back shortly.

Related patterns

API Gateway is the most common home for server-side discovery. The gateway holds a fixed, well-known address that clients target, and it queries the registry to route each request onto a live backend instance — so the caller never learns the registry exists. This is the natural split: external and cross-cutting concerns (auth, rate limiting, routing) centralize at the gateway, and the gateway leans on discovery to know where the backends are. See the API Gateway pattern for how routing composes with the registry.

Circuit Breaker is the essential safety net beneath discovery, and the two are genuinely complementary rather than overlapping. Discovery answers "which instances should be healthy" from the registry's lagging view; the breaker handles the residual reality that a just-resolved instance is actually dead right now. Discovery picks the target, the breaker fails fast when that target lies about being alive, and a retry against resolve gets you a different instance. Neither replaces the other: without the breaker, discovery happily routes you to a corpse for a full health-TTL; without discovery, the breaker protects you from one fixed address but cannot find you a better one.

A note on the ecosystem. The pattern was popularized in the JVM world by Netflix Eureka (client-side discovery) paired with Ribbon for load-balancing around 2014, and by HashiCorp Consul and CoreOS etcd as general-purpose registries with health checking built in. The center of gravity has since moved down into the platform: Kubernetes bakes discovery into the cluster (Services, EndpointSlices, and DNS), and service meshes like Istio and Linkerd push resolution and balancing into a per-pod sidecar so application code calls a stable name and the mesh does discovery, health-aware routing, and breaking transparently. The pattern is more pervasive than ever; you increasingly consume it from the infrastructure rather than implement it in your service.