The Backpressure-First Principle: Accept Only What You Can Finish

By Sergey Nosov

16 October 2026

At 2:25 a.m. Pacific time on 20 October 2025, AWS restored the DNS records for Amazon DynamoDB in its us-east-1 region. For more than two and a half hours, a race condition in the service’s DNS automation had left its regional endpoint with an empty DNS record, and systems that used that endpoint could not connect. By 2:40 a.m., as cached records expired, DynamoDB was back. Amazon EC2 did not come back with it: new instance launches kept failing, although instances already running stayed healthy.

AWS calls the physical servers that host EC2 instances droplets, and a system named DropletWorkflow Manager (DWFM) manages them. DWFM keeps a lease on each droplet, and its regular checks depend on DynamoDB, so leases had been expiring throughout the outage. When DynamoDB returned, DWFM began re-establishing them across the fleet. AWS’s post-event summary describes what happened next:

Due to the large number of droplets, efforts to establish new droplet leases took long enough that the work could not be completed before they timed out. Additional work was queued to reattempt establishing the droplet lease. At this point, DWFM had entered a state of congestive collapse and was unable to make forward progress in recovering droplet leases.

The race condition was the trigger. DWFM’s collapse had the shape of an overload: work that could not finish before its timeout, retried through a queue, until DWFM could make no forward progress. The Backpressure-First Principle exists to keep that shape from forming, and it fits in one sentence: design software so that it never accepts more work than it can safely complete.

The principle changes the first question about capacity from “How fast can we go?” to “How much can we safely handle?” Many systems default to accepting everything and sorting it out later; Backpressure-First reverses that default. Protect stability first, then maximize throughput, and plan for overload as a normal operating condition, not a surprise.

My Software Development Principles series gives the principle its own entry, next to one on load shedding. I checked the code below on Node.js 24, and a load test pits it against a server that accepts everything. After the five controls and the load test come load shedding, clients that ignore the signal, retries, and what to measure.

What Backpressure Means

Think of a dam. It does not let all the water rush downstream at once, which would destroy everything below it. It holds the water back and releases it at a rate the river can safely carry, and when the reservoir is full, a spillway lets the excess go in a controlled way. Backpressure does the same for work: it controls the rate at which work flows through a system, so that every component downstream can handle what reaches it.

The word has two uses. The Node.js guide “Backpressuring in Streams” uses it for the problem: “a buildup of data behind a buffer during data transfer.” The Reactive Manifesto, whose second version appeared on 16 September 2014, uses it for the remedy: a component under stress “should communicate the fact that it is under stress to upstream components and so get them to reduce the load.” This article means the remedy: the signal that slows the producer down.

Networks have worked this way for decades. Every TCP segment carries a window, which RFC 9293, the 2022 specification that replaced RFC 793 of September 1981, defines as “the range of sequence numbers the sender of the window (the data receiver) is currently prepared to accept.” As the receiver’s buffer fills, it advertises a smaller window, and the sender sends less.

Node.js streams carry the same signal. write() returns false once the stream’s buffer reaches its high-water mark, and the stream emits 'drain' when it can take more. This export honors the signal:

// export-rows.js
import { createWriteStream } from "node:fs";
import { once } from "node:events";

const out = createWriteStream("rows.csv");
let peak = 0;
for (let i = 0; i < 1_000_000; i++) {
  const ok = out.write(`${i},row ${i}\n`);
  peak = Math.max(peak, out.writableLength);
  if (!ok) await once(out, "drain");
}
out.end();
await once(out, "finish");
console.log(`peak buffered: ${(peak / 1024).toFixed(0)} KiB`);

With the wait in place, the export’s buffer peaked at 16 KiB, the default high-water mark on Windows, where I ran it; elsewhere, Node.js defaults to 64 KiB. Without the await, the same loop held the whole file, 17,361 KiB, in memory at once, because a loop that never waits gives the stream no chance to drain.

The Node.js documentation says where that path ends: “Node.js will buffer all written chunks until maximum memory usage occurs, at which point it will abort unconditionally.” A socket is worse: a peer that stops reading can keep the buffer from ever draining, and the documentation warns that writing to such a socket “may lead to a remotely exploitable vulnerability.”

Most systems have some backpressure, designed or not. Fred Hebert, in his 2014 essay on why queues do not fix overload, put it this way: “Usually the back-pressure in the system is implicit: ’tis slow.” Slowness holds back only callers that wait for answers before sending more, and it never says why. Designed backpressure replaces that slowness with an explicit limit and an explicit answer.

Why an Unbounded Queue Hides Overload

A queue is the natural answer to a burst, and it works because bursts end. Trouble starts when work arrives faster than it can be served for longer than a burst lasts. Suppose 500 requests a second arrive at a service that can finish 330. The line grows by 170 every second. After a minute it holds about 10,000 requests, and the next one to join waits about 30 seconds before any work on it begins, because everything ahead of it must be served first. The wait grows for as long as the overload lasts.

The queue also hides the problem. Requests are accepted, the server reports no errors, and a dashboard that counts them stays green while callers time out and memory fills. Left alone, an overloaded queue usually ends like this: memory bloats, latency grows, threads and connections block, and the failure cascades to everything that depends on the service. That is not overload handling. It is overload hiding. If a queue is unbounded, you have already lost; you are only postponing the moment the overload shows.

Queues slow recovery, too. Amazon’s article “Avoiding Insurmountable Queue Backlogs” warns that “queueing that is meant to improve availability can backfire.” If work keeps arriving at a steady rate through an hour-long outage and after it, clearing the backlog within the next hour means working at twice that rate: two hours of work in one. By the time a backlog drains, much of the work in it may no longer be worth doing.

So every queue gets a bound. Memory limits it, but time sets it. If callers give up after 1 second and the service drains 330 requests a second, requests at the back of a line longer than 330 wait longer than their callers will, and the service may still serve them after the callers have left. Multiply the longest wait in line you can afford by the drain rate, and that is the most the line should ever hold. My article on Little’s Law covers the arithmetic behind it.

The drain rate is an average, though, so bound the age of each item as well as the count. In “Using Load Shedding to Avoid Overload,” David Yanacek writes that Amazon has found it “extremely important to place an upper bound on the amount of time that an incoming request sits on a queue.”

Five Controls at Every Boundary

Backpressure-First applies the same idea wherever work enters or leaves a component: rate limits at ingress, bounded queues and concurrency caps inside, timeouts at egress, and an explicit signal back to the caller whenever a limit says no.

1. Limit What Enters

A rate limit decides how much work each caller may start. The usual algorithm is the token bucket, which Yanacek describes in “Fairness in Multi-Tenant Systems”: each admitted request takes a token out of the bucket, and tokens refill at a fixed rate, up to a maximum that sets how big a burst gets through. The classic alternative, the leaky bucket, queues requests and releases them at a steady rate, smoothing bursts instead of admitting them. This token bucket also reports how long the caller should wait, which the server turns into a Retry-After header:

// token-bucket.js
export class TokenBucket {
  #tokens;
  #updated;

  constructor({ perSecond, burst, now = () => performance.now() }) {
    this.perSecond = perSecond;
    this.burst = burst;
    this.now = now;
    this.#tokens = burst;
    this.#updated = now();
  }

  // Takes a token and returns 0,
  // or the ms until one is due.
  take() {
    const now = this.now();
    const earned = ((now - this.#updated) / 1000) * this.perSecond;
    this.#tokens = Math.min(this.burst, this.#tokens + earned);
    this.#updated = now;
    if (this.#tokens >= 1) {
      this.#tokens -= 1;
      return 0;
    }
    return Math.ceil(((1 - this.#tokens) / this.perSecond) * 1000);
  }
}

The burst size is a policy decision. The same article warns that bursts cut both ways: they absorb natural unevenness in traffic, but too large a burst defeats the rate limit. A per-client limit is also a promise about fairness, not about capacity. A thousand clients that each stay within their limits can still send more than the service can handle, which is why the next controls exist.

2. Bound Every Queue and Cache

Every buffer needs a capacity, and every full buffer needs an answer. When a bounded queue is full, the sender finds out at once and can act on it, not later from a timeout. The rule covers more than the obvious work queues: caches, in-memory maps keyed by user or tenant, retry queues, the wait list inside a connection pool, and the per-client state of a rate limiter, which the server below caps at 10,000 entries.

3. Cap What Runs at Once

A rate limit counts arrivals. A concurrency cap counts work in flight, which is what holds threads, connections, and memory. Little’s Law ties the two together: in a stable system, on average, the work in flight equals the throughput times the time each item stays inside. Ten database connections, each held for 30 milliseconds per request, can serve at most about 330 requests a second (10 ÷ 0.03 s), however many arrive.

The usual tool is a semaphore, and the usual mistake is its wait list: a semaphore with an unbounded wait list is an unbounded queue under another name. This limiter caps both the work in flight and the line in front of it, and it lets a waiter leave when its signal aborts, so a deadline also bounds the time spent in line:

// limiter.js
export class Limiter {
  #active = 0;
  #waiting = [];

  constructor({ maxActive, maxWaiting }) {
    this.maxActive = maxActive;
    this.maxWaiting = maxWaiting;
  }

  get active() {
    return this.#active;
  }

  get waiting() {
    return this.#waiting.length;
  }

  // Resolves to a release function
  // or null if the line is full.
  // Rejects if the signal aborts.
  async acquire(signal) {
    signal?.throwIfAborted();
    if (this.#active < this.maxActive) {
      this.#active++;
      return this.#releaser();
    }
    if (this.#waiting.length >= this.maxWaiting) {
      return null;
    }
    return new Promise((resolve, reject) => {
      const grant = () => {
        signal?.removeEventListener("abort", leave);
        resolve(this.#releaser());
      };
      const leave = () => {
        const i = this.#waiting.indexOf(grant);
        if (i >= 0) this.#waiting.splice(i, 1);
        reject(signal.reason);
      };
      signal?.addEventListener("abort", leave, { once: true });
      this.#waiting.push(grant);
    });
  }

  #releaser() {
    let done = false;
    return () => {
      if (done) return;
      done = true;
      const next = this.#waiting.shift();
      if (next) next();
      else this.#active--;
    };
  }
}

A released slot goes straight to the first waiter, so new arrivals cannot jump the line. Fixed caps are easy to enforce and hard to choose. Netflix’s open-source concurrency-limits library takes another route: it borrows ideas from TCP congestion control to estimate the limit from measured latency.

4. Time Out, and Cancel

A call without a timeout is an invitation for trouble: one slow dependency holds a connection, a thread, or a slot until something else gives way. But a timeout has two halves, and a common JavaScript idiom does only one of them. Promise.race against a timer ends the caller’s wait. It does not end the work. If the code then releases its slot, the work keeps running outside the cap.

This test runs forty tasks of 500 milliseconds each under a cap of four, with a 100-millisecond timeout, first with a race and then with an AbortSignal passed into the work:

// timeout-trap.js
import { setTimeout as sleep } from "node:timers/promises";
import { Limiter } from "./limiter.js";

let running = 0;
let peak = 0;

async function work(signal) {
  running++;
  peak = Math.max(peak, running);
  try {
    await sleep(500, undefined, { signal });
  } finally {
    running--;
  }
}

// Stops the wait, not the work.
function raceTimeout(promise, ms) {
  const timer = sleep(ms).then(() => {
    throw new Error("timeout");
  });
  return Promise.race([promise, timer]);
}

async function run(label, call) {
  const limiter = new Limiter({ maxActive: 4, maxWaiting: 100 });
  peak = 0;
  const tasks = Array.from({ length: 40 }, async () => {
    const release = await limiter.acquire();
    try {
      await call();
    } finally {
      release();
    }
  });
  await Promise.allSettled(tasks);
  console.log(`${label}: peak ${peak} running, ${running} left`);
  while (running > 0) await sleep(10);
}

await run("race", () => raceTimeout(work(), 100));
await run("signal", () => work(AbortSignal.timeout(100)));

It prints:

race: peak 20 running, 16 left
signal: peak 4 running, 0 left

With the race, twenty tasks ran at once under a cap of four, and sixteen were still running after every caller had given up. With the signal, the cap held, and nothing outlived its caller. So pass the signal all the way down. fetch, the timers in node:timers/promises, and many other Node.js APIs accept a signal option. AbortSignal.timeout() creates a signal that fires after a delay, and AbortSignal.any() combines a deadline with a signal that fires when the client goes away, as the server below does.

Deadlines should also travel with the request. Google’s SRE book, Site Reliability Engineering, recommends propagating them throughout the stack, so that a server deep in the call tree stops working on a request whose caller has already given up. Yanacek suggests the client’s side of it: send “timeout hints” so that the server can drop doomed requests cheaply.

5. Tell the Caller

A limit that fails silently is only a quieter collapse. When a control says no, the caller should hear it at once, in a form a program can act on. That is how you teach callers to slow down. In HTTP, two status codes carry the signal, and they mean different things:

Amazon’s fairness article draws the same line: a quota rejection “indicates that the client has exceeded its quota, not that the server is out of capacity,” so rate-limit errors generally get 429.

Either code can carry Retry-After, which RFC 9110 defines as “how long the user agent ought to wait before making a follow-up request,” in seconds or as a date. A body in the application/problem+json format of RFC 9457 (July 2023) lets a program read the details. The SRE book gives the reason for all of it: “Return a specific status when overloaded so that clients and other layers back off and do not retry.”

A Server That Says “Not Yet”

This server puts the five controls together. Each client gets a token bucket of 150 requests a second with a burst of 15, and an empty bucket means 429. Ten simulated database connections do the work, with at most ten requests waiting, and a request that finds the line full, or waits 50 milliseconds without getting a connection, gets 503. Every request has a 1-second deadline and a signal that fires if the client disconnects, and the database call honors both.

To stay short, the sample keys its buckets on an x-client-id header and keeps them in one process. A real service would use a validated API key or account, because a caller can put anything in a header, and would enforce each quota across all its replicas. The sample’s controls also start only once a request reaches the handler, so connections need caps and timeouts of their own, at the proxy or in the HTTP server.

// server.js
import { createServer } from "node:http";
import { setTimeout as sleep } from "node:timers/promises";
import { TokenBucket } from "./token-bucket.js";
import { Limiter } from "./limiter.js";

const limiter = new Limiter({ maxActive: 10, maxWaiting: 10 });
const buckets = new Map();
const MAX_CLIENTS = 10_000;

function bucketFor(client) {
  let bucket = buckets.get(client);
  if (bucket) {
    buckets.delete(client);
  } else {
    bucket = new TokenBucket({ perSecond: 150, burst: 15 });
    if (buckets.size >= MAX_CLIENTS) {
      buckets.delete(buckets.keys().next().value);
    }
  }
  buckets.set(client, bucket);
  return bucket;
}

function refuse(res, status, title, ms) {
  res.writeHead(status, {
    "content-type": "application/problem+json",
    "retry-after": String(Math.max(1, Math.ceil(ms / 1000))),
  });
  res.end(JSON.stringify({ status, title }));
}

// Stands in for a database call
// holding one of ten connections.
async function query(signal) {
  await sleep(30, undefined, { signal });
  return "ok\n";
}

const server = createServer(async (req, res) => {
  const client = req.headers["x-client-id"] ?? "anonymous";
  const waitMs = bucketFor(client).take();
  if (waitMs > 0) {
    return refuse(res, 429, "Too Many Requests", waitMs);
  }

  const gone = new AbortController();
  res.on("close", () => gone.abort());
  const deadline = AbortSignal.any([gone.signal, AbortSignal.timeout(1000)]);

  let release;
  try {
    const maxWait = AbortSignal.timeout(50);
    release = await limiter.acquire(AbortSignal.any([deadline, maxWait]));
  } catch {
    release = null;
  }
  if (!release) {
    return refuse(res, 503, "Service Unavailable", 1000);
  }

  try {
    res.end(await query(deadline));
  } catch {
    if (!res.headersSent) refuse(res, 503, "Service Unavailable", 1000);
  } finally {
    release();
  }
});

const port = Number(process.env.PORT ?? 8080);
server.listen(port, () => console.log(`listening on ${port}`));

Retry-After counts whole seconds, so the server rounds the bucket’s wait, at most 7 milliseconds, up to 1 second. A client that honors it waits longer than it must and gets less than its quota, which errs on the side of protecting the server.

The two limits on the line agree: ten requests in line, drained at about 330 a second, wait about 30 milliseconds, under the 50-millisecond timer. In my runs, the longest wait was 46 milliseconds, and every 503 came from a full line, so the timer is a backstop for work that slows down.

The simulated query never reaches the 1-second deadline. A real one can hang, and then the deadline stops it only if the driver cancels the query in the database instead of just rejecting its promise. Otherwise, the slot goes free while the query runs on, outside the cap. A real query can also fail for reasons a retry will not fix, and those call for a status such as 500, not a 503, which tells the caller that waiting will help.

For comparison, here is the accept-everything version, with the same ten connections, no rate limit, no deadline, and a line with no end:

// naive-server.js
import { createServer } from "node:http";
import { setTimeout as sleep } from "node:timers/promises";
import { Limiter } from "./limiter.js";

// Ten connections, and a line
// with no end.
const pool = new Limiter({ maxActive: 10, maxWaiting: Infinity });

const server = createServer(async (req, res) => {
  const release = await pool.acquire();
  try {
    await sleep(30);
    res.end("ok\n");
  } finally {
    release();
  }
});

const port = Number(process.env.PORT ?? 8080);
server.listen(port, () => console.log(`listening on ${port}`));

The load test starts a server in a child process, offers it 500 requests a second for 5 seconds, about one and a half times what ten connections can serve, and gives up on any request after 1 second. When the rush is over, it sends one more request and times it:

// load-test.js
import { spawn } from "node:child_process";
import { once } from "node:events";
import { setTimeout as sleep } from "node:timers/promises";

const [file, mode = "even"] = process.argv.slice(2);
const port = 20000 + Math.floor(Math.random() * 20000);
const child = spawn(process.execPath, [file], {
  env: { ...process.env, PORT: String(port) },
});
await once(child.stdout, "data");

const RATE = 500; // offered per second
const SECONDS = 5;
const stats = {};
const pending = [];

// "greedy": one client sends 400
// a second, 100 others 1 each.
// Otherwise 100 clients, 5 each.
function clientFor(i) {
  if (mode !== "greedy") return `client-${i % 100}`;
  return i % 5 !== 0 ? "greedy" : `client-${(i / 5) % 100}`;
}

async function send(client, timeoutMs = 1000) {
  const group = client.startsWith("client-") ? "others" : client;
  const s = (stats[group] ??= { sent: 0, ok: [], codes: {} });
  s.sent++;
  const started = performance.now();
  try {
    const res = await fetch(`http://127.0.0.1:${port}/`, {
      headers: { "x-client-id": client },
      signal: AbortSignal.timeout(timeoutMs),
    });
    await res.text();
    if (res.ok) s.ok.push(performance.now() - started);
    else s.codes[res.status] = (s.codes[res.status] ?? 0) + 1;
  } catch {
    s.codes["gave up"] = (s.codes["gave up"] ?? 0) + 1;
  }
}

const start = performance.now();
for (let sent = 0; sent < RATE * SECONDS; ) {
  const elapsed = (performance.now() - start) / 1000;
  const due = Math.min(RATE * SECONDS, Math.floor(elapsed * RATE));
  while (sent < due) pending.push(send(clientFor(sent++)));
  await sleep(1);
}
await Promise.all(pending);

const pct = (xs, p) =>
  xs.sort((a, b) => a - b)[Math.floor(p * (xs.length - 1))]?.toFixed(0);
for (const [group, s] of Object.entries(stats)) {
  console.log(`${group}: ${s.sent} sent`);
  const ms = `p50 ${pct(s.ok, 0.5)} ms, p99 ${pct(s.ok, 0.99)} ms`;
  console.log(`  ok ${s.ok.length}, ${ms}`);
  for (const [code, n] of Object.entries(s.codes)) {
    console.log(`  ${code}: ${n}`);
  }
}

// Then one more, after the rush.
await send("after", 60_000);
console.log(`next request: ${stats.after.ok[0]?.toFixed(0)} ms`);
child.kill();

Here is one run against each server; the numbers vary a little between runs and machines. First, node load-test.js naive-server.js:

others: 2500 sent
  ok 878, p50 514 ms, p99 991 ms
  gave up: 1622
next request: 1792 ms

Then node load-test.js server.js:

others: 2500 sent
  ok 1616, p50 63 ms, p99 77 ms
  503: 884
next request: 31 ms

The accept-everything server answered 878 of the 2,500 requests in time, at a median of about half a second. The other 1,622 callers gave up, and the server kept working for them anyway: a request sent after the test waited 1.8 seconds behind abandoned work.

The backpressure-first server answered 1,616 requests, 99 percent of them within 77 milliseconds. It turned the other 884 away almost at once, each with a Retry-After of 1 second; in separate runs that timed the refusals, 99 percent took 5 milliseconds or less. Yanacek calls the number that matters goodput: “the subset of the throughput that is handled without errors and with low enough latency for the client to make use of the response.” With the same ten connections, the bounded server nearly doubled it, because its short line kept every request it accepted well inside its caller’s patience.

The third run, node load-test.js server.js greedy, adds a greedy client that sends 400 requests a second next to a hundred well-behaved clients that send one a second each:

others: 500 sent
  ok 499, p50 32 ms, p99 48 ms
  503: 1
greedy: 2000 sent
  ok 758, p50 32 ms, p99 54 ms
  429: 1239
  503: 3
next request: 45 ms

The greedy client was refused 1,239 times with 429 but still got its 150 requests a second, and the well-behaved clients got 499 of their 500 requests through. When I raised the bucket rate to 100,000 a second, which in effect removes the per-client limit, the greedy client took about four of every five answered requests, and the well-behaved clients lost about a third of theirs to 503.

Backpressure First, Then Shedding

Backpressure and load shedding do different jobs. Backpressure asks producers to slow down, through queue limits, throttling, and flow control. Load shedding drops work the system cannot safely handle: it rejects requests with 503, defers or drops low-priority jobs, and switches off expensive features. A 503 with Retry-After does both jobs: it drops the request and asks the caller to come back later.

Hebert put the choice plainly in his essay: you “pick between blocking on input (back-pressure), or dropping data on the floor (load-shedding).” Prefer backpressure, because work that waits is delayed, not lost. Shedding is the spillway for when slowing down is not enough.

Whether backpressure is enough depends on whether the producer can wait. A stream writer, a worker that pulls from a message broker, or a batch job can pause until the consumer is ready. Users arriving from the internet do not wait for your queue to drain: they keep arriving whether it is full or not, and many of them retry. External systems, clients that ignore 429, and other traffic you do not control behave the same way. For them, the choice is between explicit shedding and a silent collapse.

Shed gracefully: drop work predictably, degrade non-critical features first, and keep the core path alive. AWS did that during the outage that opened this article. With EC2 launches still impaired, it “throttled Lambda Event Source Mappings and asynchronous invocations to prioritize latency-sensitive synchronous invocations.” Google’s RPC system, as the SRE book’s chapter “Handling Overload” describes it, tags every request with a criticality, and an overloaded task rejects the less critical requests first.

Order matters within a workflow, too: when clients call start() and then end(), the load shedding article advises serving end() requests first, because otherwise clients cannot finish the work they have started.

Shed early, where it is cheap, and log enough to know what you drop: as Yanacek notes, early rejection costs visibility. Rejection is never free, either. The same Google chapter describes a backend that can become overloaded “even though the vast majority of its CPU is spent just rejecting requests.” Its answer is client-side throttling: a client that sees many of its recent requests rejected starts refusing new ones itself, before they reach the network.

When Clients Ignore the Signal

A 429 is a request, not a wall. Some clients will not honor it: a script that retries in a tight loop, an integration written years ago, a scraper. When a client keeps sending after being told to slow down, keep rejecting it. That is not punishment. It is responsible operation, because well-behaved clients should not suffer for the ones that ignore the rules.

A client that ignores the signal disregards the wait that HTTP asks of it and takes capacity that belongs to the clients who comply. It turns a shared service into the commons of Garrett Hardin’s 1968 essay “The Tragedy of the Commons,” where each user keeps the whole gain from taking more and shares the cost with everyone.

For clients that persist, escalate in steps. Start with the normal rate limit, then longer Retry-After values that the server enforces, then a temporary block or a cooling-off period, and, for clear abuse, suspension of the key or the account. GitHub’s REST API documentation states the last step: “Continuing to make requests while you are rate limited may result in the banning of your integration.”

At the abusive end, even a polite 429 can cost too much. RFC 6585 notes that answering each request with 429 consumes resources and that “it may be more appropriate to just drop connections, or take other steps.”

During an overload, keep rejecting until the system has actually recovered. Then lift emergency throttles and blocks gradually, and base the decision on metrics, not the clock. Recovered means queues back to normal depth, work in flight with room under its caps, and connections free. AWS worked this way in the outage that opened this article: its engineers began relaxing the request throttles they had put on EC2 “as API calls and new EC2 instance launch requests stabilized,” and full recovery took almost two and a half more hours.

Isolation keeps the damage local. The SRE book is blunt about the goal: during global overload, “it’s vital that the service only delivers error responses to misbehaving customers, while other customers remain unaffected.” Per-client buckets do that for rates, as the greedy run showed; for capacity, the book describes provisioning for each customer’s negotiated usage, with per-customer quotas to match.

Write all of it into the API contract: the limits, the 429 and 503 responses and their bodies, the retry behavior you expect, and what happens to clients that ignore it. An IETF working group is drafting RateLimit header fields for HTTP, which would let servers advertise their quotas so that clients can stay under them; its latest revision, from May 2026, is still a draft.

Retries Can Undo All of It

Backpressure assumes that a rejected caller backs off. Automatic retries can turn every rejection into more load. The cascading-failures chapter of the SRE book does the multiplication: if the backend, frontend, and JavaScript layers each make four attempts against an overloaded database, “a single user action may create 64 attempts (4^3) on the database.” The book’s answer is to retry only “at the layer immediately above the layer that is rejecting them,” to cap the attempts per request, and to give each process a retry budget, such as sixty retries a minute.

Retries can also keep a system down after its trigger is gone. In a 2021 paper, Nathan Bronson, Abutalib Aghayev, Aleksey Charapko, and Timothy Zhu defined the pattern they call metastable failure, in which “a trigger causes the system to enter a bad state that persists even when the trigger is removed.” They call request retries “one of the most common failure-sustaining mechanisms,” and the way out, they write, “requires a strong corrective push, such as rebooting the system or dramatically reducing the load.”

DWFM’s collapse fits that description. DynamoDB was back by 2:40 a.m., but DWFM recovered only after a corrective push: at 4:14 a.m., engineers throttled incoming work and began restarting selected DWFM hosts. The restarts cleared its queues, and by 5:28 a.m., every droplet had a lease again. A second backlog followed in Network Manager, which propagates network configuration to new instances, and new launches did not work normally again until 10:36 a.m. AWS’s announced fixes include better throttling in its EC2 data propagation systems, to “rate limit incoming work based on the size of the waiting queue.”

On the client side, the rules are short. Retry only what is safe to repeat. Wait at least as long as Retry-After says, and without it, back off exponentially. Add random jitter, so that retries do not arrive together. And stop after a few attempts. My article on idempotency in APIs covers safe retries and jittered backoff.

See It Happening

Backpressure only works if you can see it happening. Without observability you are guessing, and nobody can tell a limit that protects the system from one that turns away work for no reason. Instrument everything that represents flow and load:

In distributed traces, mark each request that was throttled, queued, or shed, with the reason. Alarm when rejections become sustained, and watch for the opposite mistake as well: Yanacek calls a rejection made while capacity was available a false positive. He also warns that shedding set at a CPU target close to the autoscaling target can hide the load from the scaler, which then gets its signal late or never.

Test It Before Production Does

Backpressure is a team discipline as much as a coding technique. Define service level objectives and a capacity envelope, tier traffic so that everyone knows what gets shed first, and keep runbooks and circuit-breaker policies that tell operators which levers to pull. Then load test past saturation, with short bursts and long soak runs, before production traffic finds the weak spots for you. Offer the load on a schedule, as the test above does: a generator that waits for each answer before sending the next is held back by the very slowness it should expose.

The SRE book’s overload chapter sets the bar: “a backend task provisioned to serve a certain traffic rate should continue to serve traffic at that rate without any significant impact on latency, regardless of how much excess traffic is thrown at the task.” That should hold, it adds, up to “somewhere above 2x or even 10x what the task is provisioned to process.”

A Checklist

Backpressure-First means stability before speed. It is not about saying no to work. It is about saying “not yet, safely.” In real conditions, load is never steady. Systems that control their flow survive their busiest day. Systems that do not control it drown in their own success.

Further Reading