ehsan.blog
~/blog/tanstack-pacer-rate-limiter-default-window-lets-bursts-through — zsh
cat tanstack-pacer-rate-limiter-default-window-lets-bursts-through.md

I read TanStack Pacer's rate limiter source and found its default window lets nearly double the limit through

·10 min read

I was wiring TanStack Pacer into a logging library to stop it hammering an endpoint. Before trusting it, I did what I usually do: I read the source, then I wrote a small script to check that the source did what I thought it did.

The source was clean. The script surprised me.

With a limit of 5 calls per 300 milliseconds, I got 9 calls through in a 30 millisecond span.

That is not a bug in Pacer. It is the default setting doing exactly what it is documented to do. But if you skim the docs and ship it, you will get a rate limiter that is roughly half as strict as you think it is — and you will never see it in a normal test, because it only shows up at one specific moment.

Here is the whole thing, measured.

First, what a rate limiter actually has to remember

Say you want “at most 5 calls per 300ms.” To enforce that, something has to remember what already happened.

The simple way most people write it is a counter and a timer:

ts
let count = 0
setInterval(() => { count = 0 }, 300)   // wipe the counter every 300ms

function tryCall() {
  if (count < 5) {
    count++
    return true   // allowed
  }
  return false    // blocked
}

Five slots. Every 300ms the timer wipes the counter and you get five fresh slots.

This is easy to read and it is wrong in a specific way. The counter has no memory of when those five calls happened. It only knows how many happened since the last wipe. So five calls at the very end of one period and five at the very start of the next are ten calls in a blink, and the counter is perfectly happy — from its point of view, that was “5, then reset, then 5.”

What Pacer stores instead

Pacer does not keep a counter. It keeps the actual clock time of every call it allowed:

ts
executionTimes: Array<number>   // [1757500000123, 1757500000145, ...]

Then on every call it throws away the timestamps that no longer count, and looks at how many are left:

ts
// from TanStack/pacer, packages/pacer/src/rate-limiter.ts
maybeExecute = (...args) => {
  this.#cleanupOldExecutions()
  const relevantExecutionTimes = this.#getExecutionTimesInWindow()

  if (relevantExecutionTimes.length < this.#getLimit()) {
    this.#execute(...args)
    return true
  }

  this.#setState({ rejectionCount: this.store.state.rejectionCount + 1 })
  this.options.onReject?.(this)
  return false
}

Real timestamps instead of a counter means you can always answer “how many calls happened in the last 300ms” exactly. Good.

But storing timestamps is only half the job. The other half is deciding which timestamps still count. And that is where the surprise lives.

Two ways to decide which timestamps count

Pacer gives you two, through one option: windowType.

ts
#getExecutionTimesInWindow = (): Array<number> => {
  if (this.options.windowType === 'sliding') {
    // keep anything from the last `window` milliseconds
    return this.store.state.executionTimes.filter(
      (time) => time > Date.now() - this.#getWindow(),
    )
  } else {
    // 'fixed': the window starts at the OLDEST call we still have
    if (this.store.state.executionTimes.length === 0) return []

    const windowStart = Math.min(...this.store.state.executionTimes)
    const windowEnd = windowStart + this.#getWindow()

    if (Date.now() > windowEnd) return []   // whole window expired -> wipe everything

    return this.store.state.executionTimes.filter(
      (time) => time >= windowStart && time <= windowEnd,
    )
  }
}

Read that else branch slowly, because it is the important one.

sliding asks a rolling question: what happened in the last 300ms, counting backwards from right now? Every call ages out on its own, 300ms after it happened.

fixed does something different. It finds your oldest remembered call and draws a 300ms box starting there. Everything inside the box counts. And the moment now passes the end of that box, it returns [] — every timestamp is forgotten at once, all five slots come back in the same instant.

That difference is the whole post.

Which one do you get by default?

ts
const rl = new RateLimiter(fn, { limit: 3, window: 1000 })
console.log(rl.options.windowType)
plaintext
fixed

fixed. Confirmed both in the source (packages/pacer/src/rate-limiter.ts) and in the published package I installed (@tanstack/pacer@0.22.0).

So unless you typed windowType: 'sliding' yourself, you are using the box-that-empties-all-at-once version.

The measurement

Now the test. I gave it a limit of 5 per 300ms, and arranged calls so they straddle the end of that box:

ts
import { RateLimiter } from '@tanstack/pacer'

const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms))

async function run(windowType: 'fixed' | 'sliding') {
  const stamps: number[] = []
  const start = Date.now()

  const rl = new RateLimiter(() => stamps.push(Date.now() - start), {
    limit: 5,
    window: 300,
    windowType,
  })

  rl.maybeExecute()                        // t=0    one call opens the box
  await sleep(285)
  for (let i = 0; i < 4; i++) rl.maybeExecute()   // t=285  four more -> 5 of 5 used
  await sleep(25)                          // t=310  box (t=0 to t=300) has expired

  let allowed = 0
  for (let i = 0; i < 5; i++) if (rl.maybeExecute()) allowed++

  const recent = stamps.filter((t) => t >= 280)
  console.log(
    `${windowType}: allowed at t=310 -> ${allowed} | calls between t=280 and t=310: ${recent.length}`,
  )
}

await run('fixed')
await run('sliding')

Output:

plaintext
fixed:   allowed at t=310 -> 5 | calls between t=280 and t=310: 9
sliding: allowed at t=310 -> 1 | calls between t=280 and t=310: 5

Nine calls in a 30 millisecond span, under a “5 per 300ms” limit. Close to double.

Walk through why. The one call at t=0 opened the box, so the box runs from t=0 to t=300. The four calls at t=285 filled it. At t=310 the box has expired, so getExecutionTimesInWindow returns [] — all five slots free at once — and five more go straight through.

Four calls at t=285 plus five at t=310 is nine calls inside a 30ms stretch.

Switch to sliding and only one gets through, because at t=310 the four calls from t=285 are still only 25ms old and very much still counted. Exactly one slot has freed up: the one from t=0.

Is fixed broken, then?

No. It is honest about what it promises, which is “at most 5 per window,” not “at most 5 in any 300ms stretch.” Those are two different promises and people hear the second one when they read the first.

There is also a real nuance in Pacer’s fixed that makes it better than the naive counter: the box is anchored to your first call, not to a global clock ticking in the background. No setInterval running whether you use it or not, and no drift if the event loop is busy. I checked:

ts
const rl = new RateLimiter(() => {}, { limit: 2, window: 400 })

await sleep(250)     // sit idle -- no window is running yet
rl.maybeExecute()    // the 400ms box opens HERE, at this call
rl.maybeExecute()    // 2 of 2 used

await sleep(300)
console.log('blocked 300ms after first call:', !rl.maybeExecute())
await sleep(150)
console.log('allowed ~450ms after first call:', rl.maybeExecute())
plaintext
blocked 300ms after first call: true
allowed ~450ms after first call: true

The clock starts when you start. That is genuinely nicer than a background timer. It just does not remove the all-slots-free-at-once moment at the end.

So which should you pick

Ask what the limit is protecting.

Use sliding when something downstream will actually break if it gets hit too fast in a short burst — an API with its own limit that will 429 you, a database, a logging endpoint you are paying per-event for. Bursts are the exact thing you are trying to prevent, so pay for a real rolling window.

ts
new RateLimiter(sendLog, {
  limit: 5,
  window: 300,
  windowType: 'sliding',   // opt in, it is not the default
})

fixed is fine when you are roughly pacing something and a short burst costs nothing — throttling a UI callback, keeping an expensive recompute from running constantly. The looser guarantee is not doing any harm there.

What costs you is not picking. Defaults are a choice someone else made for a use case that might not be yours.

The gotcha that cost me twenty minutes

This one is unrelated to windows and worth knowing, because it fails silently.

If the limiter is disabled, maybeExecute returns true — but your function never runs:

ts
let calls = 0
const rl = new RateLimiter(() => calls++, {
  limit: 5,
  window: 1000,
  enabled: false,
})

const result = rl.maybeExecute()
console.log(`returned ${result}, function ran ${calls} times`)
plaintext
returned true, function ran 0 times

The reason is in the source: the enabled check lives inside the private #execute, which runs after maybeExecute has already decided to return true:

ts
#execute = (...args) => {
  if (!this.#getEnabled()) return   // bails here, but the caller already got `true`
  ...
}

So true from maybeExecute means “not rate limited.” It does not mean “your function ran.” If you branch on that return value — showing a toast, counting successes, deciding whether to retry — and you ever toggle enabled off behind a feature flag, you will be reporting success for work that never happened.

Why rejections are worth listening to

One design detail I did like. A blocked call is not just a quiet false. Pacer counts rejections and gives you a callback:

ts
let rejects = 0
const rl = new RateLimiter(() => {}, {
  limit: 1,
  window: 1000,
  onReject: () => rejects++,
})

for (let i = 0; i < 4; i++) rl.maybeExecute()
console.log(
  `onReject fired ${rejects}x, rejectionCount=${rl.store.state.rejectionCount}, executionCount=${rl.store.state.executionCount}`,
)
plaintext
onReject fired 3x, rejectionCount=3, executionCount=1

And inside onReject you can ask how long the caller has to wait:

ts
const rl = new RateLimiter(() => {}, { limit: 2, window: 1000 })
rl.maybeExecute()
rl.maybeExecute()
await sleep(400)

console.log(
  `rejected=${!rl.maybeExecute()}, remaining=${rl.getRemainingInWindow()}, msUntilNextWindow=${rl.getMsUntilNextWindow()}`,
)
plaintext
rejected=true, remaining=0, msUntilNextWindow=598

598ms (it lands within a millisecond or two of that on each run), which is right: 1000ms minus the ~400ms already elapsed. That number is the difference between a request silently vanishing and a UI that can say “try again in a moment” and mean it.

Rate limit, throttle, debounce — told apart by what they store

Reading all three classes in one sitting made the distinction concrete in a way the usual one-line explanations never did. Each one keeps different state, and the state tells you what it is for:

  • RateLimiter keeps an array of timestamps — “how many calls in this window?” It is the only one that can answer a question about a count.
  • Throttler keeps one number, lastExecutionTime — “has enough time passed since the last one?” One moment is all it needs, so the check is a subtraction instead of filtering an array.
  • Debouncer keeps a pending timeout handle, and clears and resets it on every call — “has it been quiet long enough?” That reset is why a nonstop stream of calls can hold a debounced function off forever, while a throttled one keeps firing on schedule no matter how continuous the input.

Different state, different question. Once you see the fields, you stop mixing them up.

What I took away

I nearly shipped this. The source looked right, the API read right, and a normal test — fire ten calls in a loop, check that five got through — passes on both window types. The gap only opens at one moment: the instant a fixed window expires, with calls clustered right before it.

That is the argument for the script. Reading the source told me how it worked. Running it told me what it does — and those turned out to be 9 calls versus the 5 I had in my head.

Two lines, before you trust any limiter you did not write:

ts
console.log(rl.options.windowType)   // know which promise you are getting

and one timing test that straddles the end of a window.

Everything above was measured against @tanstack/pacer@0.22.0 on Bun. If you run it and get different numbers, the numbers win — go read getExecutionTimesInWindow and find out why.

ls ./related
cat ./comments

Comments are not configured yet. Enable GitHub Discussions and paste the giscus repo-id / category-id into src/consts.ts.