Rate-Limit a Multi-Tenant API Without Punishing Good Tenants

For: A backend engineer at a Series A SaaS company whose shared API just got hammered into a p99 spike by one high-volume free-tier tenant, and whose current rate limiter applies a single global limit that is now throttling paying enterprise customers caught in the same bucket

If one tenant just melted your p99 and your global rate limiter is now throttling paying customers, the fix is a per-tenant sliding window backed by a Redis sorted set — where each tenant gets an isolated key, request timestamps are the scores, and the ZREMRANGEBYSCORE call that trims stale entries is the rate check. This eliminates the boundary-burst exploit that fixed-window counters have, and it isolates noisy neighbours without needing a separate limiter service. The rest of this post is a working implementation you can paste into a Node or Python service today.

We'll build it against Redis 7, walk the Lua script that makes it atomic, benchmark it, and cover the failure modes nobody mentions until you hit them in production.

Why your current limiter is punishing the wrong people

A single global counter (or worse, a single Nginx limit_req_zone keyed on nothing tenant-aware) treats your API as one shared bucket. When a free-tier tenant runs a backfill and burns 8,000 req/min, either:

• Your global limit is low enough to protect the API, so enterprise customers get 429s for the duration of the burst.
• Your global limit is high enough that the burst gets through, and your p99 goes with it.

The naive fix — a fixed-window counter per tenant, incremented in Redis with INCR and EXPIRE — solves neighbour isolation but introduces a new problem: the boundary burst. A tenant limited to 100 req/min can send 100 requests at 12:00:59 and another 100 at 12:01:00. That's 200 requests in one second, cleanly inside the limit as far as the counter can tell. If your downstream is Postgres or a third-party API with its own quota, you just doubled the effective burst.

Sliding window log fixes this because there is no window boundary. The window slides with every request.

The primitive: one Redis sorted set per tenant

The data structure is deceptively simple. For each tenant, keep a sorted set where:

• Member = a unique request ID (UUID, or timestamp+nonce)
• Score = the request timestamp in milliseconds

On every request, you do four operations in a single atomic script:

1. Remove all members with score older than now - window_ms. This is the sliding part.
2. Count remaining members with ZCARD.
3. If count < limit, add the new request and return allowed.
4. Set an expiry on the key equal to the window, so idle tenants free memory.

Trimming stale members is the rate check. There is no separate accounting step. The set's cardinality after the trim is, by definition, the number of requests in the last window_ms milliseconds.

Prerequisites

• Redis 7.0+ (any recent version works; we use 7 for the scripting engine)
• Node.js 18+ with ioredis, or Python 3.10+ with redis-py
• A local Redis instance: docker run -p 6379:6379 redis:7-alpine
• Basic familiarity with Lua (you'll read ~20 lines of it)

Step 1: Get Redis running and confirm connectivity

docker run -d --name rl-redis -p 6379:6379 redis:7-alpine
redis-cli ping

Expected output:

PONG

Step 2: Write the Lua script

Save this as sliding_window.lua. This runs server-side in Redis, so all four operations execute atomically — no race between the ZCARD and the ZADD.

-- KEYS[1] = tenant bucket key, e.g. "rl:tenant:acme:read"
-- ARGV[1] = current timestamp in ms
-- ARGV[2] = window size in ms
-- ARGV[3] = max requests in window
-- ARGV[4] = unique request id

local key = KEYS[1]
local now = tonumber(ARGV[1])
local window = tonumber(ARGV[2])
local limit = tonumber(ARGV[3])
local req_id = ARGV[4]

-- 1. Trim stale entries
redis.call('ZREMRANGEBYSCORE', key, 0, now - window)

-- 2. Count what remains
local count = redis.call('ZCARD', key)

if count < limit then
  -- 3. Record this request
  redis.call('ZADD', key, now, req_id)
  redis.call('PEXPIRE', key, window)
  return {1, limit - count - 1}
else
  -- Get oldest entry to compute retry-after
  local oldest = redis.call('ZRANGE', key, 0, 0, 'WITHSCORES')
  local retry_after = window - (now - tonumber(oldest[2]))
  return {0, retry_after}
end

Return value: [allowed, remaining_or_retry_ms]. When allowed=1, the second value is how many requests the tenant has left. When allowed=0, it's how many milliseconds until the oldest request ages out.

Step 3: Wire it up in Node

// limiter.js
import Redis from 'ioredis';
import { readFileSync } from 'fs';
import { randomUUID } from 'crypto';

const redis = new Redis();
const script = readFileSync('./sliding_window.lua', 'utf8');

redis.defineCommand('slidingWindow', {
  numberOfKeys: 1,
  lua: script,
});

export async function checkLimit(tenantId, route, limit, windowMs) {
  const key = `rl:${tenantId}:${route}`;
  const now = Date.now();
  const [allowed, meta] = await redis.slidingWindow(
    key, now, windowMs, limit, randomUUID()
  );
  return {
    allowed: allowed === 1,
    remaining: allowed === 1 ? meta : 0,
    retryAfterMs: allowed === 0 ? meta : 0,
  };
}

Step 4: Plug it into your request pipeline

Middleware for Express or Fastify. Tenant ID comes from your auth layer — JWT claim, API key lookup, whatever you already have.

// middleware.js
import { checkLimit } from './limiter.js';

const TIERS = {
  free:       { limit: 60,   windowMs: 60_000 },
  pro:        { limit: 600,  windowMs: 60_000 },
  enterprise: { limit: 6000, windowMs: 60_000 },
};

export async function rateLimit(req, res, next) {
  const tenant = req.auth.tenantId;
  const tier = req.auth.tier;
  const route = req.route.path;

  const { limit, windowMs } = TIERS[tier];
  const result = await checkLimit(tenant, route, limit, windowMs);

  res.setHeader('X-RateLimit-Limit', limit);
  res.setHeader('X-RateLimit-Remaining', result.remaining);

  if (!result.allowed) {
    res.setHeader('Retry-After', Math.ceil(result.retryAfterMs / 1000));
    return res.status(429).json({
      error: 'rate_limited',
      retryAfterMs: result.retryAfterMs,
    });
  }
  next();
}

Two things worth noticing:

• The bucket key includes the route. A tenant hammering /export doesn't burn their budget for /webhooks. Whether you want this depends on your API — for most SaaS APIs it's the right default.
• Tiers are just different (limit, windowMs) tuples handed to the same limiter. No forked code paths per plan.

Step 5: Test that noisy tenants don't affect neighbours

Save this as test_isolation.js:

import { checkLimit } from './limiter.js';

async function hammer(tenant, count) {
  let allowed = 0, blocked = 0;
  for (let i = 0; i < count; i++) {
    const r = await checkLimit(tenant, '/api/read', 60, 60_000);
    r.allowed ? allowed++ : blocked++;
  }
  return { tenant, allowed, blocked };
}

console.log(await hammer('noisy-free-tenant', 200));
console.log(await hammer('quiet-enterprise', 50));

Expected output:

{ tenant: 'noisy-free-tenant', allowed: 60, blocked: 140 }
{ tenant: 'quiet-enterprise', allowed: 50, blocked: 0 }

The noisy tenant hits their ceiling. The quiet tenant is untouched. That's the whole point.

Step 6: Verify the boundary-burst fix

Compare against a naive fixed-window counter. In a fixed-window implementation, sending 60 requests at t=59.9s and 60 more at t=60.1s both succeed — 120 requests in 200ms. With the sliding window:

// Fire 60, wait for the window to be near-full, fire 60 more
await hammer('test-tenant', 60);   // all allowed
await new Promise(r => setTimeout(r, 200));
const result = await hammer('test-tenant', 60);
console.log(result);

Expected output:

{ tenant: 'test-tenant', allowed: 0, blocked: 60 }

Every one of the second batch is blocked because they all fall inside the still-full sliding window. The oldest request has to age out before a new one is admitted. No boundary exploit exists.

Step 7: Benchmark it

Use redis-benchmark as a sanity check on the script's throughput:

redis-cli --eval sliding_window.lua rl:bench:test , $(date +%s%3N) 60000 100 $(uuidgen)

For real load, run the middleware behind autocannon or wrk. On a single Redis instance with the connection pooled, the limiter adds roughly one round trip per request. Real-world overhead we've seen in production services is sub-millisecond at the p50 and a few ms at p99 under load — but measure your own setup. Redis network latency dominates.

Step 8: Handle the operational edges

Memory bounds. Each active tenant's sorted set holds up to limit entries. With 10,000 active tenants at 6,000 req/min limit, that's up to 60M sorted set members. Each ZSET entry is ~80 bytes in practice, so plan for a few GB of Redis memory in the worst case. The PEXPIRE in the script means idle tenants free their memory automatically.

Redis outage. Decide fail-open vs fail-closed explicitly. Fail-open is usually right for SaaS APIs — a Redis blip shouldn't 429 everyone. Wrap the call in a try/catch and log the fallback.

try {
  const result = await checkLimit(...);
  // ...
} catch (err) {
  logger.error({ err }, 'rate_limiter_unavailable');
  next(); // fail open
}

Multi-region. If you run Redis per region, tenants get their region's limit independently. Usually fine. If you need a global limit, you need a globally-coordinated Redis (single primary with regional read replicas) and you'll pay for it in latency.

Common errors

NOSCRIPT error after Redis restart

If you're using EVALSHA for performance and Redis restarts, the script cache is gone. ioredis's defineCommand handles this automatically. If you're using raw eval, catch NOSCRIPT and re-send the full script.

Clock skew across app servers

The script uses ARGV[1] (the caller's timestamp), not Redis's TIME. If your app servers have drifted clocks, the window boundaries drift too. Switch to redis.call('TIME') inside the Lua script to use Redis's clock — but note this makes the script non-replicable in older cluster configurations. On Redis 7, use redis.replicate_commands() (which is default) and it's fine.

Sorted set grows unbounded for one tenant

Happens if you set windowMs to something huge (say, a daily quota) with no cap. The ZREMRANGEBYSCORE only trims older-than-window entries. For daily quotas, use a hybrid: a rolling short window for burst protection plus a simple INCR-with-daily-expire counter for the quota.

Tenant IDs from untrusted input

Never build the key from a raw header. Always derive tenant ID from an authenticated claim. Otherwise attackers rotate through fake tenant IDs and each one gets a fresh bucket.

Redis Cluster and hash tags

If you shard, the key rl:{tenant}:{route} spreads tenants across shards, which is what you want. But if you ever need multi-key operations for a tenant (unlikely here), use hash tags: rl:{acme}:read forces all keys for tenant acme onto the same shard.

How CodeNicely can help

Rate limiting is one slice of a bigger problem — a multi-tenant API architecture that stays fast as tenant count and per-tenant workload both grow. When we worked with GimBooks, a YC-backed accounting SaaS, we dealt with exactly this class of problem: a shared backend serving tenants with very different usage profiles, where noisy imports from one small business could not be allowed to slow ledger reads for another. The engagement covered request isolation, background job fairness, and read-path caching — the same pattern that applies here.

If your team is at the point where you're debugging p99 spikes and rate limiter design in the same week, that's usually a signal your platform needs a broader look. See how we work with scaleups or the full services overview.

Frequently Asked Questions

Is a sliding window log worth the extra Redis memory over a fixed window?

For most SaaS APIs, yes. The memory cost is bounded by limit × active_tenants, which is predictable. The gain is that you eliminate the boundary burst exploit and get accurate retry-after values for 429 responses. If you have millions of tenants with tiny per-tenant limits, a token bucket (two integers per tenant) is cheaper.

Should the rate limit key include the route or just the tenant?

Include the route if different endpoints have different costs — a /search that hits Elasticsearch shouldn't share a budget with a cheap /ping. Keep it tenant-only if your API is uniform. You can also assign weights: charge 10 tokens for expensive endpoints, 1 for cheap ones, by adjusting the ZADD score or adding N members.

What happens if Redis goes down?

You choose. Fail-open (skip the check, log the incident) is the safer default for user-facing SaaS APIs — a Redis outage shouldn't compound into a full API outage. Fail-closed makes sense only when the endpoint is protecting a very expensive downstream and you'd rather return 503 than let requests through.

Can I use this pattern with a serverless backend?

Yes, but the round-trip latency to a managed Redis (Upstash, ElastiCache Serverless, MemoryDB) becomes a bigger fraction of your request time. On Lambda with Upstash's REST API, expect 5-15ms of limiter overhead. Provisioned Redis in the same VPC is closer to 1ms. Batch checks across multiple limits into one Lua script if you need more than one.

How do I migrate from a global limiter without breaking existing clients?

Run both limiters in shadow mode for a week. Log what the per-tenant limiter would have done without enforcing it. Look for tenants that would suddenly get 429s under the new limits and either raise their tier, grandfather them, or reach out before cutover. For a personalized migration plan, contact CodeNicely for an assessment.