Rate Limiting

Rate limiting is built in and off until a policy names a quota. There is no limit that is right for every application, and a default one would be a number nobody chose refusing traffic nobody expected.

What is safe by default is what happens once a policy exists: the counters are bounded, the address is not taken from a header anyone can write, and a storage that stops answering stops traffic rather than stopping the limit.

A policy is several quotas

type Quota struct {
    Name   string        // the namespace its counters live under, reported to clients
    Window time.Duration // how long one counting period lasts
    Limit  int           // how many requests are allowed within one window
}

One number cannot tell a burst from sustained abuse. Three requests a second is generous for a person clicking and impossible for a script; a hundred a minute is the reverse. A policy that means "quick but not tireless" needs both.

// RateLimitPolicy is the application-wide budget every route inherits.
//
// Three windows rather than one, because a single number cannot tell a person
// clicking from a script that never stops. Every quota is counted for every
// request, so a client that overruns the short window still accrues against the
// long one.
func RateLimitPolicy() muzak.RateLimitOptions {
    return muzak.RateLimitOptions{
        Tracker: UserOrIPTracker,
        Quotas: []muzak.Quota{
            {Name: "short", Window: time.Second, Limit: 3},
            {Name: "medium", Window: 10 * time.Second, Limit: 20},
            {Name: "long", Window: time.Minute, Limit: 100},
        },
    }
}

app := muzak.New(muzak.AppOptions{Title: settings.AppName, Addr: settings.Addr},
    muzak.WithDependencies(core.GetQueryToken),
    // The application-wide budget, counted before the guard above runs so that
    // a request the guard rejects still costs the client something.
    muzak.WithRateLimit(core.RateLimitPolicy()),
)

Every quota is counted for every request, so pausing between bursts launders nothing.

A quota's Name is the namespace its counters are stored under, so two quotas that share a name share a budget and must agree on their window and limit. The application refuses to build when they do not. It must be a valid HTTP token, because it is reported to clients.

Narrowing it

// A health check: exempt. A monitor polling every second is the one client that
// should never be told to slow down.
r.Get("/healthz", handlers.Health, muzak.SkipRateLimit())

// A login route: stricter. Guessing a password is the one request worth making
// a hundred times a minute.
r.Post("/login/", handlers.Login,
    muzak.RateLimit(muzak.Quota{Name: "login", Window: time.Minute, Limit: 5}))

RateLimit replaces the quotas and keeps the storage and tracker it inherits, which is the common case. WithRateLimit layers the whole options struct field by field on top of what an enclosing scope declared, so a route can change one thing without restating the rest.

The quotas given replace those inherited rather than adding to them, so a route that wants both restates the ones it is keeping.

An exemption cannot be undone by a narrower scope: once a router is exempt, every route beneath it is.

When the count happens

By default, before the route's guards and dependencies. That is the half that matters for brute force: a request a guard rejects is still counted, so failed sign-ins cost the attacker their budget rather than being free. It also means a client past its limit is refused before anything expensive runs on its behalf.

AfterDependencies moves the count after them, which is what a tracker keying on a resolved identity needs, and gives the other half up.

r := muzak.NewRouter(muzak.WithTags("items"),
    // This is the one router whose routes resolve a caller, so it is the one
    // router where the budget is worth spending per caller rather than per
    // address. Deferring the count is what lets the tracker see the resolved
    // user; the quotas themselves are inherited unchanged.
    muzak.WithRateLimit(muzak.RateLimitOptions{AfterDependencies: true}))

Leave it off on a login route. A request rejected by a guard never reaches a limiter that defers, so a route that defers does not limit failed authentication at all, which is exactly the traffic a login route needs to limit.

Whose budget is spent

RateLimitTracker is func(ctx *muzak.Context) (string, error). It defaults to muzak.IPTracker, which keys on the address Context.ClientIP resolves.

// UserOrIPTracker spends a request from the caller's budget when there is a
// caller, and from the address's otherwise.
//
// The identity is read with TryFrom rather than From, because most routes here
// resolve no user at all and an absent dependency is a legitimate state for
// this tracker rather than a programming error. Keys from the two sources are
// prefixed differently so that a username can never collide with an address.
func UserOrIPTracker(ctx *muzak.Context) (string, error) {
    if user, ok := muzak.TryFrom[CurrentUser](ctx); ok {
        return "user:" + user.Username, nil
    }
    return muzak.IPTracker(ctx)
}

A tracker that requires a credential can insist on one. Returning an error abandons the request, and the error becomes the response exactly as one returned from a handler would:

func APIKeyTracker(ctx *muzak.Context) (string, error) {
    key := ctx.Header("X-API-Key")
    if key == "" {
        return "", muzak.Unauthorized("an API key is required")
    }
    return "apikey:" + key, nil
}

Two rules: the key must not be empty, and keys from different sources must be prefixed differently, so that a user identifier and an address can never collide into one budget.

Addresses that are cheap to change

IPTracker keys on the exact address, which stops fitting as soon as an address family is cheap to change. An IPv6 /64 is the block size most providers hand out, so a client holding one can present a different address on every request while never leaving a range only they hold, and each address is a fresh budget.

muzak.RateLimitOptions{Tracker: muzak.IPPrefixTracker(32, 64)}

That keeps IPv4 addresses exact while collapsing an IPv6 source down to the allocation it actually came from. It panics if either length is out of range for its family, which is a mistake worth catching where the tracker is built rather than on the first request.

The address itself is only as trustworthy as the deployment makes it. See Behind a Proxy.

Where the counters live

type RateLimitStorage interface {
    Increment(ctx context.Context, quota, key string, window time.Duration) (count int, reset time.Duration, err error)
}

That is the whole of what the limiter needs from the outside world.

The default, in memory

An application that names no storage gets one that counts in the process that serves the requests. It is the right answer for a single process and the wrong answer for several: counters held in one process are not shared with the next, so a limit of a hundred a minute becomes a hundred a minute per process.

The table is bounded in two directions, because it is keyed by something the client influences and an unbounded one would be a memory leak with a name. Expired counters are swept, a full table discards the counter closest to expiring, and a tracker key the client chose the length of is hashed rather than truncated, so two clients cannot be merged into one budget.

muzak.WithRateLimit(muzak.RateLimitOptions{
    Storage: muzak.NewMemoryRateLimitStorage(muzak.MemoryRateLimitOptions{
        MaxEntries:    10_000,
        SweepInterval: 30 * time.Second,
    }),
    Quotas: []muzak.Quota{{Name: "default", Window: time.Minute, Limit: 60}},
})

DefaultRateLimitMaxEntries is 100000 and DefaultRateLimitSweepInterval is one minute. Naming a storage explicitly is only necessary to change those bounds.

It implements muzak.Lifecycle, so it is started and stopped with the application, and stopping releases every counter it holds: a key is derived from whatever the tracker read, an address, a user identifier or an API key, and none of that should outlive the server that was counting it.

Len() reports how many counters it holds, which is what a metric or a test asking whether anything is accumulating wants.

A shared storage

Anything running more than once wants counters the processes share. The interface is three arguments wide so that whatever you already run can satisfy it.

// Increment counts one request against a quota for one client.
//
// The increment and the expiry happen in one round trip, so that two requests
// arriving together cannot both create the window.
func (s *RedisRateLimitStorage) Increment(
    ctx context.Context, quota, key string, window time.Duration,
) (int, time.Duration, error) {
    // The script runs INCR, then PEXPIRE when the counter is new, then PTTL.
    res, err := s.script.Run(ctx, s.client,
        []string{"ratelimit:" + quota + ":" + key},
        window.Milliseconds()).Result()
    if err != nil {
        return 0, 0, err
    }
    values := res.([]any)
    count := int(values[0].(int64))
    reset := time.Duration(values[1].(int64)) * time.Millisecond
    return count, reset, nil
}

muzak.WithRateLimit(muzak.RateLimitOptions{
    Storage: core.NewRedisRateLimitStorage(settings.RedisAddr),
    Quotas:  core.RateLimitPolicy().Quotas,
})

Two rules an implementation must follow:

• The count includes the request being counted, so the first request in a window returns one.
• The window is how long a newly created counter should live. Never extend the life of a counter that already exists, because a limit whose window restarts on every request is a limit that never resets.

The key is opaque and may contain any bytes. It is derived from client-supplied data and must never be logged, because it routinely carries an API key or a user identifier.

If the implementation also satisfies muzak.Lifecycle, the application starts it before serving and stops it after draining, so a pool or a sweeper needs no separate registration.

When the storage cannot answer

By default the request is refused with 503, because a limiter that cannot count is a limiter that is not enforcing anything, and an attacker who can reach the storage can choose the moment it stops answering.

muzak.RateLimitOptions{FailOpen: true}

That trades the guarantee for availability: a storage outage lets traffic through unmetered instead of turning into an outage of its own. The failure is logged either way, without the key.

What a client sees

Every counted request carries headers describing the budget, because a client that can see its own budget is a client that can stay inside it.

RateLimit-Policy is fixed for a route, so one response teaches a client the whole policy.

HTTP/1.1 429 Too Many Requests
RateLimit-Limit: 3
RateLimit-Remaining: 0
RateLimit-Reset: 1
RateLimit-Policy: 3;w=1, 20;w=10, 100;w=60
Retry-After: 1

{
  "error": {
    "code": "too_many_requests",
    "message": "the \"short\" rate limit of 3 requests per 1 seconds has been exceeded; retry in 1 seconds",
    "status": 429
  },
  "request_id": "0611f4b2-2f0a-4b57-9c1a-6e6a2e2f9b31"
}

Retry-After is set even when DisableHeaders turns the RateLimit headers off. Refusing a client without telling it when to come back is what produces a client that comes back immediately, forever.

Limiting a connected peer

ReadLimit bounds what one WebSocket message costs and MaxConnections bounds how many peers there are, but neither bounds a peer that stays inside both and simply never pauses. WSOptions.MessageLimits applies the same quotas, storage and tracker to messages.

r.WS("/items/{item_id}/ws", handlers.ItemSocket,
    muzak.WithWebSocket(muzak.WSOptions{
        MessageLimits: []muzak.Quota{
            {Name: "ws-messages", Window: time.Second, Limit: 10},
        },
    }))

A peer that goes over is closed with 1008 Policy Violation rather than left connected and ignored, because a message silently dropped is a protocol nobody can debug. The budget belongs to the client rather than the connection, so opening a second one does not buy a second budget.

Give those quotas names of their own unless a shared budget with the HTTP routes is what you want, since both live in one namespace. A route marked SkipRateLimit counts no messages either.

Options reference

Where to go next

Behind a Proxy covers which address a request is attributed to, and WebSockets covers the other bounds a connected peer runs into.