Rate Limiting

Rate limiting is built in and off until a policy names a quota. There is no limit that is right for every application, and a default one would be a number nobody chose refusing traffic nobody expected.

What is safe by default is what happens once a policy exists: the counters are bounded, the address is not taken from a header anyone can write, and a storage that stops answering stops traffic rather than stopping the limit.

A policy is several quotas

type Quota struct {
    Name   string        // the namespace its counters live under, reported to clients
    Window time.Duration // how long one counting period lasts
    Limit  int           // how many requests are allowed within one window
}

One number cannot tell a burst from sustained abuse. Three requests a second is generous for a person clicking and impossible for a script; a hundred a minute is the reverse. A policy that means "quick but not tireless" needs both.

// RateLimitPolicy is the application-wide budget every route inherits.
//
// Three windows rather than one, because a single number cannot tell a person
// clicking from a script that never stops. The quotas are counted longest
// window first, so a client that overruns the short window has already accrued
// against the long one.
func RateLimitPolicy() muzak.RateLimitOptions {
    return muzak.RateLimitOptions{
        Tracker: UserOrIPTracker,
        Quotas: []muzak.Quota{
            {Name: "short", Window: time.Second, Limit: 3},
            {Name: "medium", Window: 10 * time.Second, Limit: 20},
            {Name: "long", Window: time.Minute, Limit: 100},
        },
    }
}

app := muzak.New(muzak.AppOptions{Title: settings.AppName, Addr: settings.Addr},
    muzak.WithDependencies(core.GetQueryToken),
    // The application-wide budget, counted before the guard above runs so that
    // a request the guard rejects still costs the client something.
    muzak.WithRateLimit(core.RateLimitPolicy()),
)

The quotas are counted longest window first, whatever order they are declared in, and the first to refuse a request ends the count. A request refused by the short quota has already been counted against the long one, so pausing between bursts launders nothing. A request the long quota refuses is not counted against the shorter ones, whose budget it would only waste, and Retry-After is the wait of the quota that refused. RateLimit-Policy still lists the quotas as they were declared.

A window is fixed, not sliding. A client's counter starts with its first request and ends Window later, and the next request opens a new one, so a client can spend a whole Limit at the end of one window and another at the start of the next: up to twice Limit in a span of Window. The short quota in the policy above is what bounds that.

A quota's Name is the namespace its counters are stored under, so two quotas that share a name share a budget and must agree on their window and limit. The application refuses to build when they do not. It must be a valid HTTP token, because it is reported to clients.

What is counted

A route counts against the policy it inherits. So does everything else that answers a request on the application's behalf:

• A Frontend or Static mount applies the rate limit of the router it was registered on, ahead of its guards, so a file request costs budget like any other. Use SkipRateLimit() to keep a mount outside a quota.
• A handler served with Router.Mount is counted the same way, against the policy of its routers and of the options given to Mount. See Mounting net/http Handlers.
• A tool call made through an App.MCP endpoint is served in-process by the route it calls, so it is counted against that route's policy, under the calling client's address, as a request to the route would be. The message that carries it is counted against the endpoint's policy as well. See MCP Tools.
• The documentation page and /openapi.json are counted in the same budget as the routes. Once the docs ran the application's guards but not its limit, /openapi.json was a place to guess the application's token as fast as a client could send. Exempt it by passing muzak.SkipRateLimit() to muzak.New, or serve no documentation.
• OPTIONS and 405 answers are not counted. They are answered from the route table alone, as net/http.ServeMux does, so they skip a route's guards and limit. Middleware installed with App.Use still covers them: put a limit there when the existence of a path is itself a secret.
• The health endpoints of AppOptions.Health are not counted either. A probe is answered ahead of routing and of App.Use middleware, and runs no guard, provider or rate limit, since a probe that a quota refused would take the instance out of rotation. See Health Checks.

Narrowing it

// A health check: exempt. A monitor polling every second is the one client that
// should never be told to slow down.
r.Get("/healthz", handlers.Health, muzak.SkipRateLimit())

// A login route: stricter. Guessing a password is the one request worth making
// a hundred times a minute.
r.Post("/login/", handlers.Login,
    muzak.RateLimit(muzak.Quota{Name: "login", Window: time.Minute, Limit: 5}))

RateLimit replaces the quotas and keeps the storage and tracker it inherits, which is the common case. WithRateLimit layers the whole options struct field by field on top of what an enclosing scope declared, so a route can change one thing without restating the rest.

The quotas given replace those inherited rather than adding to them, so a route that wants both restates the ones it is keeping.

An exemption cannot be undone by a narrower scope: once a router is exempt, every route beneath it is.

When the count happens

By default, before the route's guards and dependencies. That is the half that matters for brute force: a request a guard rejects is still counted, so failed sign-ins cost the attacker their budget rather than being free. It also means a client past its limit is refused before anything expensive runs on its behalf.

AfterDependencies moves the count after them, which is what a tracker keying on a resolved identity needs, and gives the other half up.

r := muzak.NewRouter(muzak.WithTags("items"),
    // This is the one router whose routes resolve a caller, so it is the one
    // router where the budget is worth spending per caller rather than per
    // address. Deferring the count is what lets the tracker see the resolved
    // user; the quotas themselves are inherited unchanged.
    muzak.WithRateLimit(muzak.RateLimitOptions{AfterDependencies: true}))

Leave it off on a login route. A request rejected by a guard never reaches a limiter that defers, so a route that defers does not limit failed authentication at all, which is exactly the traffic a login route needs to limit.

Limits that vary per caller

A Quota is fixed when the application is built, and the same name may not carry two policies anywhere in it. That is right for a policy the application owns, and it cannot express one its customers do: a plan, a negotiated ceiling, a tier read from a database.

Resolver supplies quotas per request.

// The plan was resolved by a dependency, so the count has to happen after
// them. It is read from whatever the dependency already fetched rather than
// from the database, because this runs on every request.
func PlanQuotas(ctx *muzak.Context) ([]muzak.Quota, error) {
    plan, ok := muzak.TryFrom[Plan](ctx)
    if !ok {
        // Nothing resolved, nothing enforced. An unauthenticated request
        // reaching this route is limited by the static quotas, if any.
        return nil, nil
    }
    return plan.Quotas, nil
}

muzak.WithRateLimit(muzak.RateLimitOptions{
    AfterDependencies: true,
    Tracker:           core.UserOrIPTracker,
    Resolver:          core.PlanQuotas,
    // A floor everyone shares, whatever their plan says.
    Quotas: []muzak.Quota{{Name: "burst", Window: time.Second, Limit: 20}},
})

Resolved quotas are counted alongside the static ones, under their own names, in the same storage. A route can therefore carry both a floor everyone shares and a ceiling that varies, and RateLimit-Policy describes the union.

Setting a resolver turns limiting on for the route even with no static quota, because what is enforced becomes a run time question. Returning nothing enforces nothing.

A quota built at run time is not validated when the application is built, so one that breaks the rules a static quota follows fails the request with a 500 rather than being counted. Both paths share one check: a name that is not a valid token (a colon in plan:x for client y collides with plan for client x:y in the storage key recommended below, and a NUL does the same in the in-process storage), a zero window or limit, and a name repeated within one policy or repeating a static one. A repeated name used to count every request once per copy against one counter, so a limit of two refused the second request.

Nothing compares a resolver's names against the ones declared statically beyond that, so a name used in both places must mean the same thing in both.

Whose budget is spent

RateLimitTracker is func(ctx *muzak.Context) (string, error). It defaults to muzak.IPTracker, which keys on the address Context.ClientIP resolves.

// UserOrIPTracker spends a request from the caller's budget when there is a
// caller, and from the address's otherwise.
//
// The identity is read with TryFrom rather than From, because most routes here
// resolve no user at all and an absent dependency is a legitimate state for
// this tracker rather than a programming error. Keys from the two sources are
// prefixed differently so that a username can never collide with an address.
func UserOrIPTracker(ctx *muzak.Context) (string, error) {
    if user, ok := muzak.TryFrom[CurrentUser](ctx); ok {
        return "user:" + user.Username, nil
    }
    return muzak.IPTracker(ctx)
}

A tracker that requires a credential can insist on one. Returning an error abandons the request, and the error becomes the response exactly as one returned from a handler would:

func APIKeyTracker(ctx *muzak.Context) (string, error) {
    key := ctx.Header("X-API-Key")
    if key == "" {
        return "", muzak.Unauthorized("an API key is required")
    }
    return "apikey:" + key, nil
}

Two rules: the key must not be empty, and keys from different sources must be prefixed differently, so that a user identifier and an address can never collide into one budget.

Addresses that are cheap to change

IPTracker counts an IPv4 address exactly, under a key such as ip:192.0.2.7. An IPv6 address is counted by its /56, under a key such as ip:2001:db8:1::/56, because a client is delegated a whole range of IPv6 addresses and would otherwise get a fresh budget from every address in it. A /56 is what providers delegate to one home or small site (RFC 6177). Keying on the /64 still handed one subscriber 256 budgets: rotating the low bits of the fourth group of an address, all inside their own /56, turned a limit of 5 a minute into more than a thousand.

The prefix is the one the per-client connection caps count, and one setting chooses both: ClientIPOptions.ConnectionIPv6Prefix, defaulting to 56, with ConnectionIPv4Prefix at 32. Any length from 32 to 128 is accepted.

muzak.AppOptions{
    ClientIP: muzak.ClientIPOptions{ConnectionIPv6Prefix: 48},
}

• Where sites are delegated a /48, or the deployment is hostile to begin with, set 48.
• Where many unrelated users share one /56 (a campus, or a carrier that gives each device a /64), set 64 to keep the old grouping. That also loosens the connection caps to match.
• For one route only, IPPrefixTracker chooses its own lengths: 0 to 32 for IPv4 and 0 to 128 for IPv6.

muzak.RateLimitOptions{Tracker: muzak.IPPrefixTracker(32, 64)}

That keeps IPv4 addresses exact while collapsing an IPv6 source down to the allocation it actually came from. It panics if either length is out of range for its family, which is a mistake worth catching where the tracker is built rather than on the first request. IPPrefixTracker(32, 128) counts every IPv6 address on its own, which is only safe where something in front has already bounded them.

A counter kept in a shared storage under an IPv6 client's old ip:2001:db8:1:2::/64 key is not carried over, so IPv6 clients start with a full budget once after the upgrade. IPv4 keys are spelled as before.

The address itself is only as trustworthy as the deployment makes it. A zone on an address (fe80::1%eth0) is dropped before a key is made, so one client cannot mint a key per zone suffix. See Behind a Proxy.

Where the counters live

type RateLimitStorage interface {
    Increment(ctx context.Context, quota, key string, window time.Duration) (count int, reset time.Duration, err error)
}

That is the whole of what the limiter needs from the outside world.

The default, in memory

An application that names no storage gets one that counts in the process that serves the requests. It is the right answer for a single process and the wrong answer for several: counters held in one process are not shared with the next, so a limit of a hundred a minute becomes a hundred a minute per process.

The table is bounded in two directions, because it is keyed by something the client influences and an unbounded one would be a memory leak with a name. Expired counters are swept, a full table evicts the counter closest to expiring from the quota the new counter belongs to, and a tracker key the client chose the length of is hashed rather than truncated, so two clients cannot be merged into one budget. Only a quota holding no counter at all takes its first from the quota holding the most.

Eviction used to be global, so flooding one quota with new keys, which is easy from one IPv6 range with a tracker that counts every address, evicted and so reset a per-account login limit. 0.2.8 took the counter from the quota holding the most, which held only while the flooded quota was the largest: a client could mint keys under a smaller quota keyed on something it chooses, such as an API key counted before any guard, and each key evicted the client's own spent counter in a larger per-address quota. Now a flood can only evict its own quota's counters. A full table therefore keeps each quota at the size it had when it filled, and a quota whose traffic grows afterwards gains room only as counters expire, so size MaxEntries for the keys the application really sees. A count saturates at the top of an int instead of wrapping under every limit, and a window near 292 years no longer makes RateLimit-Reset negative: seconds clamp to what a 32-bit int holds.

muzak.WithRateLimit(muzak.RateLimitOptions{
    Storage: muzak.NewMemoryRateLimitStorage(muzak.MemoryRateLimitOptions{
        MaxEntries:    10_000,
        SweepInterval: 30 * time.Second,
    }),
    Quotas: []muzak.Quota{{Name: "default", Window: time.Minute, Limit: 60}},
})

DefaultRateLimitMaxEntries is 100000 and DefaultRateLimitSweepInterval is one minute. Naming a storage explicitly is only necessary to change those bounds.

It implements muzak.Lifecycle, so it is started and stopped with the application, and stopping releases every counter it holds: a key is derived from whatever the tracker read, an address, a user identifier or an API key, and none of that should outlive the server that was counting it.

Len() reports how many counters it holds, which is what a metric or a test asking whether anything is accumulating wants.

A shared storage

Anything running more than once wants counters the processes share. The interface is three arguments wide so that whatever you already run can satisfy it.

// Increment counts one request against a quota for one client.
//
// The increment and the expiry happen in one round trip, so that two requests
// arriving together cannot both create the window.
func (s *RedisRateLimitStorage) Increment(
    ctx context.Context, quota, key string, window time.Duration,
) (int, time.Duration, error) {
    // The script runs INCR, then PEXPIRE when the counter is new, then PTTL.
    res, err := s.script.Run(ctx, s.client,
        []string{"ratelimit:" + quota + ":" + key},
        window.Milliseconds()).Result()
    if err != nil {
        return 0, 0, err
    }
    values := res.([]any)
    count := int(values[0].(int64))
    reset := time.Duration(values[1].(int64)) * time.Millisecond
    return count, reset, nil
}

muzak.WithRateLimit(muzak.RateLimitOptions{
    Storage: core.NewRedisRateLimitStorage(settings.RedisAddr),
    Quotas:  core.RateLimitPolicy().Quotas,
})

Two rules an implementation must follow:

• The count includes the request being counted, so the first request in a window returns one.
• The window is how long a newly created counter should live. Never extend the life of a counter that already exists, because a limit whose window restarts on every request is a limit that never resets.

The key is opaque and may contain any bytes. It is derived from client-supplied data and must never be logged, because it routinely carries an API key or a user identifier.

If the implementation also satisfies muzak.Lifecycle, the application starts it before serving and stops it after draining, so a pool or a sweeper needs no separate registration.

Increment must return when its context ends. The limiter cancels the context after RateLimitOptions.StorageTimeout, which is muzak.DefaultRateLimitStorageTimeout (2 seconds) unless it says otherwise. Every client library's context-taking call does return, so passing ctx through is all it takes.

When the storage cannot answer

By default the request is refused with 503 and Retry-After: 5, because a limiter that cannot count is a limiter that is not enforcing anything, and an attacker who can reach the storage can choose the moment it stops answering. The Retry-After is set even with DisableHeaders, so a refused client does not retry at once against a store trying to recover.

muzak.RateLimitOptions{FailOpen: true}

That trades the guarantee for availability: a storage outage lets traffic through unmetered instead of turning into an outage of its own. The failure is logged either way, without the key.

A store that stops answering is a different failure from one that returns an error. A call that hangs never produces an error for FailOpen to act on, and used to hold every request that reached it until its client gave up. StorageTimeout bounds each call: one that outlives it has its context cancelled and counts as a storage failure, which is a 503 with Retry-After: 5, or an unmetered request with FailOpen. A negative value removes the bound. The process-local storage is called without a deadline.

muzak.RateLimitOptions{StorageTimeout: 500 * time.Millisecond}

With FailOpen, only the failing quota goes unmetered. The others in the policy still decide, so a request over the limit of a quota that answered is refused, and a store that fails for some counters only, one shard of a cluster for instance, does not stop every other limit of the route from holding. When no quota was counted at all, no RateLimit headers are sent.

What a client sees

Every counted request carries headers describing the budget, because a client that can see its own budget is a client that can stay inside it.

RateLimit-Policy is fixed for a route, so one response teaches a client the whole policy.

HTTP/1.1 429 Too Many Requests
RateLimit-Limit: 3
RateLimit-Remaining: 0
RateLimit-Reset: 1
RateLimit-Policy: 3;w=1, 20;w=10, 100;w=60
Retry-After: 1

{
  "error": {
    "code": "too_many_requests",
    "message": "the \"short\" rate limit of 3 requests per 1 seconds has been exceeded; retry in 1 seconds",
    "status": 429
  },
  "request_id": "0611f4b2-2f0a-4b57-9c1a-6e6a2e2f9b31"
}

Retry-After is set even when DisableHeaders turns the RateLimit headers off. Refusing a client without telling it when to come back is what produces a client that comes back immediately, forever.

Limiting a connected peer

ReadLimit bounds what one WebSocket message costs and MaxConnections bounds how many peers there are, but neither bounds a peer that stays inside both and simply never pauses. WSOptions.MessageLimits applies the same quotas, storage and tracker to messages, counted in the same order as a request's: longest window first, stopping at the first quota that refuses.

r.WS("/items/{item_id}/ws", handlers.ItemSocket,
    muzak.WithWebSocket(muzak.WSOptions{
        MessageLimits: []muzak.Quota{
            {Name: "ws-messages", Window: time.Second, Limit: 10},
        },
    }))

A peer that goes over is closed with 1008 Policy Violation rather than left connected and ignored, because a message silently dropped is a protocol nobody can debug. The budget belongs to the client rather than the connection, so opening a second one does not buy a second budget. Pings, pongs and empty fragments are not messages, but a connection that sends more than 65536 of them in a minute is closed with 1008 whatever else it does, so a peer cannot dodge the message count by never completing a message.

Give those quotas names of their own unless a shared budget with the HTTP routes is what you want, since both live in one namespace. A route marked SkipRateLimit counts no messages either.

Options reference

Where to go next

Behind a Proxy covers which address a request is attributed to, and WebSockets covers the other bounds a connected peer runs into.