Discord webhook rate limits

Question

I'm working on a private browser extension which extracts some information from a web page and posts it into a Discord channel via a webhook.

The browser extension does evaluate the x-ratelimit-... response headers to observe the rate limit restrictions.

While doing a "spam test" it seems that the rate limit restrictions are respected correctly and everything is working so far. However, every now and then I'm still getting rate limited after sending a stack of messages (15+) even though ratelimit-remaining is > 0.

To counter this I already stop when ratelimit-remaining is 1 and also add an additional second to the ratelimit-reset timestamp. But this doesn't seem to help.

let rateLimitRemaining = 5;
let rateLimitReset = 0;

function sendContent()
{
    if ( contentQueue.length > 0 )
    {
        console.log( "Messages in content queue: " + contentQueue.length );

        let content = contentQueue[ 0 ];
        let dateTimestamp = getCurrentUTCTimestamp();

        // Don't send if remaining rate limit is <= 1 and current UTC time is less than reset timestamp
        if ( rateLimitRemaining <= 1 && dateTimestamp <= rateLimitReset )
            return;

        contentQueue.shift();

        let url = "...";
        sendMessage( content, url );
    }
}

function sendMessage( content, url )
{
    let payload = JSON.stringify( { "content": content } );
    $.ajax(
    {
        contentType: 'application/json',
        method: "POST",
        url: url,
        data: payload,
        dataType: 'json'
    } ).done( function( response, status, jqXHR )
    {
        rateLimitRemaining = parseInt( jqXHR.getResponseHeader( 'x-ratelimit-remaining' ) );
        // Add an additional second to the reset timestamp
        rateLimitReset = parseInt( jqXHR.getResponseHeader( 'x-ratelimit-reset' ) ) + 1;

        let timeToResetRemaining = rateLimitReset - getCurrentUTCTimestamp();
        console.log( '[' + getCurrentDateTime() + '] Content sent to webhook. Remaining until rate limit: ' + rateLimitRemaining + ' / Reset @ ' + rateLimitReset + ' (' + getCurrentUTCTimestamp() + ') (' + timeToResetRemaining + ')' );
    } ).fail( function( jqXHR, status, error )
    {
        let response = jqXHR.responseJSON;

        // If we got rate limited, respect the retry_after delay
        if ( response.hasOwnProperty( 'message' ) && response.message.indexOf( 'rate limited' ) !== 0 )
        {
            rateLimitRemaining = 0;
            rateLimitReset = getCurrentUTCTimestamp() + Math.ceil( response.retry_after / 1000 ) + 1;
        }

        console.log( '[' + getCurrentDateTime() + '] Error sending request to webhook.' );
        console.log( response );
    } );
}

It is also strange that the same request, which triggers the rate limit, has its x-ratelimit-remaining response header at > 0.

What do I miss here? Where is my mistake? Do I need to take x-ratelimit-bucket and x-ratelimit-reset-after into account aswell?

Answer 1

A webhook that does not send a bot token is limited by a different set of rate limits to other API calls.

Each channel has a rate limit of maximum number of webhook messages that can be sent per minute, this limit is shared amongst all senders, so other bots and services might impact your ability to deliver a webhook. This limit is 30 messages per minute currently and not documented. Previously discord developers have tweeted this limit exists; https://twitter.com/lolpython/status/967621046277820416

The second limit is per IP, which you can monitor yourself and prevent any 429's from this. You will have to monitor the headers returned from the request to manage this.

on top of this, there is a final limit of 50 requests per second per IP; https://blog.xenon.bot/handling-rate-limits-at-scale-fb7b453cb235 - this is enforced at cloudflare.

Let me know if you have any more questions!

Answer 2

Based on the discord developer docs, it does seem strange that the rate limits in the headers are not zero. Unsure if it's the case in your application but it does mention that "Routes for controlling emojis do not follow the normal rate limit conventions. These routes are specifically limited on a per-guild basis to prevent abuse. This means that the quota returned by our APIs may be inaccurate, and you may encounter 429s.".

The developer docs state that you should rely on the Retry-After header or retry-after field to determine the retry time so it would be a good idea to do this rather than the other headers. A better pattern for API rate limiting regardless of the whether or not the remaining header is available or not is to add all queries to a queue, on success remove it from the queue, on failure with a 429 status, set a timeout for the queue based on the Retry-After header. This will ensure that your requests continue to get processed. You can definitely do this in browser side javascript, however I will leave this as an exercise for the reader as it is probably not the best way to do it anyway (see below).

If you plan on having anyone else use this code or release it then you'll want to not call the API directly from the browser as it will give access to your webhook credentials. Instead, have your browser authenticate against a server and then keep your webhook credentials hidden. Then if you were to use node.js for your server say, I would recommend using the discord.js SDK which handles rate limits automatically.