Avoiding rate-limiting (throttling) in java (IRC)

Question

So I've been writing a java-based irc-bot, which is fairly simple. Makes a socket connection to a server, attaches buffered readers / writers to the socket, and then reads / writes using those buffers.

I've not run into issues except for when I was printing out a json object and got kicked off the network for excessive flood. I'm not sure of the message rate limit for the network, but I do pass all my messages through the same simple function.

I'm hoping to add some kind of x bytes / timeframe limit to it. Someone suggested that I look into NIO, but that seems to be over-engineering the issue a little bit.

Here's what my current write-to-irc channel method looks like:

public static void writeMsg(BufferedWriter writer, String Channel, String Message) {
    try {
        System.out.println("Bot Command: PRIVMSG " + Channel + Message); //diagnostic
        writer.write("PRIVMSG " + Channel + Message);
        writer.flush();
    } catch(IOException ioe) {
        ioe.printStackTrace();
    }
}

Example of what might be passed into / out of that function:

Input:

writer (my BufferedWriter object), "#Testing1234: ", "!wiki China";

Output:

PRIVMSG #BeginnersProgramming :http://en.wikipedia.org/wiki/China

FWIW, most rate limits would probably look primarily at the amount of messages sent in a timeframe, not neccessarily at their length. E.g. let's say you get a timeout for posting 10 messages of 100 chars each in 20 seconds,then you would probably be timed out as well if all messages were only 1 char long. Thus depending on message length distribution, simply checking the number of bytes sent in a timeframe might not be enough to escape a ban; you should rather (or additionally) check the number of messages sent in the timeframe. — l4mpi, Jan 21 '15 at 11:15
Yeah, I was speaking to an admin and it's both number of messages (which is easier to implement) and the length of their contents. — A_Elric, Jan 21 '15 at 17:23
You know there are some API's for irc bot? Maybe it's better to just implement, depending of what you want. here one I used some time ago: http://www.jibble.org/pircbot.php — G Bisconcini, Jan 29 '15 at 20:37

JimmyB · Accepted Answer · 2015-02-02T11:44:04.570

I'm hoping to add some kind of x bytes / timeframe limit to it

The easiest way to do that would be along the lines of

final int rateLimit = 50; // Bytes per second

final float msPerByte = 1000.0f / rateLimit;

writer.write(message);

int numBytes = message.length();

Thread.sleep((int)(msPerByte * numBytes + 1));

However, depending on the length of the message, the single burst of one message may be already triggering the rate limit of the server. In this case you'd have to split the message into smaller parts and send one after another with the appropriate delay between.

If the rate limit is enforced only on a messages-per-timeframe basis, all you need to do is sleep() a constant time after each message.

I believe though that there are more complex algorithms at work, which may include more time-based parameters to decide.

In this case I would suggest using Guava's `RateLimiter` instead of a basic `Thread.sleep(..)`, this way you can ensure you only slow down to a set rate at all times, docs: http://docs.guava-libraries.googlecode.com/git-history/master/javadoc/com/google/common/util/concurrent/RateLimiter.html — Alex, Jan 28 '15 at 17:15

score 0 · Answer 2 · edited Mar 27 '16 at 13:30

I would put a random sleep at the beginning of your method. This way, you can simulate human typing.

Not thread safe version

private static SecureRandom sr = SecureRandom.getInstance("SHA1PRNG", "SUN"); 

public static void writeMsg(BufferedWriter writer, String Channel, String Message){
    try {
        // Sleep randomly between 2 (inclusive) and 5 (exclusive) seconds.
        int duration = 2000 + sr.nextInt(3000);
        System.out.println("Pausing execution for " + duration + " ms..."); 
        TimeUnit.MILLISECONDS.sleep(duration);

        System.out.println("Bot Command: PRIVMSG " + Channel + Message); //diagnostic
        writer.write("PRIVMSG " + Channel + Message);
        writer.flush();
    } catch(IOException ioe){
       ioe.printStackTrace();
    } catch (InterruptedException e) {
       e.printStackTrace();
    }
}

Thread safe version

If you the method will be called by multiple threads consider this code using a lock. The lock adds some more randomness when sending messages.

private static final Lock writeLock = new ReentrantLock();
private static SecureRandom sr = SecureRandom.getInstance("SHA1PRNG", "SUN"); 

public static void writeMsg(BufferedWriter writer, String Channel, String Message){
    writeLock.lock();
    try {
        // Sleep randomly between 2 (inclusive) and 5 (exclusive) seconds.
        int duration = 2000 + sr.nextInt(3000);
        System.out.println("Pausing execution for " + duration + " ms..."); 
        TimeUnit.MILLISECONDS.sleep(duration);

        System.out.println("Bot Command: PRIVMSG " + Channel + Message); //diagnostic
        writer.write("PRIVMSG " + Channel + Message);
        writer.flush();
    } catch(IOException ioe){
       ioe.printStackTrace();
    } catch (InterruptedException e) {
       e.printStackTrace();
    } finally {
       writeLock.unlock();
    }
}

This is a little more sleek than the other option in this thread, can you re-implement this to be by bytes sent over time rather than just arbitrary time? — A_Elric, Jan 21 '15 at 17:21

Avoiding rate-limiting (throttling) in java (IRC)

2 Answers2

Not thread safe version

Thread safe version