Node js Rate Limit

Question

I am trying to set API rate limit on my app using express-rate-limit. It works if it is from the same IP address. I have an error message once it reaches a max of 5. However, it fails when it is tried from different IP address/computer. Any idea how I can fix this? I tried using 127.0.0.1 to generate a key regardless of which IP address but that failed as well.

Below is my code:

// Rate Limit
var RateLimit = require('express-rate-limit');

app.enable('trust proxy');

var limiter = new RateLimit({
  windowMs: 365*24*60*60*1000, // 1 year
  max: 5, // limit each IP to 1 requests per windowMs
  delayMs: 365*24*60*60*1000, // delaying - 365 days until the max limit is reached
  message: "Sorry, the maximum limit of 50 letters sent has been reached. Thank you for participating!",
  keyGenerator: function (req) {
    req.ip = "127.0.0.1";
    // req.ip = "ip address";
    return req.ip;
  }
});

app.use('/api/letter', limiter);

This seems very odd. You're trying to limit someone to 5 requests per year? Sorry, but that isn't about rate limiting. That's something else. That's why you can't find a rate limiter to do this. Rate limiting would typically be to limit someone to something like 5 requests per minute. It appears that you are just trying to limit how much someone can use your service at all. If that's the case, then you just need to keep some sort of session use counter that you store persistently for each user and you check upon each operation. This isn't rate limiting. — jfriend00, Feb 26 '17 at 15:34
And, to enforce something over a year long period, you will have to use a persistent store too which will again rule out rate limiting options. — jfriend00, Feb 26 '17 at 15:37
And, you will have to use some sort of persistent userID (verified by login) as a key. — jfriend00, Feb 26 '17 at 15:38
I am using a 3rd party vendor which charges me $0.95 per successful call. It automates letter writing. I am building a service that allows people to write a letter to local reps which I am funding to a certain extent. I don't want to pile up on bills. Second, I don't have resources to build db and manage user info. Hence, rate limiting. — Mihir Patel, Feb 26 '17 at 15:59
You are not going to find a rate limiting module that works over a year long period. You just won't. Because it is unlikely your server will NEVER get restarted in a year long period, you have to have persistent storage of the actions. And, you have to track them by some persistent userID too. Those are your requirements to meet your needs. That is not what rate limiting does. You're looking in the wrong pond for a solution. And, you will also need persistent userIDs to track this reliably (some sort of login system) so you have to have persistence for that too. — jfriend00, Feb 26 '17 at 16:06
I appreciate your feedback. 365 is what I started testing. The project will last for only 30 days. Should've made that clear. — Mihir Patel, Feb 26 '17 at 17:09
And, do you expect your server to always go 30 days without ever restarting? The point is that you need to persist this data to a data store so your data can survive a server restart. No garden variety rate limiting solution will do that for you. They assume short time periods and that it's OK to reset the data if a server restarts. Neither of those are true for your situation. Your situation is more about account usage limits than it is about rate limiting. Different problem. Different solution. — jfriend00, Feb 26 '17 at 17:16

score 8 · Accepted Answer · answered Feb 26 '17 at 13:30

8

The memory store implementation used by express-rate-limit uses setTimeout() to clear the store after windowMs milliseconds.

According to the Node.js documentation for setTimeout(),

When delay is larger than 2147483647 or less than 1, the delay will be set to 1.

In your case, the delay is larger than that amount, namely 31536000000 milliseconds. This results in the store never storing any data for more than 1ms.

To solve this, you probably have to implement your own store (see the store option), or perhaps look for an alternative rate limiter that doesn't have this limit (it seems to me that with such large expiry times, you'll need some sort of persistent storage anyway).

answered Feb 26 '17 at 13:30

robertklep

198,204
35
394
381

Thanks. Do you recommend any other rate limiter? I would prefer not to create my own persistent storage because of lack of resources. – Mihir Patel Feb 26 '17 at 13:40
1

I can't recommend something specific, but there are a few others mentioned in the [README of `express-rate-limit`](https://github.com/nfriedly/express-rate-limit). – robertklep Feb 26 '17 at 13:42
1

The OP's problem isn't really about rate limiting at all. They need an entirely different type of solution since their timing and persistence needs will no likely be met by any rate limiter. – jfriend00 Feb 26 '17 at 15:39

score 0 · Answer 2 · answered Mar 06 '17 at 02:47

I think it's perfectly reasonable to call this "rate limiting". Just because the time period is big (yearly) doesn't mean it's not a limit per time period.

https://www.ratelim.it/documentation/once_and_only_once Takes this even further and let's you do N times per "infinite" which is super useful.

You should be able to use this service to do 5 per year. (I run ratelim.it).

score 0 · Answer 3 · answered Jun 08 '18 at 14:22

rate-limiter-flexible package with Mongo can help set up rate limits for 1 year

const { RateLimiterMongo } = require('rate-limiter-flexible');
const { MongoClient } = require('mongodb');

const mongoOpts = {
  useNewUrlParser: true,
  reconnectTries: Number.MAX_VALUE, // Never stop trying to reconnect
  reconnectInterval: 100, // Reconnect every 100ms
};

const mongoConn = MongoClient.connect(
  'mongodb://localhost:27017',
  mongoOpts
);

const opts = {
  mongo: mongoConn,
  points: 5, // Number of points
  duration: 365*24*60*60, // Per 1 year
};

const rateLimiter = new RateLimiterMongo(opts);

app.use('/api/letter', (req, res, next) => {
  rateLimiter.consume(req.ip)
    .then(() => {
      next();
    })
    .catch((rejRes) => {
      res.status(429).send('Too Many Requests');
  });
);

It is also recommended to set up insuranceLimiter and block strategy. Read more here

Node js Rate Limit

3 Answers3