1

I have file download site. What I look for is limiting bandwidth per IP (!). Limit should be set dynamically by HTTP header from backend.

My current implementation uses X-Accel-Limit-Rate (I can change that header, it's not hard-coded anywhere), but it does limit only current connection/request.

Is my idea doable in G-Wan?

Misiek
  • 103
  • 8

1 Answers1

1

Yes, this can be done.

Write a G-WAN handler to extract the X-Accel-Limit-Rate HTTP header. Then enforce this policy by using the throttle_reply() G-WAN API call documented here.

An example available called throttle.c might help you further.

The throttle_reply() G-WAN function lets you apply throttling on a global basis or per connection, so you will just apply the relevant throttling values for either IP addresses or authenticated users, depending on your needs.

throttle_reply() can change the download speed dynamically during the lifespan of each connection so you can slow-down old connections and create new ones with an adaptive download rate.

Of course, this can be enforced on a per client IP address (or cookie, or even ISP/Datacenter AS record) to deal with huge workloads.

Gil
  • 3,279
  • 1
  • 15
  • 25
  • I know _throttle_reply()_ but I don't really understand/know how to limit per IP. I need something to prevent file downloaders using too much bandwidth. I can limit single connection at 10KB/s, but if user opens 10 connection he will use 100KB/s. My question is how to dynamically slow down all of those 10 connection to 1KB/s. – Misiek Jun 02 '14 at 15:45
  • 1
    @Misiek The simpler solution that other download sites apply is simply limit the number of connections that the client can establish. I think we can make use of an IP directory: record the client IP at HDL_AFTER_ACCEPT (or reject the connection if the client IP is already present in the IP directory), remove it at HDL_AFTER_WRITE. – Nagi Jun 03 '14 at 05:07
  • I know that is simpler but I'm just looking for limiting bandwidth per-IP. Currently I have 2 connections limit per-IP but it makes another problem. Now user can't download more than 2 files from one node, because he hits connection limit. This is why I look for per-IP bandwidth limit :) – Misiek Jun 04 '14 at 11:19
  • 1
    @Misiek: **throttle_reply()** lets you DYNAMICALLY throttle on a per connection basis so you can reduce people's download rate once they open more than one connection: just keep track of the currently connected IP addresses and increase/decrease a connections counter. Then, divide the bandwidth of each client per its number of connections when using **throttle_reply()**. That's the per-IP address feature you are looking for. – Gil Jun 04 '14 at 12:09
  • Is it possible to get free or paid example? I don't need anything special, just working script. I did it in nginx lua, but in my opinion it's not memory efficient. – Misiek Jun 22 '14 at 00:00
  • How long can it tak to get reply from them? – Misiek Jun 25 '14 at 16:13
  • Apparently, not that long. To receive an answer, it helps to send an email. There was no such a thing in our logs. Try emailing Pierre directly rathre than using the contact form (his mail address is on the same page as the contact form). – Gil Jun 27 '14 at 07:54
  • I sent message on 23.06 by the contact form. You should check if it works :) In a second I will send message directly. – Misiek Jun 28 '14 at 21:46
  • We got your message, which received a reply. – Gil Jun 30 '14 at 06:48
  • Last comment question. Will G-Wan team tell us how to user G-Wan's API in other than C languages? I wonna set throttle_reply from Lua, but it's null (undefined function). Is API available only from C? I'm asking here, because I'm not able to but support for my 3-months earnings value... – Misiek Jul 14 '14 at 17:06
  • The C API is available to the language runtimes that are loaded as a module (C, C++, C#, D, Java, Objective-C, PH7, etc.), executing their code in the G-WAN memory space. Other languages are run as a CGI, an extern process, and therefore have to use their own libraries and can only communicate with G-WAN through the CGI variables. While Lua and a few other languages could in theory be used as a module, nobody sponsored the development so far. [As this reply is "too long", I will split it over 2 messages]. – Gil Jul 16 '14 at 06:43
  • G-WAN supporting 17 languages, we cannot afford to investigate each language's C interface, most of them being poorly documented - and often severely bugged in multi-threaded environments. – Gil Jul 16 '14 at 06:44
  • I wonna pay for good solution, but I'm now starting with few projects and the fee is way too big in my opinion. I just need one script and paying 140€ looks much. – Misiek Jul 19 '14 at 07:50
  • 1
    Then find a C developer to help you write your programs. G-WAN helps all users, registered or not, but no company nor any consultant can afford to write custom applications for total strangers for free. And, by the way, that Swiss Francs, not Euros (1 Euro = 1.2 CHF). – Gil Jul 20 '14 at 06:28