6

I'm getting a lot of spambot requests (requests with referrer as a spam site). How do I reject a request prematurely with process_request on middleware so that django simply don't respond to the requests made from a specific referrer?

All Іѕ Vаиітy
  • 24,861
  • 16
  • 87
  • 111
  • 7
    I think the better option would be to fend off such requests at the [webserver level](http://serverfault.com/questions/289785/where-to-find-and-how-to-block-blacklisted-ips-and-proxies) – Pynchia Jul 11 '15 at 09:34
  • 1
    If the request reaches Django, it will be taking up resources. I'm not sure if that is your main concern here, or if the spam on your site is, but it's a good idea to use a firewall or something that just drops the incoming request according to some rules, to prevent that it takes up precious resources that could be used elsewhere. – knbk Jul 11 '15 at 10:18
  • Side question: What is the purpose of such spambot requests? What do they think they will accomplish? – skaz Jul 11 '15 at 10:21
  • 1
    I agree with @Pynchia it'd be most efficient to do this at the earliest stage possible, i.e. at proxy or webserver level, before it even hits Django... Django middleware is quite late in the request processing pipeline – Anentropic Jul 11 '15 at 13:49
  • @knbk thats right. would consider that. – All Іѕ Vаиітy Jul 11 '15 at 15:41

2 Answers2

14

You can create a RejectSpambotRequestsMiddleware class which will reject the requests if the referer of the request is from a specific referrer.

It should return either None or an HttpResponse object. If it returns None, Django will continue processing this request, executing any other process_request() middlewares, then, process_view() middleware, and finally, the appropriate view. Normally, a 403 Forbidden response is sent to the user if an incoming request fails the checks performed by the middleware.

from django.http import HttpResponseForbidden    

class RejectSpambotRequestsMiddleware(object):  

    def process_request(self, request):  
        referer = request.META.get('HTTP_REFERER')
        if referer == 'spambot_site_referer':
            return HttpResponseForbidden() # reject the request and return 403 forbidden response

        return # return None in case of a valid request

Then add your middleware to the MIDDLEWARE_CLASSES in your settings.py file. 

MIDDLEWARE_CLASSES = (
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    ...
    ...
    # your custom middleware here
    'my_project.middlewares.RejectSpambotRequestsMiddleware',
)

Note: Here, RejectSpambotRequestsMiddleware will be run at the end as Django applies middleware in the order it’s defined in MIDDLEWARE_CLASSES, top-down. You can change the order of MIDDLEWARE_CLASSES as per your need.

Rahul Gupta
  • 46,769
  • 10
  • 112
  • 126
4

Added some modifications to the answer of @Rahul Gupta. Referrer is compared to a blacklist.

class RejectSpambotRequestsMiddleware(object):

    def __init__(self, get_response):
        self.get_response = get_response
        self.blacklist = ['bot1.com', 'bot2.com']

    def __call__(self, request):  
        referer = request.META.get('HTTP_REFERER')
        response = self.get_response(request)

        if not referer:
            return response

        for bad in self.blacklist:
            if bad in referer:
                return HttpResponseForbidden()
        return response
ohlr
  • 1,839
  • 1
  • 13
  • 29