How should I test a database-driven Django CMS for 404 errors?

Question

I have designed a basic content management system in Django that uses the database to track Article objects, then displays these Article objects on my website.

Each Article uses template tags that I frequently update. Occasionally, I will accidentally break one or more articles on my site when I update a template tag. For example, if I change the required arguments for a template tag referenced by a given article, and forget to update the template tag code within that article, the article will break, resulting in a 404.

I would like an easy way to ensure that all of my article pages are still working after I make an update to my template tags. Unfortunately, as far as I know, there's no easy way to test to ensure that all of my articles serve 200 status codes — other than combing through them manually.

I've investigated the Django testing framework, but it doesn't seem to be the right solution to my problem. To automatically test all my articles for 404s, I'd need to duplicate the Articles table in my database and load it in as a fixture for my tests — and numerous people have warned me against cloning my live database.

Is there an easy way to test all of the articles on my website for 404 errors after I deploy a change to my code?

Answer 1

This is a much simpler problem than the one given in your previous question. It has quite a few solutions.

Google Webmaster Tools

If you don't mind a slight delay, you can make use of Google Webmaster Tools which will report all 404 errors detected by it's crawler.

Django TestCase

The very thing I was reluctant to recommend in the previous question. Why I am suggesting it now? Well the problem seems simpler. You don't need all the data in your live database. So it's much easier and quicker to create your fixtures.

You probably have a read only replica that you are using for backup. If you invoke ./manage.py test with the -k flag on the read only replica you can actually do quite a lot of testing. You can do quite a bit more if your code does not use any manual transactions.

A scraping framework

Though not exactly designed for finding 404 errors, a web scraping framework such as 404 can be used against your site to detect both internal and external 404 errors.

python requests

You can write a pytest test case to read all the URLs on your site and hit them using python requests or any other http api. By not using django unit tests, you are not troubled by having to setup a test database. pytest can asked to pull the data direct from your live server.

Selenium / Beautifullsoup

Similar to scraping framework above. This too can be used with pytest so that you don't need to set up a test database. Read access can be granted to a test user account on the main database.