CRSF Token Interfering With TDD - Is there a variable that stores csrf output?

Question

So, I kept returning a Failing test in Django when comparing expected to actual html with form input, so I printed out the result and realized the difference was the rather simple line, caused by my {% csrf_token %}, as follows:

<input type='hidden' name='csrfmiddlewaretoken' value='hrPLKVOlhAIXmxcHI4XaFjqgEAMCTfUa' />

So, I expect a simple answer, but I haven't been able to find it: How do I render the result of a csrf_token for use in testing?

Here's the Test setup and failure:

def test_home_page_returns_correct_html_with_POST(self):
        request = HttpRequest()
        request.method = 'POST'
        request.POST['item_text'] = 'A new list item'

        response = home_page(request)

        self.assertIn('A new list item', response.content.decode())

        expected_html = render_to_string(
        'home.html',
        {'new_item_text': 'A new list item'},
******this is where I'm hoping for a simple one-line mapping******

    )
    self.assertEqual(response.content.decode(), expected_html)

Here's the rendering from views.py:

def home_page(request):
    return render(request, 'home.html', {
        'new_item_text': request.POST.get('item_text'),
    })

And here's the test failure, when I run the test with python manage.py test

FAIL: test_home_page_returns_correct_html_with_POST (lists.tests.HomePageTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Me\PycharmProjects\superlists\lists\tests.py", line 29, in test_home_page_returns_correct_html_with_POST
    self.assertEqual(response.content.decode(), expected_html)
AssertionError: '<!DO[298 chars]     <input type=\'hidden\' name=\'csrfmiddlew[179 chars]tml>' != '<!DO[298 chars]     \n    </form>\n\n    <table
 id="id_list_t[82 chars]tml>'

----------------------------------------------------------------------

Answer 1

Judging by the code snippet you provided, it seems you are working through the examples from the book "Test Driven Development with Python", but are not using Django 1.8.

This post from the book's Google Groups discussion addresses the test failure, as you are experiencing it:

https://groups.google.com/forum/#!topic/obey-the-testing-goat-book/fwY7ifEWKMU/discussion

And this GitHub issue (from the book's official repository) describes a fix consistent with your question:

https://github.com/hjwp/book-example/issues/8

Answer 2

If I may, I would like to propose a better way of performing this test, using the built-in Django test client. This will handle all CSRF checks for you, as well as being easier to use. It would look something like this:

def test_home_page_returns_correct_html_with_POST(self):
    url = reverse('your_home_page_view_url_name')
    response = self.client.post(url, {'item_text': 'A new list item'})
    self.assertContains(response, 'A new list item')

Note that this also uses assertContains, which is an assertion provided by the Django test suite.

Answer 3

The CSRF token is part of the template context data that you have available if you are using the Django TestCase classes:

response = self.client.get(url)
print(response.context)

https://docs.djangoproject.com/en/1.9/topics/testing/tools/#django.test.Response

The key is csrf_token.

https://docs.djangoproject.com/en/1.9/_modules/django/template/context_processors/

EDIT: As you have asked how you can compare the HTML rendered in your test to the output by your test server:

Because you are using {% csrf_token %} in your template you cannot provide the CSRF token form the response context to the render_to_string method to make it use the same value. Instead, you would have to replace it in the result of render_to_string, maybe by first looking for the input element using selenium (making that a test itself). However, how useful this test is is questionable. It will only help to assure that a CSRF token is present but that is already checked on the server in regular work mode, anyway.

Basically, you should test anything that you are influencing directly in your code and not anything offered by Django magic. E.g. if you are doing custom form validation you should test for that and not for any validation that is brought to you by Django. If you are changing querysets (custom filtering etc.) in ListViews or get_object() in DetailViews you should check that the resulting lists and 404 errors happen according to your custom code.

Answer 4

I ran in to this problem too (using latest python 3.6.12 and django 1.11.29 according to the 2nd edition of the book).

My solution doesn't answer your question 'how do I render the token' but it does answer 'how do I pass the test that compares the rendered template to the returned view response'.

I used the following code:

class HomePageTest(TestCase):
    def remove_csrf_tag(self, text):
        '''Remove csrf tag from text'''
        return re.sub(r'<[^>]*csrfmiddlewaretoken[^>]*>', '', text)

    def test_home_page_is_about_todo_lists(self):
        # Make an HTTP request
        request = HttpRequest()

        # Call home page view function
        response = home_page(request)

        # Assess if response contains the HTML we're looking for

        # First read and open the template file ..
        expected_content = render_to_string('lists/home.html', request=request)

        print(len(response.content.decode()))

        # .. then check if response is equal to template file
        # (note that response is in bytecode, hence decode() method)
        self.assertEqual(
            self.remove_csrf_tag(response.content.decode()),
            self.remove_csrf_tag(expected_content),
        )

PS: I based this on this answer.

Answer 5

I had similar problem so made a function to remove all the csrf tokens.

def test_home_page_returns_correct_html(self):
    request = HttpRequest()

    # Removes all the csrf token strings
    def rem_csrf_token(string):
        # Will contain everything before the token
        startStr = ''
        # Will contain everything after the token
        endStr = ''
        # Will carrry the final output
        finalStr = string

        # The approach is to keep finding the csrf token and remove it from the final string until there is no
        # more token left and the str.index() method raises value arror
        try:
            while True:
                # The beginning of the csrf token
                ind = finalStr.index('<input type="hidden" name="csrfmiddlewaretoken"')
                # The token end index
                ind2 = finalStr.index('">', ind, finalStr.index('</form>'))

                # Slicing the start and end string
                startStr = finalStr[:ind]
                endStr = finalStr[ind2+2:]

                # Saving the final value (after removing one csrf token) and looping again
                finalStr = startStr +endStr
        except ValueError:
            # It will only be returned after all the tokens have been removed :)
            return finalStr

    response = home_page(request)
    expected_html = render_to_string('lists/home.html')
    csrf_free_response = rem_csrf_token(response.content.decode())

    self.assertEqual(csrf_free_response,
                    expected_html, f'{expected_html}\n{csrf_free_response}')