0

I'm trying to write some unit tests for a flask application i'm building, this is the file that contains the flask application

app.py

from flask import Flask

app = Flask(__name__)


@app.route('/')
def home():
    return 'hello world'

and then the file that contains the tests

tests.py

from unittest import TestCase, main
from app import app
from multiprocessing import Process
import requests


class Test(TestCase):

    @classmethod
    def setUpClass(cls):
        cls.hostname = 'http://localhost:8000'

        with app.app_context():
            p = Process(target=app.run, kwargs={'port': 8000})
            p.start()
            cls.p = p

    def test_options(self):
        # print(self.p.is_alive()) # returns True but requests doesn't connect
        result = requests.post(self.hostname).text
        self.assertEqual(result, 'hello world')

    @classmethod
    def tearDownClass(cls):
        cls.p.terminate() # works fine if i comment it out
        pass


if __name__ == '__main__':
    main()

I came up with an idea to use the requests module to test the application instead of the test_client that comes with flask.

From my understanding of the unittest module,

The setUpClass method is called once, that creates a new process and starts the application.

After that i'm free to run various tests with request.post or request.get.

Then at the end of the day, when all the tests have been run tearDownClass is supposed to terminate the process.

When i try to run the tests this is what i get (The stacktrace is actually longer than this, but it all boils down to this)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=5000): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7faeeba183c8>: Failed to establish a new connection: [Errno 111] Connection refused',))

Because the flask process doesn't start and requests is unable to connect to localhost via port 8000.

but the strange part is if i comment out cls.p.terminate everything works properly and the test passes, the only problem is that i won't be able to kill the flask process again.

I know i can make the process daemonic or use the test_client like this:

class Test(TestCase):

    @classmethod
    def setUpClass(cls):
        cls.hostname = 'http://localhost:8000'

        with app.app_context():
            app.config['TESTING'] = True
            cls.app = app.test_client()

    def test_options(self):
        # print(self.p.is_live())
        result = self.app.get().data
        self.assertEqual(result, b'hello world')

and just move on with my life, but i really want to understand why my code doesn't work when i try to kill the process when i'm tearing down the test suite.

I've tried to use pdb to debug, but for some reason i don't know my terminal gets messed up when i try to type anything in the pdb prompt (Some letters don't show up in the terminal as i type, but when i hit enter i discover that the letters were actually there but were not visible, E.g to type print i might end up having prrrinnnttt because r and t refused to show up when i was pressing them)

danidee
  • 9,298
  • 2
  • 35
  • 55
  • At this point, your test harness is more convoluted and fragile than your app and would likely remain so even if you got it working. You're probably better off starting with just mocking, if needed moving on to the test setup outlined in the flask docs. – pvg Dec 10 '16 at 18:33
  • Yeah i know...but on the other hand, the `test_client` isn't really suitable for testing REST api's, cuz AFAIK the Response object only returns bytes (You have to handle the conversion to other data types yourself). My question is more of a "why question" than a "how question" – danidee Dec 10 '16 at 20:01
  • This might be because the flask app is not actually fully initialized before you try to tear it down. You should add some basic print logging so you can see the sequence of operations. You can also stick (as a bandaid measure, to check) a 5-10 second sleep after you p.start and leave the teardown uncommented - that will let you check that theory easily. – pvg Dec 10 '16 at 20:08
  • @pvg That was actually the problem!, FacePalm moment (i should have thought of that). Before the Flask process gets started, `requests` tries to make a connection to it and after several attempts it fails. Worst part was that i was killing the process in `tearDownClass` before it even got started... I really need to rest :) – danidee Dec 10 '16 at 20:22
  • Now i don't know if i should keep this question open – danidee Dec 10 '16 at 20:24
  • Yeah this sort of thing gets finicky easily. Generally, if you're finding yourself typing `import multiprocessing` it's probably time for a long contemplative walk or a nap. If you're in the early stages of development of your app, it'd still start with the mocks because that's going to catch the kind of problems you are likely to encounter - logic not handling possible conditions, etc. If you really need this functionality, you can almost certainly find existing harnesses (that target http servers in general, not flask specifically) that do this and someone has done the suffering for you. – pvg Dec 10 '16 at 20:36

1 Answers1

1

I don't know what's happening in your case, but in one of my projects I use a bash script like this to test:

#!/bin/bash

set -eux

python app.py &
sleep 3

set +e

python tests.py
result=$?
kill $(ps aux | grep app.py | grep -v grep | awk '{print $2}')
exit $result

This handles bringing up the server, running the tests, and then killing the server, and there is no mention of handling the server in the python tests.

Alex Hall
  • 34,833
  • 5
  • 57
  • 89
  • Thanks @AlexHall, i also knew i could handle this with a bash script, i just wanted to do it all from python, i've been able to now (by moving the process creation out of the Test class, to the top of the script and making it daemonic). so i guess the problem has something to do with unitTests methods and how they are work internally. – danidee Dec 10 '16 at 20:05