Resolve django test RuntimeError on channels/async testcases

Question

I'm trying to write a testcase for my django (3.2) channels consumer to implement websocket communication. I've managed to get the testcase to pass, but now it seems that the testcase teardown fails when trying to cleanup the test database.

System check identified no issues (0 silenced).
.
----------------------------------------------------------------------
Ran 1 test in 0.130s

OK
Destroying test database for alias 'default'...

...

psycopg2.errors.ObjectInUse: database "test_myproject" is being accessed by other users
DETAIL:  There is 1 other session using the database.

...
  File ".../python3.8/site-packages/django/core/management/commands/test.py", line 55, in handle
    failures = test_runner.run_tests(test_labels)
  File ".../python3.8/site-packages/django/test/runner.py", line 736, in run_tests
    self.teardown_databases(old_config)
  File ".../python3.8/site-packages/django/test/runner.py", line 674, in teardown_databases
    _teardown_databases(
  File ".../python3.8/site-packages/django/test/utils.py", line 313, in teardown_databases
    connection.creation.destroy_test_db(old_name, verbosity, keepdb)
  File ".../python3.8/site-packages/django/db/backends/base/creation.py", line 282, in destroy_test_db
    self._destroy_test_db(test_database_name, verbosity)
  File ".../python3.8/site-packages/django/db/backends/base/creation.py", line 298, in _destroy_test_db
    cursor.execute("DROP DATABASE %s"
  File ".../python3.8/contextlib.py", line 162, in __exit__
    raise RuntimeError("generator didn't stop after throw()")
RuntimeError: generator didn't stop after throw()

The consumer is written using async, and it seems I'm having trouble on how async integrates with the testcases. What else do I need to do to get the testcases to run properly.

I created and configured a fixture to load the data to the database, and suspect this has something to do with the testcase db interaction there...

I tried the standard django "TestCase" in place of "TransactionTestCase", but apparently the configured fixture wasn't loaded to the db as expected.

class ConsumerTestCase(TransactionTestCase):
    fixtures = ("basic-data", )

    async def test_alertconsumer__receive__valid(self):
        # defined in fixture
        username = "test-username"
        alert_id = 1

        url = "/alerts/ws/"
        communicator = WebsocketCommunicator(AlertConsumer.as_asgi(), url)
        connected, subprotocol = await communicator.connect()
        assert connected

        message = {"username": "me", "alert_id": 1 }
        await communicator.send_to(text_data=json.dumps(message))
        response = await communicator.receive_from()
        expected = {
            "message": "ACCEPTED"
        }
        self.assertDictEqual(actual, expected)

I have a similar problem. Did you come right? – Sheena Jan 11 '22 at 10:38 — Sheena, Jan 11 '22 at 10:38

monkut · Answer 1 · 2022-01-11T13:55:23.997

@Sheena

Not sure where I found this workaround, but I ended up creating a custom test runner to perform teardown of the database.

from django.conf import settings
from django.db import connection
from django.test.runner import DiscoverRunner


class DatabaseConnectionCleanupTestRunner(DiscoverRunner):
    def teardown_databases(self, old_config, **kwargs):
        # Django's ORM isn't cleaning up the connections after it's done with them.
        # The query below kills all database connections before dropping the database.
        with connection.cursor() as cursor:
            cursor.execute(
                f"""SELECT
                pg_terminate_backend(pid) FROM pg_stat_activity WHERE
                pid <> pg_backend_pid() AND
                pg_stat_activity.datname =
                  '{settings.DATABASES["default"]["NAME"]}';"""
            )
            print(f"Killed {len(cursor.fetchall())} stale connections.")
        super().teardown_databases(old_config, **kwargs)

And then defined the testrunner in the settings.py file.

(I created a 'commons' app and placed the DatabaseConnectionCleanupTestRunner there)

TEST_RUNNER = "commons.tests.tests.DatabaseConnectionCleanupTestRunner"

score 0 · Answer 2 · answered Jun 01 '22 at 08:59

The root cause of your problem is probably that the DB connections created by sync_to_async() are not killed off. The supported method around this is to use database_sync_to_async() which cleans up old DB connections when it exits. It can be imported from from channels.db import database_sync_to_async

some_model = await database_sync_to_async(SomeModel.objects.create)(
    name="A", param="hello"
)

Another point might be to use thread_sensitive=False which caused problems when upgrading to asgiref 3.5.2 from 3.5.1 or lower.

some_model = await database_sync_to_async(
    SomeModel.objects.create, thread_sensitive=False
)(name="A", param="hello")

Resolve django test RuntimeError on channels/async testcases

2 Answers2