0

I use django unit tests to test my application. It worked very well until today. I integrated elasticsearch to the project with the 2 modules django_elasticsearch_dsl and django_elasticsearch_dsl_drf. Since then, almost none of my unit tests are working.

I got the same trace for all failed tests:

Traceback (most recent call last):
  File "/home/aurelien/dev/gsport/accounting_bo/tests.py", line 66, in setUp
    club = create_club()
  File "/home/aurelien/dev/gsport/dashboard/tests.py", line 38, in create_club
    club.save()
  File "/home/aurelien/dev/gsport/clubs/models.py", line 81, in save
    super().save(*args, **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django/db/models/base.py", line 739, in save
    self.save_base(using=using, force_insert=force_insert,
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django/db/models/base.py", line 787, in save_base
    post_save.send(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django/dispatch/dispatcher.py", line 180, in send
    return [
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django/dispatch/dispatcher.py", line 181, in <listcomp>
    (receiver, receiver(signal=self, sender=sender, **named))
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django_elasticsearch_dsl/signals.py", line 58, in handle_save
    registry.update(instance)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django_elasticsearch_dsl/registries.py", line 141, in update
    doc().update(instance, **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django_elasticsearch_dsl/documents.py", line 225, in update
    return self._bulk(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django_elasticsearch_dsl/documents.py", line 202, in _bulk
    return self.bulk(*args, **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/django_elasticsearch_dsl/documents.py", line 151, in bulk
    response = bulk(client=self._get_connection(), actions=actions, **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/helpers/actions.py", line 410, in bulk
    for ok, item in streaming_bulk(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/helpers/actions.py", line 329, in streaming_bulk
    for data, (ok, info) in zip(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/helpers/actions.py", line 256, in _process_bulk_chunk
    for item in gen:
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/helpers/actions.py", line 195, in _process_bulk_chunk_error
    raise error
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/helpers/actions.py", line 240, in _process_bulk_chunk
    resp = client.bulk(*args, body="\n".join(bulk_actions) + "\n", **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/client/__init__.py", line 472, in bulk
    return self.transport.perform_request(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 417, in perform_request
    self._do_verify_elasticsearch(headers=headers, timeout=timeout)
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 606, in _do_verify_elasticsearch
    raise error
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/transport.py", line 569, in _do_verify_elasticsearch
    _, info_headers, info_response = conn.perform_request(
  File "/home/aurelien/dev/gsport/venv/lib/python3.10/site-packages/elasticsearch/connection/http_urllib3.py", line 280, in perform_request
    raise ConnectionError("N/A", str(e), e)
elasticsearch.exceptions.ConnectionError: ConnectionError(<urllib3.connection.HTTPConnection object at 0x7f4be02be6e0>: Failed to establish a new connection: [Errno 111] Connection refused) caused by: NewConnectionError(<urllib3.connection.HTTPConnection object at 0x7f4be02be6e0>: Failed to establish a new connection: [Errno 111] Connection refused)

I think the cause of the problem is that I use the following code to populate ES automatically when the postgresql database is:

@registry.register_document
class ClubDocument(Document):
    class Index:
        # Name of the Elasticsearch index
        name = 'clubs'
        # See Elasticsearch Indices API reference for available settings
        settings = {'number_of_shards': 1,
                    'number_of_replicas': 0}

    class Django:
        model = Club # The model associated with this Document

What should I do to either ignore ES during unit tests or better integrate an ES instance during these tests?

darkvodka
  • 303
  • 1
  • 3
  • 11
  • 1
    Your error implies there's no ElasticSearch server running. Just like your tests need the SQL database server to be running, they need the ES server to run. – AKX May 23 '22 at 07:59
  • Yes, you are right. How to do with gitlab CI? – darkvodka May 23 '22 at 08:03
  • 1
    I googled "gitlab ci elasticsearch" and found https://stackoverflow.com/q/43373011/51685 ... – AKX May 23 '22 at 13:41

0 Answers0