0

I'm using MongoDB Atlas. I'm have this URL for connecting: mongodb+srv://<LOGIN>:<PASSWORD>@<URL>/<DB>?retryWrites=true&w=majority&authSource=admin.

I'm using this stack: flask, gunicorn, pymongo, mongoengine (i don't think that it is related, because mongoengine actually uses pymongo).

gunicorn config:

  • sync workers,
  • threads = 1,
  • workers count = 8

When connecting, i'm using connect=False because pymongo itself is not fork-safe. With connect=False it will connect to the DB before first operation. It need to be set to False, because gunicorn uses pre-fork model.

But I am faced with the following situation:

  1. On first request (endpoint makes some DB operations) i'm getting this error: pymongo.errors.OperationFailure: Authentication failed., full error: {'ok': 0, 'errmsg': 'Authentication failed.', 'code': 8000, 'codeName': 'AtlasError'}
  2. On second request everything is fine (i.e., DB operations were successfully completed).
  3. If you press many times on F5, then sometimes you will occur a "Authentication failed" error, and sometimes everything will be fine.

If i'm set connect=True, everything will be fine. I'm never seen this "Authentication failed" error.

But then i'm getting this warning message: UserWarning: MongoClient opened before fork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-pymongo-fork-safe.

So, obviously, connect need to be set to False.

I'm tested with two workers. Looks like this situation occurs:

  1. Let's say i have two gunicorn sync workers with following pid's: 22429 and 22430.
  2. I'm getting request. Worker 22429 handles it. Everything is fine, because it is first worker and looks like it successfully made a DB connection.
  3. I'm pressing F5, worker 22429 again handles it and everything is fine.
  4. I'm pressing F5 again, and now worker 22430 handles it. It throws an error "Authentication failed."
  5. I'm pressing F5 again, worker 22430 handles it. Now everything is fine, because that worker has made a connection.
  6. Now, all my workers (i have only two) were connected to the DB. Doesn't matter how many times i'm press on F5, every request will be successfully completed.

But why it happens on first reques? From pymongo: "if connect=False, then connect on the first operation.". If i'm understanding it right, pymongo should connect before actual first operation, successfully completed the connection, and only then perform that operation.

Then why my first DB operation fails for each worker? How i should handle this?

Amaimersion
  • 111
  • 8

1 Answers1

0

Actually, the problem was with Mongoengine, not PyMongo. On Mongoengine GitHub I didn't find anything that is related to this issue. I tried many things with Mongoengine to fix this issue, but nothing has worked. The only solution was to refuse Mongoengine and use only PyMongo.

I rewrote my project to use only PyMongo. Fortunately, it was only beginning of the project, so, it didn't take much time. After rewriting everything worked as expected with exact same configuration.

Unrelated to this question:

For those who thinking about using Mongoengine or PyMongo. Use PyMongo. It is much easier to work with PyMongo directly, because it implements actual MongoDB documentation. Mongoengine changes some concepts, and eventually it is becoming hard to read both official MongoDB documentation and Mongoengine documentation which implements some concepts from MongoDB documentation in a different manner. And yes, by refusing one more abstraction you will avoid problems like my question.

Amaimersion
  • 111
  • 8