Is there a way to use 'pool_reset_connection' from mysql-connector-python with MariaDB 10.4.7?

Question

I want to change my python program from a normal connection to a Connection Pool so that the database connection doesn't get lost when no queries are sent for some hours, as well as the database not being overwhelmed with a bunch of queries at once at peak usage.

I'm using Python 3.7.4, mysql-connector-python 8.0.17, and MariaDB 10.4.7. It works fine when I use a normal connection, but MariaDB apparently doesn't support the pool_reset_session setting of mysql.connector.pooling.MySQLConnectionPool

At the start of my code, it tries to create the database if it doesn't yet exist, and that is causing the errors I get.

import mysql.connector as mariadb
from mysql.connector import errorcode
from mysql.connector import pooling

cnx = mariadb.pooling.MySQLConnectionPool(user='root', password='password', host='localhost', 
                                          pool_name='connectionpool', pool_size=10, pool_reset_session=True)

try:
    db = cnx.get_connection()
    cursor = db.cursor()
    cursor.execute("CREATE DATABASE IF NOT EXISTS tests")
    print("Created database")
except mariadb.Error as err:
    print(f"Failed creating database: {err}")
finally:
    print("Finally (create)")
    db.close()

I expected that this snippet would just create the database tests but instead I got the following two errors:

mysql.connector.errors.NotSupportedError: MySQL version 5.7.2 and earlier does not support COM_RESET_CONNECTION.

as well as

mysql.connector.errors.OperationalError: 1047 (08S01): Unknown command

From the traceback logs, it looks like this gets caused by trying to execute db.close() in line 17.

Full output with traceback: https://pastebin.com/H3SAvA9N

I am asking what I can do to fix this and if it's possible at all to use this sort of connection pooling with MariaDB 10.4.7 (I get confused because it says that MySQL <= 5.7.2 doesn't support this reset of connections after use even though I use MariaDB 10.4.7)

I also found out that MariaDB Connector/J does provide such an option, called useResetConnection but I don't want to learn Java just for this feature.

Answer 1

I was facing the same issue with mysql-connector-python for mariaDB, and I downgraded mysql-connector-python version to 8.0.12 and it worked for me

Answer 2

As @Georg Richter indicate, MariaDB for historical reason return a version like "5.5.5-10.4.10-MariaDB-1:10.4.10+maria~bionic-log"

MySQL python connector check explicitly version (https://github.com/mysql/mysql-connector-python/blob/b034f25ec8037f5d60015bf2ed4ee278ec12fd17/lib/mysql/connector/connection.py#L1157) and since MariaDB server appears as version 5.5.5, an error is thrown.

Since MariaDB 10.2.6, you can explicitly add version to the [server] section of the cnf file.

With configuration like:

[server]
. . . 
version=5.7.99-10.4.10-MariaDB

Connector will see version 5.7.99, and behave accordingly.

Answer 3

When bumping the server version to 10.0, MariaDB had to add a prefix to the server version to avoid breaking replication (replication protocol expects a one digit major version number, for more information check this answer).

No matter if you use MariaDB 10.0 or 10.4, MySQL Connector/Python will always return version number 5.5.5:

>>> conn= mysql.connector.connect(user="foo")
>>> print(conn.get_server_version())
(5, 5, 5)
>>> cursor=conn.cursor()
>>> cursor.execute("select version()")
>>> row=cursor.fetchone()
>>> print(row)
('10.4.7-MariaDB-log',)

COM_RESET_CONNECTION which resets the connection on server side was introduced in MariaDB 10.2, so to make it work you have to change the code of MySQL Connector/Python, e.g. in _check_server_version (abstracts.py):

+        if server_version.startswith("5.5.5-")
+            server_version= server_version[6:]

This is of course not a generic solution, since it will not work for MariaDB versions prior 10.2. It might have also bad side effects when checking for certain features like X-Protocol, which isn't supported by MariaDB.