1

I'm trying to use bulk_insert_mapping with SQL Alchemy and Firebird and I'm facing an issue. Note that a normal insert with session.add() works well.

# This is a simple example that bulk insert filenames in history table
session.bulk_insert_mappings(
    Historique,
    [dict(fichier=f'fichier_{i}') for i in range(1, 10)]
)
session.commit()

And I'm getting the following error:

ERROR    sqlalchemy.pool.impl.QueuePool:base.py:1868 Error closing cursor
Traceback (most recent call last):
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1786, in _execute_context
    result = context._setup_result_proxy()
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1406, in _setup_result_proxy
    result = self._setup_dml_or_text_result()
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 1488, in _setup_dml_or_text_result
    strategy.alternate_cursor_description or self.cursor.description
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/fdb/fbcore.py", line 3554, in __get_description
    if self.__valid_ps():
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/fdb/fbcore.py", line 3552, in __valid_ps
    and not dir(self._ps))
ReferenceError: weakly-referenced object no longer exists

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1865, in _safe_close_cursor
    cursor.close()
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/fdb/fbcore.py", line 3643, in close
    if is_dead_proxy(self._ps):
  File "/home/manu/Dev/my_project/env/lib/python3.8/site-packages/fdb/fbcore.py", line 481, in is_dead_proxy
    return isinstance(obj, weakref.ProxyType) and not dir(obj)
ReferenceError: weakly-referenced object no longer exists

This is how I create the session:

@lru_cache()
def _get_session_maker(user, password, host, port, name, echo):
    try:
        engine = create_engine(
            f'firebird://{user}:{password}@{host}'
            f':{port}/{name}?charset=utf8',
            echo=echo)
        # Test que la base existe bien sur le serveur
        engine.execute(text("SELECT 1 FROM RDB$DATABASE")).fetchone()
        return sessionmaker(autocommit=False, autoflush=False, bind=engine)
    except DatabaseError as e:
        raise DatabaseError(f"-- Base '{name}': Connexion impossible", e.params, e.orig)

I'm using bulk_insert_mapping on a PostgreSQL database in the same way and I don't have any problem. How can I solve this?

Mark Rotteveel
  • 100,966
  • 191
  • 140
  • 197
ebelair
  • 844
  • 11
  • 27
  • It looks like a bug in sqlachemy so its tech support is the right place to go. – user13964273 Jul 13 '21 at 12:27
  • I'm afraid there is no tech support for firebird :-( So I'm looking for a way to solve this even if I have to correct the library by myself. But I don't understand what is happening here. – ebelair Jul 13 '21 at 12:31
  • In this case it is good to look at similar errors with other databases like this one: https://stackoverflow.com/questions/61429078/python-mysql-referenceerror-weakly-referenced-object-no-longer-exists or that: https://gist.github.com/shuson/11209728 – user13964273 Jul 13 '21 at 12:35
  • Have you tried using the recommended [external dialect](https://github.com/pauldex/sqlalchemy-firebird)? – snakecharmerb Jul 13 '21 at 12:53
  • Yes I'm currently using it but it's not heavily maintained – ebelair Jul 13 '21 at 13:12
  • _"I'm afraid there is no tech support for firebird"_, but user13964273 was talking about support for SQL Alchemy (see https://www.sqlalchemy.org/support.html), not Firebird (and if you're talking about Firebird itself, there is community support and various companies offering commercial support). – Mark Rotteveel Jul 13 '21 at 13:20
  • This is why I'm asking here, maybe a SA dev or FB dev could help me. – ebelair Jul 13 '21 at 13:26
  • My goal is to find a way to solve the problem and fix the library if I can. So any help is welcome :-) – ebelair Jul 13 '21 at 13:27
  • 1
    Looks similar to [this](https://github.com/FirebirdSQL/fdb/issues/96) – snakecharmerb Jul 13 '21 at 16:42
  • Thanks @snakecharmerb, I could solve the problem by applying the fix to the library – ebelair Jul 15 '21 at 08:25

1 Answers1

0

This is a known issue as @snakecharmerb mentioned. I've created a fork that fixes it and a PR

ebelair
  • 844
  • 11
  • 27