Enable executing multiple statements while execution via sqlalchemy

Question

I have a DDL object (create_function_foo) that contains a create function statement. In first line of it I put DROP FUNCTION IF EXISTS foo; but engine.execute(create_function_foo) returns:

sqlalchemy.exc.InterfaceError: (InterfaceError) Use multi=True when executing multiple statements

I put multi=True as parameter for create_engine, engine.execute_options and engine.execute but it doesn't work.

NOTE: engine if my instance of create_engine

NOTE: I'm using python 3.2 + mysql.connector 1.0.12 + sqlalchemy 0.8.2

create_function_foo = DDL("""\
DROP FUNCTION IF EXISTS foo;
CREATE FUNCTION `foo`(
    SID INT
) RETURNS double
READS SQL DATA
BEGIN
  ...
END
""")

Where I should put it?

warvariuc · Accepted Answer · 2013-11-05T07:50:22.723

35

multi=True is a requirement for MySql connector. You can not set this flag passing it to SQLAlchemy methods. Do this:

conn  = session.connection().connection
cursor = conn.cursor()  # get mysql db-api cursor
cursor.execute(sql, multi=True)

More info here: http://www.mail-archive.com/sqlalchemy@googlegroups.com/msg30129.html

edited Nov 05 '13 at 07:50

answered Nov 05 '13 at 07:08

warvariuc

57,116
41
173
227

DonkeyKong · Answer 2 · 2019-08-18T03:03:24.930

6

Yeah... This seems like a bummer to me. I don't want to use the ORM so the accepted answer didn't work for me.

I did this instead:

with open('sql_statements_file.sql') as sql_file:
    for statement in sql_file.read().split(';'):
        if len(statement.strip()) > 0:
             connection.execute(statement + ';')

And then this failed for a CREATE function.... YMMV.

edited Aug 18 '19 at 03:03

answered Aug 18 '19 at 02:57

DonkeyKong

1,005
14
18

7

A more reliable way of splitting the script into statements is to use a SQL parsing library e.g. `sqlparse.split(sqlparse.format(self.sql, strip_comments=True))` – Dzamo Norton Apr 20 '21 at 14:01
1

Although this is a good idea for simple scripts that you can guarantee don't have oddly placed semicolons, there are some common edge cases that make this a bad idea in practice. Note that semicolons in comments or strings (IE `--;` or `';'`) will cause confusion to this setup. I would also recommend using a sql parsing lib. – Jakob Lovern Nov 04 '22 at 21:16
1

@JakobLovern for sure. The idea here was never meant to be more than a solution to a problem. I was using it for DDL; a clean script; if any comments, none with semi-colons. I just needed each query to run, individually. That is all. – DonkeyKong Dec 19 '22 at 20:48

score 4 · Answer 3 · answered Aug 22 '18 at 00:13

There are some cases where SQLAlchemy does not provide a generic way at accessing some DBAPI functions, such as as dealing with multiple result sets. In these cases, you should deal with the raw DBAPI connection directly.

From SQLAlchemy documentation:

connection = engine.raw_connection()
try:
    cursor = connection.cursor()
    cursor.execute("select * from table1; select * from table2")
    results_one = cursor.fetchall()
    cursor.nextset()
    results_two = cursor.fetchall()
    cursor.close()
finally:
    connection.close()

You can also do the same using mysql connector as seen here:

operation = 'SELECT 1; INSERT INTO t1 VALUES (); SELECT 2'
for result in cursor.execute(operation, multi=True):
  if result.with_rows:
    print("Rows produced by statement '{}':".format(
      result.statement))
    print(result.fetchall())
  else:
    print("Number of rows affected by statement '{}': {}".format(
      result.statement, result.rowcount))

sqlalchemy solution doesn't seem to work. One query works, two queries throw ProgrammingError. Has there been a change in sqlalchemy API? — Hamza Zubair, Nov 02 '20 at 18:04

Enable executing multiple statements while execution via sqlalchemy

3 Answers3

Linked