I am working to add Alembic migration to a legacy project. I am hoping to execute some raw sql, including some insert statements with unicode that I want to be encoded with UTF-8, but am getting a UnicodeDecodeError. To reproduce this error, I created this example:
def upgrade():
op.execute("SELECT ''")
When I run this migration, I get:
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1263, in execute
return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 324, in _execute_on_connection
self, multiparams, params, execution_options
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1462, in _execute_clauseelement
cache_hit=cache_hit,
File "virtual_env/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1678, in _execute_context
e, util.text_type(statement), parameters, None, None
UnicodeDecodeError: 'ascii' codec can't decode byte 0xf0 in position 8: ordinal not in range(128)
I confirmed that my Postgres database encoding is set to UTF-8. I also tried to work out whether execution_options parameters exist to set the encoding, but I was not able to discern this. I also tried abandoning op.execute() and tried creating my own engine instance using sa.create_engine(url, encoding='utf-8'), which mysteriously still gave me an encoding error and was wanting to use ascii.
Which Alembic, SQLAlchemy or Psycopg2 subsystem is expecting ascii, and is there a way to change that expectation either in Alembic's configuration or for a specific migration?
I'm happy to dig into the internal APIs and use a hacky solution, but would hope there is a straightforward way to do this that I have just not encountered in a couple hours searching the documentation.
Note: this project uses Python 2.7.16, and despite my efforts I do not have authorization to port it to Python3 yet.
