3

I'd like to connect to a second external database during my migration to move some of its data into my local database. What's the best way to do this?

Once the second db has been added to the alembic context (which I am not sure of how to do), how can run SQL statements on the db during my migration?

This is what my env.py looks like right now:

from alembic import context
from sqlalchemy import engine_from_config, pool
from logging.config import fileConfig
from migration_settings import database_url
import models

# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config

# Interpret the config file for Python logging.
# This line sets up loggers basically.
fileConfig(config.config_file_name)

# add your model's MetaData object here
# for 'autogenerate' support
target_metadata = models.Base.metadata

# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.


def run_migrations_offline():
    """Run migrations in 'offline' mode.
    This configures the context with just a URL
    and not an Engine, though an Engine is acceptable
    here as well.  By skipping the Engine creation
    we don't even need a DBAPI to be available.
    Calls to context.execute() here emit the given string to the
    script output.
    """
    url = database_url or config.get_main_option("sqlalchemy.url")
    context.configure(url=url, target_metadata=target_metadata, literal_binds=True, version_table_schema='my_schema')

    with context.begin_transaction():
        context.run_migrations()


def run_migrations_online():
    """Run migrations in 'online' mode.
    In this scenario we need to create an Engine
    and associate a connection with the context.
    """
    config_overrides = {'url': database_url} if database_url is not None else {}
    connectable = engine_from_config(
        config.get_section(config.config_ini_section),
        prefix='sqlalchemy.',
        poolclass=pool.NullPool, **config_overrides)

    with connectable.connect() as connection:
        context.configure(
            connection=connection,
            target_metadata=target_metadata,
            version_table_schema='my_schema'
        )

        connection.execute('CREATE SCHEMA IF NOT EXISTS my_schema')

        with context.begin_transaction():
            context.run_migrations()


if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()
jgozal
  • 1,480
  • 6
  • 22
  • 43
  • If this operation is needed in only one migration, I would suggest simply creating a new connection *inside* the migration script itself and working with it. – exhuma Sep 05 '20 at 12:15
  • 3
    @exhuma I was doing that and the migration was hanging for some reason, so I assumed (incorrectly) that this had to do with the fact that the new connection wasn't in alembic's context. Fast forward 10 hours of troubleshooting, this issue was related to my SSH tunnel (the one I was using to talk to the second db) not closing correctly https://github.com/pahaz/sshtunnel/issues/138 – jgozal Sep 05 '20 at 18:12

0 Answers0