3

I know multiple databases work on flask-sqlalchemy with __bind_key__. But I don't know how to migrate those databases on alembic(flask-migrate). Here's env.py:

from flask import current_app
config.set_main_option('sqlalchemy.url', current_app.config.get('SQLALCHEMY_BINDS')['bind_main'])

target_metadata = {
    'bind_main': current_app.extensions['migrate'].db.metadata,
    'bind_follower': current_app.extensions['migrate'].db.metadata,
}

How can I set follower db on target_metadata? flask-migrate doesn't care about bind databases. Thanks.

s2t2
  • 2,462
  • 5
  • 37
  • 47
Jony cruse
  • 815
  • 1
  • 13
  • 23

2 Answers2

4

To create a multiple database migration repository, add the --multidb argument to the init command:

$ python app.py db init --multidb

For more details please refer to flask-migrate documentation

Darshan Patel
  • 515
  • 1
  • 8
  • 14
0

It may be easier to scrap the old "migrations" folder and initialize the database again with the new setup, applying any customizations to the fresh directory. Most likely this would include modifying env.py or script.py.mako.

Diff init vs init --multidb

I ran a diff of the results for flask db init (i.e. single.migrations) vs flask db init --multidb (i.e. multi.migrations):

diff --suppress-common-lines single.migrations/README multi.migrations/README
1c1
< Single-database configuration for Flask.
---
> Multi-database configuration for Flask.
diff --suppress-common-lines single.migrations/env.py multi.migrations/env.py
5a6
> from sqlalchemy import MetaData
9a11,12
> USE_TWOPHASE = False
> 
26a30,42
> bind_names = []
> if current_app.config.get('SQLALCHEMY_BINDS') is not None:
>     bind_names = list(current_app.config['SQLALCHEMY_BINDS'].keys())
> else:
>     get_bind_names = getattr(current_app.extensions['migrate'].db,
>                              'bind_names', None)
>     if get_bind_names:
>         bind_names = get_bind_names()
> for bind in bind_names:
>     context.config.set_section_option(
>         bind, "sqlalchemy.url",
>         str(current_app.extensions['migrate'].db.get_engine(
>             bind=bind).url).replace('%', '%%'))
28a45
> 
34a52,62
> def get_metadata(bind):
>     """Return the metadata for a bind."""
>     if bind == '':
>         bind = None
>     m = MetaData()
>     for t in target_metadata.tables.values():
>         if t.info.get('bind_key') == bind:
>             t.tometadata(m)
>     return m
> 
> 
47,50c75,76
<     url = config.get_main_option("sqlalchemy.url")
<     context.configure(
<         url=url, target_metadata=target_metadata, literal_binds=True
<     )
---
>     # for the --sql use case, run migrations for each URL into
>     # individual files.
52,53c78,99
<     with context.begin_transaction():
<         context.run_migrations()
---
>     engines = {
>         '': {
>             'url': context.config.get_main_option('sqlalchemy.url')
>         }
>     }
>     for name in bind_names:
>         engines[name] = rec = {}
>         rec['url'] = context.config.get_section_option(name, "sqlalchemy.url")
> 
>     for name, rec in engines.items():
>         logger.info("Migrating database %s" % (name or '<default>'))
>         file_ = "%s.sql" % name
>         logger.info("Writing output to %s" % file_)
>         with open(file_, 'w') as buffer:
>             context.configure(
>                 url=rec['url'],
>                 output_buffer=buffer,
>                 target_metadata=get_metadata(name),
>                 literal_binds=True,
>             )
>             with context.begin_transaction():
>                 context.run_migrations(engine_name=name)
70,85c116,169
<             if script.upgrade_ops.is_empty():
<                 directives[:] = []
<                 logger.info('No changes in schema detected.')
< 
<     connectable = current_app.extensions['migrate'].db.get_engine()
< 
<     with connectable.connect() as connection:
<         context.configure(
<             connection=connection,
<             target_metadata=target_metadata,
<             process_revision_directives=process_revision_directives,
<             **current_app.extensions['migrate'].configure_args
<         )
< 
<         with context.begin_transaction():
<             context.run_migrations()
---
>             if len(script.upgrade_ops_list) >= len(bind_names) + 1:
>                 empty = True
>                 for upgrade_ops in script.upgrade_ops_list:
>                     if not upgrade_ops.is_empty():
>                         empty = False
>                 if empty:
>                     directives[:] = []
>                     logger.info('No changes in schema detected.')
> 
>     # for the direct-to-DB use case, start a transaction on all
>     # engines, then run all migrations, then commit all transactions.
>     engines = {
>         '': {'engine': current_app.extensions['migrate'].db.get_engine()}
>     }
>     for name in bind_names:
>         engines[name] = rec = {}
>         rec['engine'] = current_app.extensions['migrate'].db.get_engine(
>             bind=name)
> 
>     for name, rec in engines.items():
>         engine = rec['engine']
>         rec['connection'] = conn = engine.connect()
> 
>         if USE_TWOPHASE:
>             rec['transaction'] = conn.begin_twophase()
>         else:
>             rec['transaction'] = conn.begin()
> 
>     try:
>         for name, rec in engines.items():
>             logger.info("Migrating database %s" % (name or '<default>'))
>             context.configure(
>                 connection=rec['connection'],
>                 upgrade_token="%s_upgrades" % name,
>                 downgrade_token="%s_downgrades" % name,
>                 target_metadata=get_metadata(name),
>                 process_revision_directives=process_revision_directives,
>                 **current_app.extensions['migrate'].configure_args
>             )
>             context.run_migrations(engine_name=name)
> 
>         if USE_TWOPHASE:
>             for rec in engines.values():
>                 rec['transaction'].prepare()
> 
>         for rec in engines.values():
>             rec['transaction'].commit()
>     except:  # noqa: E722
>         for rec in engines.values():
>             rec['transaction'].rollback()
>         raise
>     finally:
>         for rec in engines.values():
>             rec['connection'].close()
diff --suppress-common-lines single.migrations/script.py.mako multi.migrations/script.py.mako
1c1,4
< """${message}
---
> <%!
> import re
> 
> %>"""${message}
19,20c22,48
< def upgrade():
<     ${upgrades if upgrades else "pass"}
---
> def upgrade(engine_name):
>     globals()["upgrade_%s" % engine_name]()
> 
> 
> def downgrade(engine_name):
>     globals()["downgrade_%s" % engine_name]()
> 
> <%
>     from flask import current_app
>     bind_names = []
>     if current_app.config.get('SQLALCHEMY_BINDS') is not None:
>         bind_names = list(current_app.config['SQLALCHEMY_BINDS'].keys())
>     else:
>         get_bind_names = getattr(current_app.extensions['migrate'].db, 'bind_names', None)
>         if get_bind_names:
>             bind_names = get_bind_names()
>     db_names = [''] + bind_names
> %>
> 
> ## generate an "upgrade_<xyz>() / downgrade_<xyz>()" function
> ## for each database name in the ini file.
> 
> % for db_name in db_names:
> 
> def upgrade_${db_name}():
>     ${context.get("%s_upgrades" % db_name, "pass")}
> 
21a50,51
> def downgrade_${db_name}():
>     ${context.get("%s_downgrades" % db_name, "pass")}
23,24c53
< def downgrade():
<     ${downgrades if downgrades else "pass"}
---
> % endfor
Common subdirectories: single.migrations/versions and multi.migrations/versions

Note: lines prefixed with < are in the single and lines prefixed with > are found in the multiple. StackOverflow isn't the best medium to display the diff. Alternative diff -y --color=always (or another diff tool) is much easier to view.

For summary it's a few lines were removed in env.py and new lines were added to env.py script.py.mako to accommodate bind keys.

Specify New Default

To start with a new directory and retain the old in the codebase for comparison, specify the directory during the initialization:

flask init db --multidb --directory"multi.migrations"

The new directory can be specified in the program's Migrate constructor:

migrate = Migrate(directory="multi.migrations")

Likewise, in python interactive, a directory argument can be passed to the API calls:

show(directory='migrations', revision='head')
Mike
  • 1,279
  • 7
  • 18