2

Is it possible to configure or customise Alembic to store a datetime in the database to record when each migration was run?

The alembic_version table has a single version_num column, which stores the unique revision identifier of each migration that was run; e.g:

version_num
------------
525d62b9a5fe
23c511df834d
03931f038a4d

I would like to also record when each migration ran in another column; e.g:

version_num   inserted_at
----------------------------------
525d62b9a5fe  2022-08-05 10:22:30
23c511df834d  2022-08-02 15:01:09
03931f038a4d  2022-07-30 09:02:59

This functionality could be implemented with a trigger, but I am curious to know if Alembic can be configured, extended or customised to the same effect.

I'd also like to be able to record the message string that can be included when generating a migration with alembic revision -m "create user table". This is an ideal however, not a deal breaker.

Darragh Enright
  • 13,676
  • 7
  • 41
  • 48
  • 1
    Based on the [table creation code in the source](https://github.com/sqlalchemy/alembic/blob/cfe92fac6794515d3aa3b995e288b11d5c9437fa/alembic/runtime/migration.py#L182-L193) it's not possible to add extra columns via configuration. – Darragh Enright Aug 05 '22 at 11:56
  • 1
    As this is quite good idea, might be better just to create a patch to Alembic directly. – Mikko Ohtamaa Aug 09 '22 at 10:14

1 Answers1

3

You can do this using the env.py and/or the script.py.mako files (see the UPDATE below).

Note : I always had only one row in my alembic_version table. The following solution will be based on this asumption and will create an history table instead of updating the alembic_version table. The mako file can be easily adapted in other cases.

  1. Create the history table
CREATE TABLE alembic_version_history (
    version_num VARCHAR(32),
    inserted_at TIMESTAMP,
    message TEXT
)
  1. Update the alembic/env.py to prevent auto generated migrations to delete the newly created table
# add this function
def include_name(name, type_, parent_names):
    if type_ == "table":
        if name == "alembic_version_history":
            return False
    return True

[...]

# Use it in BOTH run_migrations_offline and run_migration_online functions as follows
context.configure([...], include_name=include_name)

  1. Update the code in the script.py.mako file, specifically the upgrade function :
[...]
def upgrade():
    ${upgrades if upgrades else "pass"}
    op.execute("INSERT INTO alembic_version_history (version_num, inserted_at, message) VALUES ('${up_revision}' ,NOW(), '${message}')")


Generate a migration alembic revision [--autogenerate] -m "test message". Then alembic upgrade head to update alembic_version table and add a new row to alembic_version_history.

UPDATE :

Adding operations to the mako file means that anyone can edit/update/delete those. To avoid this issue, we can again use the env.py:

def update_history(ctx, step, heads, run_args):
    revision_id = step.up_revision_id
    if step.is_upgrade:
        message = step.up_revision.doc
        ctx.connection.execute(
            f"INSERT INTO alembic_version_history (version_num, inserted_at, message) VALUES ('{revision_id}' ,NOW(), '{message}')"
        )
        print("INSERT NEW ROW IN HISTORY")
    else:
        ctx.connection.execute(f"DELETE FROM alembic_version_history where version_num = '{revision_id}'")
        print("DELETE LAST ROW IN HISTORY")

configure.context([...], include_name=include_name, on_version_apply=update_history)
edg
  • 930
  • 6
  • 17
  • Hi Firstly, you're absolutely correct — `alembic_version` only stores the current revision hash. I only realised this after I wrote the question. I'm used to Doctrine and Ecto from PHP and Elixir, both save a historical record of migrations with datetime. – Darragh Enright Aug 08 '22 at 14:00
  • 1
    There has been some discussion (and even a library, now sadly out of date) centred around the idea you suggest — a separate history table. I think this is a good idea. I was hoping it would be possible to hook into an event system in Alembic but your implementation is simple and straightforward which I like a lot, especially since generating a lot of the boilerplate should be possible in the migration template. I'm going to take a lead from your suggestion now and see how I get on. I'll update presently. – Darragh Enright Aug 08 '22 at 14:06
  • One thing jumps out in the meantime, I'm thinking that inserting the history probably should come afterwards to account for migration failure. – Darragh Enright Aug 08 '22 at 14:07
  • 1
    You are right, makes much more sense inserting it after the migration. I edited the code accordingly. – edg Aug 08 '22 at 14:31
  • 1
    Hi @edg — yeah this approach will work really well for my use case. Lateral thinking! And the fact that we can customise the migration template brings a lot of possibilities. I am going to post an answer with some details of my implementation as that might be useful extra information for someone — when I do that I will mark your post as the answer. Credit where it's due! – Darragh Enright Aug 09 '22 at 09:58
  • I updated the solution to only use the `env.py` file, which is IMO safer than using the mako file. – edg Aug 09 '22 at 11:37
  • Okay. This is way better! For silly reasons I didn't think `on_version_apply` was still an available callback — but it is and that's a game changer, and exactly what I want. Thank you! – Darragh Enright Aug 10 '22 at 10:37