5

Is there any way to migrate an existing Airflow instance onto Google Cloud Composer?

We're currently running our own instance of Airflow using postgres for the db. Ideally we'd be able to preserve the existing history of the DAGs, which I believe would require copying the entire database over. Is this possible?

Ayden Wong
  • 61
  • 2

1 Answers1

3

All of the Airflow metadata is kept in the database instance. For PostgreSQL, you could do a pg_dump on your current database followed by a pg_restore into your Composer database in core-sql_alchemy_conn. In the Composer docs, I don't see it specifically mentioned anywhere which database they use, but I'm assuming they're on PostgreSQL as well.

You want to be careful to make sure that your existing database is running the same Airflow version as Composer. Composer is currently on Airflow 1.9.0.

Taylor D. Edmiston
  • 12,088
  • 6
  • 56
  • 76
  • 1
    I would also add that as per [this discussion](https://groups.google.com/forum/#!msg/cloud-composer-discuss/E8spQ3QYE9s/SHDzMTzKAQAJ;context-place=forum/cloud-composer-discuss) you may want to be aware that the DB instances are bonded to an environment and deleting one would also eliminate the metadata. – Michele 'Ubik' De Simoni Jun 18 '18 at 07:20