redefine local liquibase properies that are still visible in included files

Question

Some of my tables implement concepts that need certain columns. I want to avoid having to copy the changesets for those columns to all relevant tables and would instead rather reuse a template to create the actual changesets.

Based on different ideas I found on the net I tried this:

db/templates/0004-create-validity-period-data-template.xml

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
                            http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">

    <include file="../templates/0002-create-versioned-data-template.xml" relativeToChangelogFile="true" />

    <changeSet author="${table.author}" id="Create ValidityPeriodData for ${table.name}" >
        <comment>Create columns of super class ValidityPeriodData for table ${table.name}</comment>
        <addColumn tableName="${table.name}" >
            <column name="valid_from" type="date" remarks="the earliest point in time when this entry is valid" />
            <column name="valid_until" type="date" remarks="the last point in time when this entry is valid" />
        </addColumn>
    </changeSet>

</databaseChangeLog>

db/changelog/0017-create-credential-table.xml

<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog
        xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog
                            http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">

    <property name="table.name" value="login" />
    <property name="table.author" value="guess ;-)" />

    <include file="../templates/0004-create-validity-period-data-template.xml" relativeToChangelogFile="true" />

    <changeSet author="${table.author}" id="Create Credential table">
        <addColumn tableName="${table.name}">
            <column name="login_id" type="${uuid_type}"/>
            <column name="password" type="VARCHAR(255)">
                <constraints nullable="false"/>
            </column>
            <column name="salt" type="VARCHAR(128)"/>
        </addColumn>
        <addForeignKeyConstraint baseColumnNames="login_id"
                                 baseTableName="${table.name}"
                                 constraintName="fk_credential_of_login"
                                 onDelete="RESTRICT"
                                 onUpdate="RESTRICT"
                                 referencedColumnNames="id"
                                 referencedTableName="login"/>
    </changeSet>

</databaseChangeLog>

The Problem

The properties (table.name, table.author) are by default global="true", which means they are visible in all changeset files, but they can not be overwritten/redefined. So this only works for a single table, which defeats the purpose. If I declare them as global="false" I can redefine them for every table that needs those columns, however now the included files do not see those properties.

The Question

Is there a way to redefine or overwrite properties for different changeset files (like what global="false" does) but still have them behave like global="true" in included files even recursively.

Holly · Accepted Answer · 2020-12-12T13:12:43.093

Background

I had a look at the source code of liquibase. It turns out the current implementation (in versions 3.6.2, 3.6.3, 3.8.0) works a bit strange.

private ChangeLogParameter findParameter(String key, DatabaseChangeLog changeLog) {
    ChangeLogParameter result = null;

    List<ChangeLogParameter> found = new ArrayList<>();
    for (ChangeLogParameter param : changeLogParameters) {
        if (param.getKey().equalsIgnoreCase(key) && param.isValid()) {
            found.add(param);
        }
    }
    
    if (found.size() == 1) {
        // this case is typically a global param, but could also be a unique non-global param in one specific
        // changelog
        result = found.get(0);
    } else if (found.size() > 1) {
        for (ChangeLogParameter changeLogParameter : found) {
            if (changeLogParameter.getChangeLog().equals(changeLog)) {
                result = changeLogParameter;
            }
        }
    }
    
    return result;
}

You also need to know that expressions get expanded as soon as a changeSet is parsed. Parameters defined in changeSets which have not been read yet, can not be considered.

So the first time a parameter is encountered, it is in effect treated, as if it was global=true no matter if that's the case or not.

A second global parameter with the same name will not be recorded. However a second local parameter will be recorded.

Once the parameter list contains a second ChangeLogParameter with the same name, those will be expanded differently. Now it looks for the last defined version of that parameter defined in the same changeset file - but again not matter if it global or local.

Answer

So to answer my own question. The current implementation of liquibase does not support that kind of thing. On top of that it behaves somewhat inconsistent in regards to expanding expressions.

Solution?

I filed a bug report with liquibase CORE-3493 and provided a possible solution on GitHub.

Edit: This change has been incorporated in version 3.10.2. However be aware that I unknowingly introduced a breaking change issue #1293. This does not matter for new projects, but may break existing projects if variables have been used already. A workaround for that is suggested in the issue.

score 0 · Answer 2 · answered Oct 17 '19 at 16:26

0

I think you have a misconception about liquibase changeSets usage.

ChangeSet should be atomic and follow "one action - one changeSet" concept. It's not supposed to be some kind of "reusable function". Such approach may damage your database schema. For that reason, you don't delete old changeSets

So I suggest you write changeSets for each and every table you need to add columns to.

Also, take a look at this article: Trimming ChangeLog Files

answered Oct 17 '19 at 16:26

htshame

6,599
5
36
56

I am writing changeSets for every table (if you really look at my examble, you will see that), but just as I put common stuff into abstract super classes in JAVA, I would like to put to put the creation of those common parts of a table in a "superChangeSet", copying the same code, the same changeSet over and over is way more error prone in my book – Holly Oct 17 '19 at 18:01
the template does create an atomic changeSet for each table it is used in, each of those has its own ID based on the table.name property value, I see nothing that breaks the basic concept of liquibase – Holly Oct 17 '19 at 18:05
Yes, I get your idea, but I don't think feasible with liquibase. At least, it's beyond my expertise. – htshame Oct 18 '19 at 10:00

redefine local liquibase properies that are still visible in included files

2 Answers2