TeamCity upgrade fails with "Can't take exclusive lock when db lock is not held"

Question

When upgrading to TeamCity 2023.05 (using the Docker image), I received the error "Can't take exclusive lock when db lock is not held" and the detailed instruction:


<pre>java.lang.IllegalStateException: Can't take an exclusive lock when db lock is not held
    at jetbrains.buildServer.serverSide.db.Heartbeat.tryGetExclusiveLock(Heartbeat.java:726)
    at jetbrains.buildServer.serverSide.db.TeamCityDatabaseManager.tryGetExclusiveLock(TeamCityDatabaseManager.java:857)
    at jetbrains.buildServer.maintenance.StartupProcessor.lockDatabaseExclusively(StartupProcessor.java:860)
    at jetbrains.buildServer.maintenance.StartupProcessor.doUpgradeRequired(StartupProcessor.java:773)
    at jetbrains.buildServer.maintenance.StartupProcessor.checkUpgradeRequired(StartupProcessor.java:826)
    at jetbrains.buildServer.maintenance.StartupProcessor.doDatabaseChecking(StartupProcessor.java:1377)
    at jetbrains.buildServer.maintenance.StartupProcessor.access$1000(StartupProcessor.java:1410)
    at jetbrains.buildServer.maintenance.StartupProcessor$3.call(StartupProcessor.java:12)
    at jetbrains.buildServer.util.NamedThreadFactory.executeWithNewThreadName(NamedThreadFactory.java:91)
    at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStage(StartupProcessor.java:779)
    at jetbrains.buildServer.maintenance.StartupProcessor.processConcreteStageSafe(StartupProcessor.java:1165)
    at jetbrains.buildServer.maintenance.StartupProcessor.processTeamCityLifecycle(StartupProcessor.java:1100)
    at jetbrains.buildServer.maintenance.StartupProcessor.access$000(StartupProcessor.java:1120)
    at jetbrains.buildServer.maintenance.StartupProcessor$1.run(StartupProcessor.java:3)
    at java.base/java.lang.Thread.run(Thread.java:833)
</pre>

I spent a lot of time trying to get to the bottom of this, as I initially thought it was something wrong with the actual database, but could not find any locks.

score 1 · Accepted Answer · answered Jun 23 '23 at 11:00

The solution was to drop the records in the db_heartbeat table in the teamcity database.

Check if there are any records:

select * from db_heartbeat;
[...]

Remove all records:

TRUNCATE TABLE db_heartbeat;

Verify that they were dropped:

select * from db_heartbeat;
[...]

Once done, you should be able to restart TeamCity and redo the migration.

TeamCity upgrade fails with "Can't take exclusive lock when db lock is not held"

1 Answers1