22

When I run maven flyway:migrate, I get the error

Failed to execute goal org.flywaydb:flyway-maven-plugin:6.5.5:migrate (default-cli) on project myProject: org.flywaydb.core.api.FlywayException: Unable to connect to the database. Configure the url, user and password!

I have my Spring Boot settings in my application.yml file, but I guess the error means it doesn't detect the database config. This documention says, "Spring Boot will then automatically autowire Flyway with its DataSource and invoke it on startup." If I add the configuration to my pom.xml in the flyway plugin section, it connects to the database successfully, but I want it to use my application.yml config. Not the pom.xml. So what am I doing wrong?

Link to repo with issue: https://github.com/jack-cole/BrokenSpringBoot

application.yml

spring:
    datasource:
        driverClassName: org.postgresql.Driver
        url: "jdbc:postgresql://localhost:5433/myDB"
        username: postgres
        password: test123

Dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jooq</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.2.16</version>
</dependency>
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-core</artifactId>
    <version>7.0.0</version>
</dependency>

Plugins:

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
<plugin>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
    <version>6.5.5</version>
</plugin>
Jack Cole
  • 1,528
  • 2
  • 19
  • 41

5 Answers5

20

Run With maven:

Failed to execute goal org.flywaydb:flyway-maven-plugin:6.5.5:migrate (default-cli) on project myProject: org.flywaydb.core.api.FlywayException: Unable to connect to the database. Configure the url, user and password!

You can configure url, user and password in flyway-maven-plugin configuration see First Steps Maven

<plugin>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
    <version>7.0.0</version>
    <configuration>
        <url>jdbc:postgresql://localhost:5433/myDB</url>
        <user>postgres</user>
        <password>test123</password>
    </configuration>
</plugin>

or with environment variables:

mvn flyway:migrate -Dflyway.url=jdbc:postgresql://localhost:5433/myDB -Dflyway.user=postgres -Dflyway.password=test123

More approaches in https://www.baeldung.com/database-migrations-with-flyway

Run with spring-boot:

Spring Boot auto configure and trigger Flyway at the application startup when you include the Flyway core library into the project. See usage of @ConditionalOnClass(Flyway.class) in FlywayAutoConfiguration :

@Configuration(proxyBeanMethods = false)
@ConditionalOnClass(Flyway.class)
@Conditional(FlywayDataSourceCondition.class)
@ConditionalOnProperty(prefix = "spring.flyway", name = "enabled", matchIfMissing = true)
@AutoConfigureAfter({ DataSourceAutoConfiguration.class, JdbcTemplateAutoConfiguration.class,
      HibernateJpaAutoConfiguration.class })
@Import({ FlywayEntityManagerFactoryDependsOnPostProcessor.class, FlywayJdbcOperationsDependsOnPostProcessor.class,
      FlywayNamedParameterJdbcOperationsDependencyConfiguration.class })
public class FlywayAutoConfiguration {
    ...
}

Use mvn spring-boot:run or java -jar app.jar to run the application

NB : Check also that migration scripts are in db/migration otherwise provide the locations with spring.flyway.locations property

Resources:

https://flywaydb.org/documentation/configuration/parameters/

https://flywaydb.org/documentation/getstarted/firststeps/maven/

https://docs.spring.io/spring-boot/docs/current/reference/html/howto.html#howto-execute-flyway-database-migrations-on-startup

Issam El-atif
  • 2,366
  • 2
  • 17
  • 22
  • 1
    This doesn't work when deploying the jar though, which is why I need it to use the application.yml file. – Jack Cole Oct 07 '20 at 05:27
  • Ah ok. I see now what you want to achieve. I edited my answer. You have also to put migration scripts in db/migration directory or if it differs provide a location with spring.flyway.locations property – Issam El-atif Oct 07 '20 at 11:56
  • I can run flyway fine if I put the config in pom.xml, but it doesn't run on boot with `mvn spring-boot:run`. I put breakpoints in all methods in `FlywayAutoConfiguration ` to see if I could glean anything, but it did not pause on any of them. – Jack Cole Oct 07 '20 at 16:28
  • Try removing postgresql and flyway versions in pom.xml and let spring-boot picks the versions – Issam El-atif Oct 07 '20 at 22:24
  • That didn't seem to work. I uploaded the repo here: https://github.com/jack-cole/BrokenSpringBoot – Jack Cole Oct 08 '20 at 07:02
  • Got this logs running your repo 2020-10-08 10:44:43.058 DEBUG 8452 --- [ main] org.flywaydb.core.Flyway : DDL Transactions Supported: true 2020-10-08 10:44:43.059 DEBUG 8452 --- [ main] org.flywaydb.core.Flyway : Schemas: 2020-10-08 10:44:43.059 DEBUG 8452 --- [ main] org.flywaydb.core.Flyway : Default schema: null – Issam El-atif Oct 08 '20 at 08:48
  • Filtering out resource: db/migration/V1_1_0__Init.sql (filename: V1_1_0__Init.sql) ..... 2020-10-08 10:44:43.110 INFO 8452 --- [ main] o.f.core.internal.command.DbValidate : Successfully validated 1 migration (execution time 00:00.034s) 2020-10-08 10:44:43.119 INFO 8452 --- [ main] o.f.core.internal.command.DbMigrate : Current version of schema "public": 1.1.0 2020-10-08 10:44:43.120 INFO 8452 --- [ main] o.f.core.internal.command.DbMigrate : Schema "public" is up to date. No migration necessary. – Issam El-atif Oct 08 '20 at 08:49
  • Got this logs running app in Intellij but jooq error "Cannot execute query. No Connection configured" when running with mvn spring-boot:run – Issam El-atif Oct 08 '20 at 09:07
  • I got it to work with mvn spring-boot:run see https://github.com/jack-cole/BrokenSpringBoot/pull/1 – Issam El-atif Oct 08 '20 at 09:41
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/222729/discussion-between-issam-el-atif-and-jack-cole). – Issam El-atif Oct 08 '20 at 16:06
  • From the looks of it, Flyway MUST be configured outside of application.yml. Since flyway built into spring runs at spring startup. So essentially have to run flyway twice. Once before compiling (due to jooq), and once before spring starts up. I saw some maven plugin where you can read application.yml and fill in variables from there, but that's probably too much effort to set up for the little payoff. – Jack Cole Oct 12 '20 at 01:11
9

You quoted a part of spring boot docs, but you launched your migration not by spring boot, but as a maven task.

Flyway maven plugin is not aware of spring boot configuration, it only takes the following sources into account: Overriding order

  1. System properties
  2. Environment variables
  3. Custom config files
  4. Maven properties
  5. Plugin configuration section
  6. Credentials from settings.xml
  7. /flyway.conf
  8. Flyway Maven plugin defaults

On my PC, I used the environment variables approach - I have the same environment variables defined for build plugin and for spring boot.

Lesiak
  • 22,088
  • 2
  • 41
  • 65
  • How would I launch a flyway through spring boot? – Jack Cole Oct 06 '20 at 07:27
  • 1
    This is precisely described in the link you posted: https://flywaydb.org/documentation/plugins/springboot. With `flyway-core` dependency and DataSource defined in your `application.properties`, any pending migrations will be executed on application startup (using application credentials). If you want to use this feature, you need to give the application the rights to change the schema. – Lesiak Oct 06 '20 at 07:55
  • 2
    Flyway should run when you start Spring Boot, since it is autoconfigured. Use `mvn spring-boot:run` to start the Spring Boot App from Maven. Spring Boot has further properties for Flyway configuration: https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-application-properties.html#data-migration-properties – Sebastian Oct 06 '20 at 07:57
  • When I run `mvn spring-boot:run`, flyway does not appear to run (does not update database), and there is nothing logged in the output about flyway when I execute that command. – Jack Cole Oct 07 '20 at 05:25
  • 1
    1. Make sure you have log output from `org.flywaydb` package turned on. 2. Check if appropriate objects are being constructed (like breakpoint in DbValidate constructor). 3. This may be obvious: @Sebastian suggested launching the app via maven command, but you may prefer to launch if from IDE during development. 4. Spring Boot relies on auto-configuration to conditionally create beans. In our case, the auto-config class if `FlywayAutoConfiguration`, check if all conditions are positive (like: do you have a DataSource bean present etc) – Lesiak Oct 07 '20 at 07:07
  • 1. Logging is set to TRACE but nothing appears in console with the word "flyway" on startup. 2. Breakpoints in the methods of DBValidate were not reached, so it's not being ran. 3. Ran through the command line after downloading maven, instead of IDE, but still exact same issue. 4. Put breakpoints in every method of FlywayAutoConfiguration and none were reached. – Jack Cole Oct 07 '20 at 16:45
  • How do you define your DataSource bean? Manually? (please show us relevant snippet) Via auto-configuration and connection pool properties? Again, please show us the config. (https://docs.spring.io/spring-boot/docs/1.4.0.M1/reference/html/boot-features-sql.html#boot-features-connect-to-production-database) – Lesiak Oct 08 '20 at 06:06
  • Here is a link to the repo with the issue https://github.com/jack-cole/BrokenSpringBoot – Jack Cole Oct 08 '20 at 06:59
  • @JackCole 1. This has no code, only resources 2. have you checked the DataSource bean? – Lesiak Oct 08 '20 at 07:03
  • Do I need a datasource bean if I have the datasource in application.yml? – Jack Cole Oct 08 '20 at 07:14
  • I added that to the properties and nothing changed. Are you able to get the application running on your system? – Jack Cole Oct 08 '20 at 07:29
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/222701/discussion-between-lesiak-and-jack-cole). – Lesiak Oct 08 '20 at 07:45
  • So in conclusion, I have to use flyway as a regular plugin outside spring boot, since I use jooq. And I also need spring boot setup to use flyway at startup. It's all working and fixed now. But my configuration comes from maven for the first flyway migrate, it's not dynamically based on application.yml. There are maven plugins to read it, so I don't need to have the same info in two places, but I think it's too much effort to do. – Jack Cole Oct 12 '20 at 01:14
0

If you want Flyway to be executed when launching your app (with spring-boot), just remove the flyway-maven-plugin plugin, you don't need it.

-- EDIT

Btw, if you run maven flyway:migrate, you will indeed need to have the credentials set on the plugin (it isn't accessing the java resources).

  • So either you need to run this task => you have to set the credentials on in the plugin on pom.xml.
  • Or you don't need it, so you just remove the plugin and set the credentials on the application.yaml file for your application startup
LE GALL Benoît
  • 7,159
  • 1
  • 36
  • 48
  • This did not change anything unfortunately. I added my repo to the original post to see if you or others can try building it. – Jack Cole Oct 08 '20 at 07:31
  • OK, I'll have a look directly – LE GALL Benoît Oct 08 '20 at 08:20
  • I made a branch `fix` which works without the `flyway-maven-plugin` plugin You'll also see some dependencies that are commented on your `pom.xml`, as I don't know them and they seem to required some configuration to works. Try first like this branch, and step by step, add your required dependencies – LE GALL Benoît Oct 08 '20 at 08:38
  • can't push the `fix` branch on your repository :/ – LE GALL Benoît Oct 08 '20 at 09:13
0

Have you tried?

application.yml

spring:
    flyway:
        driverClassName: org.postgresql.Driver
        url: "jdbc:postgresql://localhost:5433/myDB"
        user: postgres
        password: test123
        enabled: true
reflexdemon
  • 836
  • 8
  • 21
0

Your configuration has a few missed moments:

  • you use too many spaces for Spring Boot's .yml configuration file. Every next level should contain 2 more spaces, like:
spring:
  datasource:
    url: jdbc:postgresql://localhost/myDB
    username: postgres
    password: root

No more. Otherwise, it would not work!

If you using IntelliJ Idea you could use autoformat shortcut for formatting .yml: Ctrl + Alt + L.

Or you could use the same style as for .properties:

spring.datasource.url: ...
spring.datasource.username: ...

More redundant. However, less chance to make it incorrect.

  • you are using Spring Boot. Thus, you don't need to include the version directly.
    Much better will be left to Spring's dependency management:
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-core</artifactId>
</dependency>

And

<plugin>
    <groupId>org.flywaydb</groupId>
    <artifactId>flyway-maven-plugin</artifactId>
</plugin>

For most cases, you could be sure that such a combination of versions will work fine.

  • you should put your migration file under resources/db/migration. Also naming is very important. The first file should be something like:

V1__Init_Db.sql

By default exactly two underscores after V1 -> V1__ !

Here more info: Migrations information.

  • Also, check if you need additional values for your config file, like:
spring.jpa.show-sql: true
spring.jpa.hibernate.ddl-auto: create
catch23
  • 17,519
  • 42
  • 144
  • 217
  • @JackCole have you tried something for solving your issue? – catch23 Oct 11 '20 at 09:58
  • So I was able to resolve the issue. Flyway must be ran twice: Once before compile, and once when starting the server. To run it before starting the server, you must define a config in the pom.xml. The one that runs before the server starts is a spring-boot plugin that uses application.yml. – Jack Cole Oct 12 '20 at 03:45