gitlab CI/CD with maven doesn't setup environment variables in application.properties

Question

I am trying to build CI/CD pipeline with maven. Problem that I meet is that in application.properties I set variables like that:

database.MongoPass=${MONGO_PASS}
database.Secret=${SECRET}
database.connectionString=${ATLAS_STRING}
spring.data.mongodb.uri=${ATLAS_STRING}

and I cannot setup them it gitlab. Every time if gitlab will build package all time I cannot run it because connection string is wrong I get error: "The connection string is invalid. Connection strings must start with either 'mongodb://' or 'mongodb+srv://"

here example of variable that I set up in gitlab CI/CD settings

and here code that I tried run in gitlab CI/CD echo works correct and show correct variable value each mvn script I tried didn't work

 script:
    - echo $SECRET
    - echo $MONGO_PASS
    - echo $ATLAS_STRING
    - mvn install -B #  (I hope that application properties automatically get variables from gitlab env) 
    - mvn -DSECRET=$SECRET -DMONGO_PASS=$MONGO_PASS -DATLAS_STRING=$ATLAS_STRING clean install -B #  (I found this solution on stack) 
    - mvn -Dspring-boot.run.arguments=--database.Secret=$SECRET,--database.MongoPass=$MONGO_PASS,--spring.data.mongodb.uri=$ATLAS_STRING clean install -B #  (if I change here env variables for normal string it wont't build on gitlab)

I don't have idea what I should do with that I don't want have variables saved in my repo and don't have idea what toDo with that. Could someone give me advice ? mvn script builds jar file in artifacts after each run I download it and run to test it with command

java -jar filename.jar

Update: I made small investigation and make class to test variables after spring startup:

  @PostConstruct
    public void test() {
        log.info("VARIABLES TEST");
        log.info("properties.getSecret(): {}", properties.getSecret());
        log.info("properties.getConnectionString(): {}", properties.getConnectionString());
        log.info("properties.getMongoPass(): {}", properties.getMongoPass());
    }

and variables are all time not set:

properties.getSecret(): ${SECRET}
properties.getConnectionString(): ${ATLAS_STRING}
properties.getMongoPass(): ${MONGO_PASS}

gitlab-ci.yml:

image: maven:3.8.1-jdk-11

build_artifact:
  stage: build
  script:
    - export
#    - mvn install -B -P no-tests
    - mvn -DSECRET=$SECRET -DMONGO_PASS=$MONGO_PASS -DATLAS_STRING=$ATLAS_STRING clean install -B -P no-tests #  (I found this solution on stack)
#    - mvn -Dspring-boot.run.arguments=--database.Secret=$SECRET,--database.MongoPass=$MONGO_PASS,--spring.data.mongodb.uri=$ATLAS_STRING clean install -B -P no-tests #  (if I change here env variables for normal string it wont't build on gitlab)
  artifacts:
    paths:
      - target/*.jar
    expire_in: 10 minutes

Example pipeline result:

Running with gitlab-runner 14.4.0-rc1 (bc99a056)
  on docker-auto-scale ed2dce3a
Preparing the "docker+machine" executor
00:23
Using Docker executor with image maven:3.8.1-jdk-11 ...
Pulling docker image maven:3.8.1-jdk-11 ...
Using docker image sha256:5b508b1fe19e290255c9e077a1c7af028a576cabb70eab4abdfee574599f729f for maven:3.8.1-jdk-11 with digest maven@sha256:aaf506d47cd2ec8f62fc1ff74065eda5614738e8ea61bad9b32da0360b9498cd ...
Preparing environment
00:01
Running on runner-ed2dce3a-project-16772800-concurrent-0 via runner-ed2dce3a-srm-1634103033-dfd4e8e6...
Getting source from Git repository
00:03
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/**/***/.git/
Created fresh repository.
Checking out 60bf3869 as feature/branch
Skipping Git submodules setup
Executing "step_script" stage of the job script
$ mvn -DSECRET=$SECRET -DMONGO_PASS=$MONGO_PASS -DATLAS_STRING=$ATLAS_STRING clean install -B -P no-tests
***
Downloading all dependencies 
***
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:00 min
[INFO] Finished at: 2021-10-13T05:34:25Z
[INFO] ------------------------------------------------------------------------
Uploading artifacts for successful job
00:07
Uploading artifacts...
target/*.jar: found 1 matching files and directories 
Uploading artifacts as "archive" to coordinator... ok  id=1674250996 responseStatus=201 Created token=z2qnoeL8
Cleaning up project directory and file based variables
00:00
Job succeeded

In gitlab, you need to protect your branch in order to able to access the environment variable. Try that — Kumar Ashutosh, Oct 12 '21 at 19:46
Let the first line of your script be `export`. Then you can see what is available. If everything Is as expected gitlab is fine and the problem is elsewhere. — Thorbjørn Ravn Andersen, Oct 12 '21 at 21:24
would you please add your `gitlab-ci.yml` file and the output or screenshot of your pipeline log? — Mostafa Ghadimi, Oct 13 '21 at 04:01
@KumarAshutosh protected branch is needed when variables are protected so in my case I don't need protected branch. Anyway I tried on protected and still same. — Bartosz Kolej, Oct 13 '21 at 05:48
@ThorbjørnRavnAndersen I done it and variables are ok, maven don't want so move them into application — Bartosz Kolej, Oct 13 '21 at 05:50

score 1 · Accepted Answer · answered Oct 12 '21 at 19:37

1

Educated guess: you haven't enabled maven filtering for your application.properties property file.

Without filtering, those placeholders won't be replaced.

So have something like this in your pom file:

<project>
  ...
  <build>
    ...
    <resources>
      <resource>
        <directory>src/main/resources</directory>
        <filtering>true</filtering>
      </resource>
      ...
    </resources>
    ...
  </build>
  ...
</project>

answered Oct 12 '21 at 19:37

eis

51,991
13
150
199

It doesn't help :( – Bartosz Kolej Oct 13 '21 at 05:38
@BartoszKolej did you confirm filtering is enabled and replacing the values during your maven build? – eis Oct 13 '21 at 12:28
if I run maven build on my machine it works perfect build working jar file with correct values, problem is only on gitlab CI same build command but my variablem in application.properties stay all time like: ${ATLAS_STRING} – Bartosz Kolej Oct 13 '21 at 12:48
1

Your solution was correct only thing I had to change is int this post: https://stackoverflow.com/questions/36501017/maven-resource-filtering-not-working-because-of-spring-boot-dependency example: database.MongoPass=${MONGO_PASS} to database.MongoPass=@MONGO_PASS@ then it works for spring boot :) – Bartosz Kolej Oct 13 '21 at 16:30

Mostafa Ghadimi · Answer 2 · 2021-10-17T03:30:28.693

0

Try the following steps and make sure that each one is working properly:

Check whether your branch is protected or not. If it is not protected, you won't be able to access to defined environment variables.
Check whether the environment variables set properly or not using export command:
```
script:
    - export
    # - <other commands>
```
Set filtering to true in your POM file.

Change application.properties to the following:

database.MongoPass=$MONGO_PASS$
database.Secret=$SECRET$
database.connectionString=$ATLAS_STRING$
spring.data.mongodb.uri=$ATLAS_STRING$

edited Oct 17 '21 at 03:30

answered Oct 13 '21 at 04:00

Mostafa Ghadimi

5,883
8
64
102

1 protected branch is needed when variables are protected so in my case I don't need protected branch. Anyway I tried on protected and still same. 2 export shjow me that env variables are correct 3 filtering didn't help. – Bartosz Kolej Oct 13 '21 at 05:49
Filtering would work! @BartoszKolej – Mostafa Ghadimi Oct 17 '21 at 03:28

gitlab CI/CD with maven doesn't setup environment variables in application.properties

2 Answers2