I've configured a pipeline for maven/java/spring project using a local gitlab-runner in my Mac. My project is hosted on gitlab.com. At every commit, it always takes forever, because it downloads all the dependencies.
Using the shared gitlab-runner the build and test is faster, but it uses my quota minutes. How can I improve it using local gitlab-runner? I can't wait 1 hour at every run.. Thanks in advance!
Preparing the "docker" executor
00:22
Using Docker executor with image maven:3.8.5-openjdk-17-slim ...
Starting service docker:dind ...
Pulling docker image docker:dind ...
Using docker image sha256:7b7c9f4ae103441d225d2b3ea4a5dd0f8cb5673b44ec984353853b2791099bde for docker:dind with digest docker@sha256:999fc127a51b8a86593ff9ba2518f14cbd18555849f8927fd56fa82395effe16 ...
Waiting for services to be up and running (timeout 30 seconds)...
Pulling docker image maven:3.8.5-openjdk-17-slim ...
Using docker image sha256:7337649a0aedd0916ce8bc19a3e1d58779afec34e06ec893fec37fde07661d94 for maven:3.8.5-openjdk-17-slim with digest maven@sha256:502e781d39f0b40fbd02eb23f5b7663618b76ba52034da218c64e92f6c5647be ...
Preparing environment
00:00
Running on runner-v5bnsne3-project-37118324-concurrent-0 via giuseppefalco-QT0Q7VN4KR...
Getting source from Git repository
00:02
Fetching changes with git depth set to 20...
Reinitialized existing Git repository in /builds/gfalco77/maverick/.git/
Checking out 82e8826f as master...
Removing .m2/
Removing .sonar/
Removing target/
Skipping Git submodules setup
Restoring cache
00:00
Checking cache for master-1-protected...
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
Executing "step_script" stage of the job script
Using docker image sha256:7337649a0aedd0916ce8bc19a3e1d58779afec34e06ec893fec37fde07661d94 for maven:3.8.5-openjdk-17-slim with digest maven@sha256:502e781d39f0b40fbd02eb23f5b7663618b76ba52034da218c64e92f6c5647be ...
$ echo "Building $MODULE"
Building
$ mvn $MAVEN_CLI_OPTS -T 4 compile
Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
Maven home: /usr/share/maven
Java version: 17.0.2, vendor: Oracle Corporation, runtime: /usr/local/openjdk-17
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.10.124-linuxkit", arch: "aarch64", family: "unix"
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO] Downloading from spring-milestones: https://repo.spring.io/milestone/com/gfs/phoenix/1.32/phoenix-1.32.pom
[INFO] Downloading from gitlab-maven: https://gitlab.com/api/v4/projects/37118324/packages/maven/com/gfs/phoenix/1.32/phoenix-1.32.pom
At the end
[INFO] Compiling 18 source files to /builds/gfalco77/maverick/target/classes
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:09 min (Wall Clock)
[INFO] Finished at: 2022-10-16T10:37:49Z
[INFO] ------------------------------------------------------------------------
Saving cache for successful job
00:02
Creating cache master-1-protected...
.m2/repository/: found 4492 matching files and directories
No URL provided, cache will not be uploaded to shared cache server. Cache will be stored only locally.
Created cache
Cleaning up project directory and file based variables
00:00
Job succeeded
Same for the other jobs , release, test - docker push - deploy
My gitlab-pipeline is (just showing build and test jobs)
variables:
MAVEN_OPTS: "-Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository"
MAVEN_CLI_OPTS: "-s settings.xml --batch-mode --errors --fail-at-end --show-version"
MAVEN_IMAGE: maven:3.8.5-openjdk-17-slim
IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
DOCKER_HOST: "tcp://docker:2375"
DOCKER_DRIVER: "overlay2"
DOCKER_TLS_CERTDIR: ""
TESTCONTAINERS_RYUK_DISABLED: "true"
services:
- docker:dind
stages:
- build
- test
- release
- docker-build
- deploy
cache:
paths:
- .m2/repository/
key: "$CI_BUILD_REF_NAME"
build-job:
image: $MAVEN_IMAGE
stage: build
tags:
- local-runner-maverick
script:
- echo "Building $MODULE"
- mvn $MAVEN_CLI_OPTS -T 4 compile
only:
- merge_request
- master
test-job:
image: $MAVEN_IMAGE
stage: test
tags:
- local-runner-maverick
variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task
cache:
key: "${CI_JOB_NAME}"
paths:
- .sonar/cache
script:
- echo "Testing $MODULE"
- mvn $MAVEN_CLI_OPTS verify -e sonar:sonar -Dsonar.qualitygate.wait=true -Dsonar.projectKey=gfalco77_maverick
allow_failure: false
only:
- merge_requests
- master
I'm using JIB to build the image and this is the maven plugin configuration
<plugin>
<groupId>com.google.cloud.tools</groupId>
<artifactId>jib-maven-plugin</artifactId>
<version>${google.jib.version}</version>
<configuration>
<container>
<creationTime>USE_CURRENT_TIMESTAMP</creationTime>
</container>
<from>
<image>openjdk:17-jdk-slim-buster</image>
</from>
</configuration>
<executions>
<execution>
<id>Skip-build-on-release</id>
<phase>none</phase>
</execution>
</executions>
</plugin>
Finally this is the config.toml
[[runners]]
name = "local-runner-maverick"
url = "https://gitlab.com/"
token = "..."
executor = "docker"
environment = ["DOCKER_DRIVER=overlay2"]
[runners.custom_build_dir]
[runners.cache]
[runners.cache.s3]
[runners.cache.gcs]
[runners.cache.azure]
[runners.docker]
tls_verify = false
image = "maven:3.8.5-openjdk-17-slim"
privileged = true
disable_entrypoint_overwrite = false
oom_kill_disable = false
disable_cache = false
volumes = ["/certs/client", "/cache", "/.m2"]
shm_size = 0