15

I am new to docker and jenkins. However ultimately I am creating a job in jenkins so that I can delete the volume cache of gitlab-runner which is stored in our linux machines (CentOS7)

To achieve this, I am creating a periodic job every 6 hours with the following command in jenkins:

docker volume prune -f

However it doesn't clean up the space at all. This is the output of jenkins job :

Started by timer
Running as SYSTEM
Building remotely on buildbng17 (gitlab) in workspace /mnt/data0/jenkins/workspace/gitlab-cleanup
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
[gitlab-cleanup] $ /bin/sh -xe /tmp/jenkins3799570002967825583.sh
+ find /mnt/data0/gitlab/data/backups/ -name '*.tgz' -mtime +30
/mnt/data0/gitlab/data/backups/etc-gitlab-1611415968.tgz
Started calculate disk usage of build
Finished Calculation of disk usage of build in 0 seconds
Started calculate disk usage of workspace
Finished Calculation of disk usage of workspace in 0 seconds
Finished: SUCCESS

When I went to my buildbng17 machine and checked if there is any volume their, so there was volume which I had to clean up by performing

docker volume prune

The only drawback is that I need to do it manually and give that "y" when it asks for confirmation. And the data gets clean.

What command should I give in jenkins so that it will automatically clean up the volume without asking for confirmation? (because docker volume prune -f is not working)

I even tried to run the docker volume prune -f command manually in my linux machine, it still doesn't clean up the volume and shows 0kb freed up (However there is the volume which I checked from docker volume ls)

Is there any better approach we can do to automate it inside gitlab-runner itself ? Or some better feature of jenkins or docker of which I am not aware of?

The gitlab-runner keeps on running and covers the diskspace.

One more thing : We are using Gitlab and Gitlab-runner on docker.

Thanks in advance, Sameer

Sameer Atharkar
  • 382
  • 1
  • 3
  • 17

2 Answers2

38

Run these commands to do cleanup:

# Remove exited containers
docker ps -a -q -f status=exited | xargs --no-run-if-empty docker rm -v

# Remove dangling images
docker images -f "dangling=true" -q | xargs --no-run-if-empty docker rmi

# Remove unused images
docker images | awk '/ago/  { print $3}' | xargs --no-run-if-empty docker rmi

# Remove dangling volumes
docker volume ls -qf dangling=true | xargs --no-run-if-empty docker volume rm
Glen Thomas
  • 10,190
  • 5
  • 33
  • 65
  • 1
    Thanks a lot @Glen these commands perfectly worked for me. However can you tell me one thing what exactly does the { print $3 } means? – Sameer Atharkar Feb 25 '21 at 12:02
  • awk is a text processor like 'sed'. `/ago/` is a regular expression that selects the lines containing the text 'ago', this removes the header row from the docker images output. The `{ print $3 }` is a script that prints the 3rd token found on each line, which is the Image ID value. If you changed `$3` to `$2` it would output the image tag and `$1` would print the image repository + name. – Glen Thomas Feb 26 '21 at 17:05
  • 1
    @SameerAtharkar if this worked for you, would you like to mark this as an answer? Thanks – Himanshu Apr 04 '22 at 12:57
  • This worked for us, too. Thanks a lot. It should be the accepted answer. – Marc Saleiko Apr 28 '22 at 12:16
  • 1
    Isn't it the same as running a single `docker system prune` command? – scythargon Jul 28 '22 at 06:59
  • Can someone explain why the last command sometimes removes volumes `docker volume prune` refuse to remove? (ie.: dangling volumes are sometimes not removed by `docker volume prune`) – olejorgenb Mar 26 '23 at 21:27
  • For docker 23: https://github.com/docker/cli/issues/4028#issuecomment-1429538124 – olejorgenb Mar 26 '23 at 21:39
1

docker system prune command clean can affect the network too

https://docs.docker.com/engine/reference/commandline/system_prune/

user20984081
  • 11
  • 1
  • 2
    As it’s currently written, your answer is unclear. Please [edit] to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers [in the help center](/help/how-to-answer). – Community Jan 13 '23 at 15:37