1

I'm trying to create my first GitHub Docker Compose workflow in order to:

  • Build a Docker Compose
  • Push images to a custom registry (Digital Ocean)

All services images are prefixed with my registry and owner, i.e: registry.digitalocean.com/myregistry/php-prod. If I run docker compose build --push locally, it works just fine.

Solution 1 the workflow runs just fine, but doesn't push anything:

jobs:
  build:
    name: Build
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Install doctl
        uses: digitalocean/action-doctl@v2
        with:
          token: ${{ secrets.DIGITALOCEAN_ACCESS_TOKEN }}

      - name: Log in into Container Registry
        run: doctl registry login --expiry-seconds 600

      - name: Build and push
        run: docker compose build --push

Last jobs log lines:

#31 exporting to docker image format
#31 exporting layers
#31 exporting layers 0.4s done
#31 exporting manifest sha256:3f84ebb06488d40b1b70bc1434c5ac5f18da8d2ea730d6e9680d2663705c47e6 done
#31 exporting config sha256:c143d0652b248b4ecd500f8a8a7d9f44cab3968e59dff8e1c073946eee8106a1 done
#31 sending tarball
#31 sending tarball 2.2s done
#31 DONE 2.6s

#32 importing to docker
#32 DONE 1.5s

Solution 2: split the "Build and push" which gives me an authorization error:

- name: Build and push images
  run: |
    docker compose build
    docker compose push

With this step definition, I've got an error about

Pushing caddy: 5bc340f6d4f5 Waiting 
unauthorized: access token has insufficient scopes
Error: Process completed with exit code 1.

Why solution 1 doesn't push my images? Why, on the contrary, solution 2 seems to push something but it's failing for the token?

gremo
  • 47,186
  • 75
  • 257
  • 421
  • Why don't you use a dedicated `docker compose push` command? Or at least, have you tried that? – hakre May 23 '23 at 10:18
  • Yes, but my dockerfile uses COPY --link which isn't supported. I'd like to understand why isn't working and why there is no error in debug log. – gremo May 23 '23 at 10:24
  • You are right, let me edit the question, thanks. – gremo May 23 '23 at 10:32
  • From your edit, at the end of the day please ask the vendor and contract them for the very official answer, but given the error in _solution **2**_ leads me to an educated guess that the errors message is the reason why _solution **1**_ does not push the images (as it is only an option and not the main command, it does not fatal). This is the reason why you should chain the two commands here, `docker compose build && docker compose push`, to make the step failing during automation. This _is_ important. Now use the correct token and done, this looks like a plain configuration issue. – hakre May 23 '23 at 10:58
  • Thanks @hakre! But it doesn't answer the question why docker compose build --push doesn't fail (and it's a single command)... – gremo May 23 '23 at 11:11
  • In addition to that, please note that the login action to DO registry works fine - no errors. – gremo May 23 '23 at 11:12
  • You may think this problem goes away when you argue with me about it and I really enjoy having a chat with you, this is pretty relaxing. But rest assured, you can't argue with your computer systems the same way, so whether or not you deal with the error for `docker compose push`, at the end of the day this is your choice. Now as you've found an error - and it looks new to you as you're still figuring out how to understand it - take all the time you need. – hakre May 23 '23 at 11:15
  • @hakre Sure, I enjoiy the chat too :) I'm now investigating, of course, but the primary question was "why docker compose build --push" doesn't trigger any error and runs just fine? Then you talked about splitting the command - ending up with the token error, and i thank you for that, is a clue ;) – gremo May 23 '23 at 11:20
  • I've not checked the source code, but the answer to why you'll find there. So you can ask me about my opinion, but then I've shared it already, as written it is an educated guess, but I've seen `docker` commands done more strange things so far than that just for the parameters, and it was perhaps that, why I asked if you have tried to run the command that is specifically pushing. That's all. There is not much magic behind these things at the end of the day. – hakre May 23 '23 at 11:24
  • And turns out it was - like you said - a configuration problem, i've done some debug runnign `docker image` after build. You made my day! I think I'll going to delete this question as it's not useful for the community. – gremo May 23 '23 at 11:40
  • Oh I think it is useful, at least when a conclusion could be run, because as you wrote: The question remains to `--push`: Could it be you have buildkit disabled locally but enabled in the Microsoft Github Action? I've added an answer. – hakre May 23 '23 at 11:50
  • Locally, buildkit is enable (I see "true" in the docker json configuration) – gremo May 23 '23 at 13:00

1 Answers1

1

Why finally docker compose build --push does not push in Github, I could not test fully, but the source shows at least different paths the command may take and I could only see there that the --push command line option is probably taken into account only if buildkit¹ is false.

So at the end of the day this is even more a reminder to run

docker compose build && docker compose push

in a pipeline to make it not run into this build gap (you expect the images pushed to the registry, but the images are never pushed).

¹ cf. BuildKit https://docs.docker.com/build/buildkit/

hakre
  • 193,403
  • 52
  • 435
  • 836
  • As I said in the other comment, I have buildkit enable also on my machine... but I looked at the code and seems like you said! – gremo May 23 '23 at 18:50
  • but that place is only part of the story, there is the whole source in buildx and there is an option to suppress errors during push. at the end of the day it is probably more close to my initial educated guess, but the takeaway should be that the concrete reason is not that important: if you have a more precise command that is emitting a good error message and has non-zero exit status, then this is the command you want to use in your build pipeline as it has the capability to be a build test. this is what you want to have in/for CI/CD. [1/2] – hakre May 23 '23 at 19:45
  • [2/2] I can understand that it can be disappointing to not fully understand the cause of the problem. But sometimes it is better to take the notes of so far and get it functional so that the gaps are narrowed already. With more time and more practice comes more wisdom. Increase the velocity first, that will help you to get better results then faster. – hakre May 23 '23 at 19:47
  • I guess it’s my limit - to always try to find an answer, but you are right, one should focus to the objective. – gremo May 23 '23 at 22:03
  • 1
    @gremo: I think this is human nature, it is important for us to understand, this is one of our senses, if not one of the strongest ones. And here another example that while it found the answer, the conclusions are actual similar: https://stackoverflow.com/a/76181305/367456 – hakre May 24 '23 at 09:43