Swarm stack is deployed before the new images are pushed

Question

I use CircleCI and the pipeline is as follows:

build
test
build app & nginx Docker images and push them to a GitLab registry
deploy Docker stack to the development server (currently the Swarm manager)

I just pushed my develop branch to my repository and faced a "Symfony4 new Controller page" on the development server after a successful message from CircleCI.

I logged via SSH in it and executed (with output for the application service):

docker stack ps my-development-stack  --format "{{.Name}} {{.Image}} {{.CurrentState}}"

my-stack_app.1    gitlab-image:latest-develop    Running 33 minutes ago

On my GitLab repository's registry, the application image has been "Last Updated" 41 minutes ago. The service's image has apparently been refreshed before with the last version.

Is it a common issue/error ?
How could (or should) I fix this timing issue ?
Can CircleCI help about this ?

score 0 · Answer 1 · answered Jul 06 '19 at 18:35

0

Perhaps it is best ( though not ideal ) to introduce a delay between build and deploy , you can refer to this example here CircelCI Delay Between Jobs

answered Jul 06 '19 at 18:35

Soumen Mukherjee

2,953
3
22
34

As a pipeline should not exceed a certain amount of time, this could work. A nightly deploy could do the trick too but I think these solutions are just workarounds, not technical solutions. – AymDev Jul 06 '19 at 18:45
Perhaps you can try this as well https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#push-events – Soumen Mukherjee Jul 06 '19 at 18:54
These are push webhooks for gitlab repositories, not for pushing to a registry. I only use gitlab for the registry, the application git repository is elsewhere – AymDev Jul 06 '19 at 19:33
I found [registry webhooks](https://gitlab.com/gitlab-org/gitlab-ce/issues/18293) but it still isn't implemented :-/ – AymDev Jul 06 '19 at 19:36

score 0 · Answer 2 · answered Jul 16 '19 at 09:04

I found a workaround using a CircleCI scheduled workflow triggered by a CRON. I scheduled a nightly build workflow which will run every day at midnight.

Sample of my config.yml file

# Beginning of the config.yml
# ...

workflows:
    version: 2

    # Push workflow
    # ...

    # Nightly build workflow
    nightly-dev-deploy:
        triggers:
            - schedule:
                  cron: "0 0 * * *"
                  filters:
                      branches:
                          only:
                              - develop
        jobs:
            - build
            - test:
                requires:
                    - build
            - deploy-dev:
                requires:
                    - test

Read more about scheduled workflow with a nightly build example in the CircleCI official documentation

Looks more like a workaround to me. I'd be glad to hear how do you avoid this issue, which could lead to a better answer to the question.

Swarm stack is deployed before the new images are pushed

2 Answers2