Is there a way to cache DockerHub image in bitbucket pipeline?

Question

I am using external docker image from dockerhub.

In each step the dockerimage is pulled from dockerhub again and again. Yes it is desired workflow.

My question is can we cache this image, so that it wont pull from dockerhub in each step? This DockerImage is not going to change frequently, as it has only node and meteor as preinstalled.

So is it possible to cache the docker image?

Original bitbucket-pipeline.yml

image: tasktrain/node-meteor-mup

pipelines:
  branches:
    '{develop}':
      - step:
          name: "Client: Install Dependencies"
          caches:
            - node
          script:
            - npm install
            - npm run setup-meteor-client-bundle
          artifacts:
            - node_modules/**
      - step:
          name: "Client: Build for Staging"
          script:
            - npm run build-browser:stag
          artifacts:
            - dist/**
      - step:
          name: "Client: Deploy to Staging"
          deployment: staging
          script:
            - pipe: atlassian/aws-s3-deploy:0.2.2
              variables:
                AWS_ACCESS_KEY_ID: $AWS_ACCESS_KEY_ID
                AWS_SECRET_ACCESS_KEY: $AWS_SECRET_ACCESS_KEY
                AWS_DEFAULT_REGION: $AWS_DEFAULT_REGION
                S3_BUCKET: $S3_STAGING_BUCKET_NAME
                LOCAL_PATH: 'dist'
                ACL: "public-read"
                DELETE_FLAG: "true"
                EXTRA_ARGS: "--follow-symlinks --quiet"
      - step:
          name: "Server: Build and Deploy to Staging"
          script:
            - cd server
            - mup setup --config=.deploy/mup-settings.stag.js
            - mup deploy --config=.deploy/mup-settings.stag.js --settings=meteor-settings.stag.json

score 11 · Answer 1 · edited Feb 09 '22 at 08:33

As the OP said in comments to the other answer, defining a Docker cache doesn't work for the build image itself

image: tasktrain/node-meteor-mup

which is always downloaded for each step and then the step scripts are executed in that image. Afaik, the Docker cache

    services:
      - docker
    caches:
      - docker

only works for images pulled or built in a step.

However, Bitbucket Pipelines has recently started caching public build images internally, according to this blog post:

Public image caching – Behind the scenes, Pipelines has recently started caching public Docker images, resulting in a noticeable boost to startup time to all builds running on our infrastructure.

There is also an open feature request to also cache private build images.

This is indeed the correct answer to this question as of 2021 — David Costa, Dec 01 '21 at 12:03

score 5 · Accepted Answer · answered Mar 21 '19 at 21:21

5

It is indeed possible to cache dependencies and docker is one of the pre-defined caches of Bitbucket Pipelines

pipelines:
  default:
    - step:
        services:
          - docker
        caches:
          - docker
        script:
          - docker pull my-own-repository:5000/my-image

answered Mar 21 '19 at 21:21

β.εηοιτ.βε

33,893
13
69
83

Thank you so much for reply! I'm new it to bitbucket pipeline. I just made "emjimadhu/meteor-node-mup" on the image property in TAML. As I can see from your example, we manually have to pull repo from bitbucket! But for now, bitbucket automatically pulls and does the steps for me. Is it enough to mention "services: docker" and "cache: docker" is enough? – Em Ji Madhu Mar 21 '19 at 21:41
1

Yes, the `docker pull` step I put in there was just for the example, you can pull, or build or use whatever docker command there, the `cache` is what makes the dependency caching indeed – β.εηοιτ.βε Mar 21 '19 at 21:45
1

And if you don't need to run any `docker` command inside your step, you can even leave out the `services` – β.εηοιτ.βε Mar 21 '19 at 21:47
Hi, thank you so much for clarifying my doubts. I have one more doubt. I promise this will be the last one. Say i have three steps and specifying `cache: docker` at first step is enough, so that cache be used in next steps or do i need to mention it in every step? thanks. – Em Ji Madhu Mar 22 '19 at 10:31
1

From what I see in the documentation, yes indeed, you need to specify that you want to use the cache on each and every step you need it, otherwise you won't have the cache benefits on those steps on which you omit it. – β.εηοιτ.βε Mar 22 '19 at 11:25
1

Hi, i am sorry to disturb you, i cant cache docker image, this is the output `We only found system docker images so we won't create a docker cache` – Em Ji Madhu Mar 22 '19 at 12:18
Maybe I missed something from your actual implementation then. Care to share your actual `bitbucket-pipelines.yml` in an edited version of your question? – β.εηοιτ.βε Mar 22 '19 at 13:08
Hi i edited the question. As you can see, i am using custom docker image from dockerhub. I want to cache that image, so it wont be downloaded again and agin. Thanks. – Em Ji Madhu Mar 22 '19 at 13:25

Is there a way to cache DockerHub image in bitbucket pipeline?

2 Answers2