8

I am exploring the docker build process using the Buildkit. I have enabled it by setting an environment variable DOCKER_BUILDKIT=1 on the CLI before invoking docker build. From the buildkit documentation it says,

  • To use an external Dockerfile frontend, the first line of your Dockerfile needs to be # syntax=docker/dockerfile:1 pointing to the specific image you want to use.

and from the syntax documentation it says,

  • This feature is only available when using the BuildKit backend, and is ignored when using the classic builder backend.

What does this frontend and backend means in terms of docker build process? Can anyone help me to understand it in simple terms?

Kishor Unnikrishnan
  • 1,928
  • 4
  • 21
  • 33

1 Answers1

15

TLDR; The frontend and backend concept was born with Buildkit and didn't exist in docker before. Frontend is like a compiler that converts a user's file (eg: Dockerfile) to LLB. Backend executes LLB in the most efficent way to build a docker image.

History

Without Buildkit, a docker image is built directly using instructions in a Dockerfile. No intermeditate representation of these instructions is created. The instructions are passed to the docker engine (also called the Moby Engine or classic builder) which then builds the image.

Then it was realised that to improve and optimise the build process further most of the fundamentals of the build operation would have to be redefined. Hence a proposal was made to create a new engine, and Buildkit was born along with frontend and backend separation of the build process.

One of the main design goals of buildkit is to separate frontend and backend concerns during a build process. A frontend is something designed for the users to describe their build definition. Backend solves the problem of finding a most efficient way to solve a common low-level description of the build operations, that has been prepared for them by the frontends.

LLB

The separation of frontend and backend is acheived by LLB(low-leve builder).

Everything about execution and caching of your builds is defined only in LLB.

Frontend

Frontends are components that run inside BuildKit and convert any build definition(file written by the user) to LLB. BuildKit supports loading frontends dynamically from container images by specifying: #syntax=.... A famous frontend is the dockerfile frontend because it is used with the docker engine. You can specify this container image with: #syntax=docker/dockerfile:latest.

There are plenty of other frontends that can be used, for example the mockerfile frontend with: #syntax=r2d4/mocker. This then allows you to use a slightly different syntax compared to the usual Dockerfile syntax.

Backend

The Buildkit backend solves the LLB generated from any of a variety of frontends. Since the LLB is a dependency graph, it can be processed to: detect and skip executing unused build stages, parallelize building independent build stages etc. This is why Buildkit is able to improve performance, storage management etc. over the older build process. Also, the caching model has been entirely rewritten.

The core part of the builder(Buildkit) is a solver that takes a DAG of low-level build(LLB) instructions from the frontend and finds a way to execute them in a most efficient manner while keeping the cache for the next invocations.

To use the Buildkit backend specifyDOCKER_BUILDKIT=1.

Starting with version 18.09, Docker supports a new backend for executing your builds that is provided by the moby/buildkit project.

The Moby Engine(the classic builder) can be called the original backend but remember it doesn't use LLB, therefore its build process doesn't have a frontend and backend as such.

References and resouces:

viggnah
  • 1,709
  • 1
  • 3
  • 12
  • Thanks for the explanation. A few questions remain confused here are 1) So before the version 18.09, do docker holds a frontend component (dockerfile.v0) to parse the Dockerfile and give to the classic backend ? If yes, does it produce LLB code previosuly also? 2) docker/dockerfile is the gateway frontend we are writing in the first line of Dockerfile. What is then the default docker frontend image? – Kishor Unnikrishnan Jul 22 '22 at 08:59
  • @KishorU, sorry for any confusion caused. I have revamped the answer to make it much clearer. I think question 1) is now answered above. Question 2) Without buildkit, there is no *default* image as the docker engine doesn't allow external frontends to be used. With buildkit, the default would be dockerfile because BuildKit ships with dockerfile frontend. But it is recommended to specify an external image even for dockerfile. Hope this clarifies! – viggnah Jul 23 '22 at 06:37
  • Thanks a lot for very detailed note. Actually the 3rd and 4th link points to same page. If it's by mistake please update with that Docker blog, that will be so helpful. Also one more question, if buildkit is shipped with a default dockerfile frontend, how can I view the default dockerfile: shipped with my Buildkit in a particular docker release. – Kishor Unnikrishnan Jul 23 '22 at 22:36
  • 1
    @KishorU, Thanks for pointing out, I have corrected the link. I'm sorry but I can't see how to see the version exactly in a particular release using docker-cli or something similar. However, I spotted it in the release notes: https://github.com/moby/buildkit/releases. For example, in the latest release of buildkit it is `1.4.2`. – viggnah Jul 25 '22 at 12:21