Repeating the problem of transferring large images to the docker container registry

Question

I have a problem with the docker image push to Google Container Registry (GCP). However, image size is 7.97GB and the docker push is too slow. That is, as follows. enter image description here

It fills a portion of 3GB, then it starts loading again. One look at 700 MB, then 50 MB has been re-filled. I'm not getting an error, but this process repeats itself.

EDİT:

below is a description of docker image.

[
{
    "Id": "sha256:f2186ef59e404371d61ed9c8d46ed54070c0c957ec459b225e737c3af87a144e",
    "RepoTags": [
        "us.gcr.io/{PROJECT_NAME}/{IMAGE}:v15"
    ],
    "RepoDigests": [],
    "Parent": "sha256:3c3d250681a43128fc7f325a8bd36b1babf8c819f6f12ebee5641322c4554757",
    "Comment": "",
    "Created": "2020-01-05T00:53:33.8719212Z",
    "Container": "cfe1f5cf7bbdd66b200d5f8165525e92f4fba5f448c9f82ccd3685cb8a007f3e",
    "ContainerConfig": {
        "Hostname": "{HOSTNAME}",
        "Domainname": "",
        "User": "",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "LANG=C.UTF-8",
            "GPG_KEY={KEY}",
            "PYTHON_VERSION=3.7.3",
            "PYTHON_PIP_VERSION=19.1.1"
        ],
        "Cmd": [
            "/bin/sh",
            "-c",
            "#(nop) ",
            "CMD [\"/bin/sh\" \"-c\" \"cd ./dist     && FLASK_ENV=staging PEX_SCRIPT=gunicorn ./ws.pex ws.cloud.exchangewiser2.main:app --timeout 7200 --workers 2 --threads 2\"]"
        ],
        "Image": "sha256:3c3d250681a43128fc7f325a8bd36b1babf8c819f6f12ebee5641322c4554757",
        "Volumes": null,
        "WorkingDir": "",
        "Entrypoint": null,
        "OnBuild": null,
        "Labels": {}
    },
    "DockerVersion": "19.03.5",
    "Author": "",
    "Config": {
        "Hostname": "",
        "Domainname": "",
        "User": "",
        "AttachStdin": false,
        "AttachStdout": false,
        "AttachStderr": false,
        "Tty": false,
        "OpenStdin": false,
        "StdinOnce": false,
        "Env": [
            "PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
            "LANG=C.UTF-8",
            "GPG_KEY={KEY},
            "PYTHON_VERSION=3.7.3",
            "PYTHON_PIP_VERSION=19.1.1"
        ],
        "Cmd": [
            "/bin/sh",
            "-c",
            "cd ./dist     && FLASK_ENV=staging PEX_SCRIPT=gunicorn ./ws.pex ws.cloud.{WS_NAME}.main:app --timeout 7200 --workers 2 --threads 2"
        ],
        "Image": "sha256:3c3d250681a43128fc7f325a8bd36b1babf8c819f6f12ebee5641322c4554757",
        "Volumes": null,
        "WorkingDir": "",
        "Entrypoint": null,
        "OnBuild": null,
        "Labels": null
    },
    "Architecture": "amd64",
    "Os": "linux",
    "Size": 7968083980,
    "VirtualSize": 7968083980,
    "GraphDriver": {
        "Data": {
            "LowerDir": "/var/lib/docker/overlay2/21a848725e66c7932d91c0af86f7672a2d9235b1ec20bf68c689f0f11a8a6ff0/diff:/var/lib/docker/overlay2/3371822a45b13729e8d3bf53a33e3d06ad070885b9afbfda566a7b3019d0bc5a/diff:/var/lib/docker/overlay2/2c6bffe1c79ba4ea645a8fc6eb8e2437528eb0d0cf9b6ac95c96478262841dc2/diff:/var/lib/docker/overlay2/f4485137b22ae0c3146d884d79d9926ed81be001cf087d906303f1cbaa78e1ad/diff:/var/lib/docker/overlay2/52bf732e4116b4a3aee22de4c2c8132798d70799cd02107b503f4f173df1a86d/diff:/var/lib/docker/overlay2/120e18c051c7eb3a5e19d11860d849526c63f8caf4198013d3b13a0b1035b7f2/diff:/var/lib/docker/overlay2/b2fff569671c7f0469d190ed79ecbcd443822a097a6f10a835b6b479995b21ad/diff:/var/lib/docker/overlay2/dbc6cb47ba9b29f8c140618dadd5c4c1a7185273675653fe3aa68c99d2d4c5ad/diff:/var/lib/docker/overlay2/b648b220c3a9df26dc817f2e94878955f9c1b0005823bcae3f6fea4694a35606/diff:/var/lib/docker/overlay2/86ae30cf96c859dfece3f334a8ac329eeb434f607d5cc29e6fa4d07c60a6e9fe/diff:/var/lib/docker/overlay2/4204c124cd6540fbf07a7f437547dd24b518aa3f1b46198fe196aa644821263f/diff",
            "MergedDir": "/var/lib/docker/overlay2/d1dbdcd0c014f778cdf995a1838081bc24ba2e08e1af734360b7608d40a618c0/merged",
            "UpperDir": "/var/lib/docker/overlay2/d1dbdcd0c014f778cdf995a1838081bc24ba2e08e1af734360b7608d40a618c0/diff",
            "WorkDir": "/var/lib/docker/overlay2/d1dbdcd0c014f778cdf995a1838081bc24ba2e08e1af734360b7608d40a618c0/work"
        },
        "Name": "overlay2"
    },
    "RootFS": {
        "Type": "layers",
        "Layers": [
            "sha256:0db06dff9d9aeb9bed4edde7bd772ad0f3aea497c129d48ed587664d098c6c41",
            "sha256:f32868cde90bcb9aec7676c725c0d91738806c0eff429da4e9c99faaa40a9766",
            "sha256:7b76d801397d8df8cec8db17415e525376be29edd2f479c3cfeac5acd8220b58",
            "sha256:2c8d31157b818657c14d13625f3271f872842ed16d5028d5946fcea95719fb77",
            "sha256:a637c551a0dafc0bd915048ec09d40a1f6de5744c89da81311b1596a93e7751a",
            "sha256:bb9c02680a15456f6c58563789736424af6788f1bd66f96789604d26439aa139",
            "sha256:c9d608035aef2682df49cb8969a44ba13d74f3b7b2ada4b8cc3fbf9e76b1ff16",
            "sha256:7154504689406027d2321b65ddf95d882ffef86daa52f60dffa149d5ed9e3e8c",
            "sha256:799a7872c8c76e8286d9840fb8dd6e671e02b6b01430a74e2ddd8de3a3ac88a3",
            "sha256:44879110f87103379852e3ab7baea426c72e6c2cb9629b54772b28eae8585247",
            "sha256:5cc4fb492486b5fcb3bb95f9072445e3c99350cd307e79260b67304cc429b370",
            "sha256:55cadcb4d229a605c23ba10aa262887b441361f1c71f5baac1eac4cdf11a2f0c"
        ]
    },
    "Metadata": {
        "LastTagTime": "2020-01-05T01:17:48.5740547Z"
    }
}

]

Status of the terminal for 12 hours. When you press enter key in the middle of the bus.

The push refers to repository [us.gcr.io/{PROJECT_NAME}/{IMAGE_NAME}]
55cadcb4d229: Pushing [====>                                              ]  336.4MB/3.405GB
55cadcb4d229: Pushing [====>                                              ]  337.5MB/3.405GB
5cc4fb492486: Pushing [======>                                            ]    467MB/3.617GB
799a7872c8c7: Layer already exists
55cadcb4d229: Pushing [====>                                              ]  338.1MB/3.405GB
c9d608035aef: Layer already exists
55cadcb4d229: Pushing [====>                                              ]  338.7MB/3.405GB
5cc4fb492486: Pushing [======>                                            ]  467.5MB/3.617GB
55cadcb4d229: Pushing [====>                                              ]  339.2MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]  340.9MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]    342MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]  343.1MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]  345.3MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]  349.2MB/3.405GB
55cadcb4d229: Pushing [=====>                                             ]  353.1MB/3.405GB
5cc4fb492486: Pushing [======>                                            ]  478.1MB/3.617GB


55cadcb4d229: Pushing [======>                                            ]  441.7MB/3.405GB
5cc4fb492486: Pushing [========>                                          ]  580.4MB/3.617GB
55cadcb4d229: Pushing [======>                                            ]  442.8MB/3.405GB

55cadcb4d229: Pushing [======>                                            ]  444.4MB/3.405GB
5cc4fb492486: Pushing [========>                                          ]    582MB/3.617GB

55cadcb4d229: Pushing [======>                                            ]  448.9MB/3.405GB
5cc4fb492486: Pushing [========>                                          ]  587.6MB/3.617GB




5cc4fb492486: Pushing [========>                                          ]  588.2MB/3.617GB
55cadcb4d229: Pushing [>                                                  ]  12.78MB/3.405GB
5cc4fb492486: Pushing [>                                                  ]  29.93MB/3.617GB

5cc4fb492486: Pushing [>                                                  ]  30.49MB/3.617GB






55cadcb4d229: Pushing [>                                                  ]  13.34MB/3.405GB

















5cc4fb492486: Pushing [>                                                  ]  31.04MB/3.617GB








55cadcb4d229: Pushing [==>                                                ]  140.9MB/3.405GB
5cc4fb492486: Pushing [==>                                                ]    148MB/3.617GB

55cadcb4d229: Pushing [==>                                                ]  148.7MB/3.405GB
55cadcb4d229: Pushing [==================================================>]  3.405GB/3.405GB
5cc4fb492486: Pushing [==================================================>]  3.617GB/3.617GB










received unexpected HTTP status: 504 Gateway Timeout

This might help: https://stackoverflow.com/questions/43318225/google-container-registry-max-image-size — h3yduck, Jan 05 '20 at 08:24
thanks your answer @h3yduck. Yeah, I'm actually doing this. In the method you suggested, an error was received. But I'm not getting errors. I'm not getting any errors for about 8-9 hours and my terminal process is in progress. However, the load does not accelerate and restarts. To be more specific. Let's say that the publishing size is over 1,500 MB and it starts from 50MB again. I think it will start again after a default publish time. But I don't know where to fix it. — sumeyyeemir, Jan 05 '20 at 12:10
Is "enter image description here" the actual error message you get back from `docker push`? If not, can you add that error to the question? — David Maze, Jan 05 '20 at 12:41
In my past experience, having images (and especially single layers) substantially over 1 GB will hit trouble like this and you should look at ways to cut them down. — David Maze, Jan 05 '20 at 12:42
In fact, I'm using "pants build" as Mono-Repo. And I just want to deploy a web service. So I'm adding my entire Mono-Repo to DockerFile. @DavidMaze — sumeyyeemir, Jan 05 '20 at 12:45
Adding a mono repo to a Docker image is liable to be the cause of its large size. Can you create a build process that just copies the relevant code to the image? — halfer, Jan 05 '20 at 12:55
Unfortunately, I can't do it because the OS is MACOS, and if I build the corresponding code, the requirement library is downloaded as 'macos_x86_64'. But Google wants the App-Engine with the Linux operating system and 'linux_x86_64' libraries. That's why I built it using Docker. @halfer — sumeyyeemir, Jan 05 '20 at 13:04
I am not querying the use of Docker, I am suggesting you do not checkout the whole of your monorepo to the one image. Would paring this down to just the code you need make the image substantially smaller? 8G is way too large - you need to aim for lower than 1G (and then ideally carry on cutting). — halfer, Jan 05 '20 at 13:07
Yes, in fact, if my operating system was Linux, I would have done this reduction. In my previous reply, I explained why. But I'm thinking about this now. Can I create a docker image in the image and push it into the container? @halfer — sumeyyeemir, Jan 05 '20 at 13:17
Yes, you can build images inside a Docker container (Docker in Docker). Or you could start up Ubuntu under VirtualBox and use Docker from there. — halfer, Jan 05 '20 at 13:19
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/205382/discussion-between-sumeyye-and-halfer). — sumeyyeemir, Jan 05 '20 at 14:14

score 0 · Answer 1 · answered Jan 05 '20 at 14:06

0

I couldn't solve the problem because the size of the file (ws.pex) that I buil in the docker image is 6GB I am having this problem because of the mono-repo technology I use (Pants build). I found it right to move to Bazel as a solution.

I hope this solves my problem.

answered Jan 05 '20 at 14:06

sumeyyeemir

215
2
3
13

Is Pants something you plan to use at runtime? It sounds like a build tool. If so I wonder if you would benefit from a traditional hosted CI/build system like Jenkins, so that it does not have to download the whole of 6G dependency every time. – halfer Jan 05 '20 at 14:14
Actually, We only use Pants for mono-repo. Not CI. I think I'm gonna have to run some jetkins.@halfer – sumeyyeemir Jan 05 '20 at 14:30
"We only use Pants for mono-repo" - I am not sure what this means. Do you mean your runtime images are based on Pants? – halfer Jan 05 '20 at 14:39
No, runtime "custom" in app.flexible.yaml. I use pants to compile web-service in Docker image. @halfer – sumeyyeemir Jan 05 '20 at 15:01

Repeating the problem of transferring large images to the docker container registry

1 Answers1