5

I'm having an issue where an App Engine project will no longer build remotely (via gcloud app deploy)

This has started out of the blue, with no code changes at this end. Not sure if relevant, but it's a go 1.9 project deploying to the App Engine Flex environment.

I'm not sure how to test this in the same environment as the build, since the error is coming from Google's Container Registry

Here is the log from the Container Registry console

starting build "73f85b4d-7370-41bd-bbb2-bcf42fc38873" 

FETCHSOURCE 
Fetching storage object: gs://staging.[project].appspot.com/us.gcr.io/[project]/appengine/default.1ed3c690ead06f27aa651a30fab342611:latest#1531698266413753 
Copying gs://staging.[project].appspot.com/us.gcr.io/[project]/appengine/default.1ed3c690ead49f731806f27aa630fab342611:latest#1531698266413753... 
Operation completed over 1 objects/1.7 MiB.  

BUILD 
Starting Step #0 
Step #0: Pulling image: gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db 
Step #0: sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db: Pulling from gcp-runtimes/go1-builder 
Step #0: Digest: sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db 
Step #0: Status: Downloaded newer image for gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db 
Step #0: exec: "gcc": executable file not found in $PATH Finished 
Step #0 ERROR ERROR: build step 0 "gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db" failed: exit status 2
calcinai
  • 2,567
  • 14
  • 25
  • Can you check your PATH value in your ~/.bashrc? Does it include /usr/bin? – VonC Jul 16 '18 at 04:24
  • In my ~/.bashrc or the remote one? I don't know how to access the running instance in GCR... – calcinai Jul 16 '18 at 05:27
  • The remote one. – VonC Jul 16 '18 at 05:34
  • Is it possible to access the remote shell in GCR? I guess that's is really my question, if I could get in and have a look I might be able to fix it. – calcinai Jul 16 '18 at 06:14
  • Maybe using cloud shell? https://console.cloud.google.com/getting-started?cloudshell – VonC Jul 16 '18 at 06:23
  • I've had a look through the docs and I can't see any specific references to it. Do you know if it's possible to test with the `gcr.io` images locally? Even if I manage that, and I fix the `PATH`, I don't see a way to modify Google's container templates. – calcinai Jul 16 '18 at 08:58
  • Not sure: https://github.com/GoogleContainerTools/container-structure-test/issues/93 maybe? – VonC Jul 16 '18 at 09:16
  • You can run a build with a build step that overrides the entrypoint to use the bash shell and echo your $PATH -- set the `entrypoint` on `BuildStep` to `/bin/bash` and use args `[ "-c", "whatever bash command you want to run" ]`. – David Bendory Jul 16 '18 at 15:46

1 Answers1

3

It looks like you are using container gcr.io/gcp-runtimes/go1-builder as your build step. Looking at the source in GitHub, I see that there have been no updates since ~late June. I see in the Dockerfile that the base image in the FROM directive is gcr.io/google-appengine/debian9:latest, and a look at that image reveals no gcc installed. I see no step in the Dockerfile installing gcc, and looking at your build step image confirms that it isn't there:

~$ docker run --rm -t -i --entrypoint /bin/bash gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db -- which gcc
Unable to find image 'gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db' locally
sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db: Pulling from gcp-runtimes/go1-builder
e154cec6816f: Pull complete 
<pulls elided>
Digest: sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db
Status: Downloaded newer image for gcr.io/gcp-runtimes/go1-builder@sha256:c62ac3fbec31ddec70601d6c5b44d07063bcff6a823bdcf5e0bbaa9d3799d1db
~$

Perhaps an earlier version of the base debian9 image had it installed, you could dig into history to look. But it looks like there is no recent change to the go1-builder image to remove gcc.

If you need gcc, you can always separate building your app from deploying it. Build with your own cloudbuild.yaml via gcloud container builds submit and then deploy the built container using gcloud app deploy --image-url=... With full control over the build, you can always based on the go-builder image and install additional tooling you need like gcc on top of that before using Docker to build your final app container.

David Bendory
  • 1,258
  • 8
  • 14
  • I guess this is my best bet. It does seem strange that that Google's build image can't even build the App Engine SDK! (when I remove it as a dep, it deploys fine) – calcinai Jul 17 '18 at 02:35
  • I filed this issue: https://github.com/GoogleCloudPlatform/golang-docker/issues/141; if this is critical to you, they no doubt accept pull requests... – David Bendory Jul 18 '18 at 12:34
  • Another option is to change the `app.yaml` to have `runtime: custom` and create your own Dockerfile with your favorite image to build the app: ``` FROM golang:latest RUN mkdir /app ADD . /app/ WORKDIR /app RUN go build -o main . CMD ["/app/main"] ``` – Montaro Mar 15 '20 at 00:34
  • Got here after failing to deploy the "Hello World!" example in the tutorial on Google Cloud Platform. Turns out I need to build with my own cloudbuild.yaml, or create my own Dockerfile and image, or discover where the dependencies on gcc are. Welcome to Entry Level. – simonpa71 Apr 19 '22 at 17:50