We have a Golang application running in the Google App Engine, which reads a signing key using cloud.google.com/go/storage. When trying to deploy a newly updated version, it started receiving errors saying that it could not find default credentials, and thus not access the storage.
bucketName, err := file.DefaultBucketName(ctx)
if err != nil {
log.Errorf(ctx, "failed to get default GCS bucket name: %v", err)
return nil, nil, err
}
client, err = storage.NewClient(ctx)
if err != nil {
log.Errorf(ctx, "failed to create client: %v", err)
return nil, nil, err
}
(ctx is the context.Context we create using appengine.NewContext() with the incoming http.Request object as parameter; file is google.golang.org/appengine/file)
This yielded this error log:
"2020-06-15 09:16:51.809 CEST failed to create client: dialing: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information"
Tracing the error through the golang libraries, we found that it seems that the application fails to resolve the metadata hostname that is supposed to be set when this runs on the Google Cloud platform.
We have worked around the issue by adding this to the app.yaml file, but it feels wrong to have to do that:
env_variables:
GCE_METADATA_HOST: "169.254.169.254"
This forces testOnGCE() in metadata.go (https://github.com/googleapis/google-cloud-go/blob/master/compute/metadata/metadata.go#L117) to return true, and lets us read the signing key from storage.
Is this a bug in the Google Cloud platform that we need to work around, or is there anything wrong with the code above? It has worked fine since we added it in 2018 and only started failing when deploying code in June 2020, and the deploy that broke only contained changes to the client JavaScript code, the golang backend was not touched. We tried redeploying it several times, but the error message was 100 % reproducible until we added the workaround.
