2

I'm trying to set up a sink that will export a certain set of Google Cloud Platform logs to a Google Cloud Storage bucket but can't get it to work and the documentation doesn't seem to match what's happening on the GCP console.

Steps (all using the GCP console):

1) I set a filter on the log viewer which is showing me the expected logs

2) I choose "Create Export" and fill in the fields:

  • Sink Name = defaultServiceToGCSSink
  • Sink Service = Google Cloud
  • Storage Sink Destination = mylogsBucket

After hitting OK, I get a message: Unknown user email address: defaultServiceToGCSSink@logging-somedigits.iam.gserviceaccount.com

Apparently the sink is trying to use the name I gave it as the user that will be writing to the storage bucket.

When I check the bucket I can see that a user with that email was added as an owner to mylogsBucket. but still no logs in the bucket.

I also added the group cloud-logs@google.com as an owner to the bucket (as the documentation states) but nothing works and no logs are exported to the bucket (and I've waited for more than a couple of hours).

Should I be adding that new user to IAM? I tried to but it wouldn't accept the email address as a valid user name.

Shai Ben-Tovim
  • 892
  • 8
  • 20

2 Answers2

1

Remove the gserviceaccount.com user from the bucket ALCs and then try creating the sink.

Is there any chance you successfully created the sink at some point in the past and later deleted it? My guess is the service account was put on the bucket earlier, and now the sink creation is failing because it's trying to add the account again.

Kirk Kelsey
  • 4,259
  • 1
  • 23
  • 26
  • Thanks. I started clean from scratch and everything works now. I was unclear about who's responsibility it was to add the account permissions to the bucket but apparently it is indeed done behind the scenes. – Shai Ben-Tovim Apr 21 '17 at 05:44
0

In usual scenario, it might take some time before the first entries begin to appear in the google storage bucket because log entries are saved to Cloud Storage buckets in hourly batches.

When you export logs to a Cloud Storage bucket, logging writes a set of files to the bucket that are organized in directory hierarchies by log type and date.

Detailed explanation for what happens to the exported logs : https://cloud.google.com/logging/docs/export/using_exported_logs

Kishor Unnikrishnan
  • 1,928
  • 4
  • 21
  • 33