1

I have multiple GKE clusters with logically separated K8s objects using namespaces, each namespace has GCS bucket attached to it (Statefulset) I wanted to know billing information about each namespace and what each namespace costs for resources usage. I found very useful feature called "GKE Metering" and yea it showed up some useful information about CPU and Memory billing info for each namespace by gathering billing information and import it into BigQuery, then I can visualize the results using Google Data Studio for example.

The Question Is:

  1. How can I get billing information of each GCS bucket which attached to each namespace to be visualised?
  2. How to get billing information fo each namespace usage of Filestore Instance?

Any help will be appreciated Thank you

Kartik Domadiya
  • 29,868
  • 19
  • 93
  • 104
Ahmed Khaled
  • 13
  • 4

1 Answers1

1

For that, you need to add a label on your GCS bucket

Then to export the billing to BigQuery

And to get the data like that in BigQuery

SELECT * 
FROM `project.dataset.exportBillingTable` 
WHERE "<Your Label Key>" IN UNNEST(labels.key)

You can perform "group by", "sum", filter on the label key or the label value,...

guillaume blaquiere
  • 66,369
  • 2
  • 47
  • 76
  • I added labels to GCS buckets I have, and exported the data to BigQuery... I can see in the tables there are data about "Compute Instances", "Cloud SQL", and everything but I cannot find anything related to GCS buckets – Ahmed Khaled May 27 '21 at 19:16
  • Wait a day. It took about 20H for me between the label set up and the propagation in BigQuery billing table – guillaume blaquiere May 27 '21 at 20:06
  • Thank you for your valuable information another question please, Am I being billed for GCS buckets even if it is empty? As for testing purposes I created two empty buckets and not showing in the table after more than 24 hours all resources appeared except empty buckets is that right or even if it is empty it must appear in BigQuery? – Ahmed Khaled May 27 '21 at 20:16
  • Yes, empty bucket don't appear. But you can put a very very small file, only to generate few dollars per months (the minimum is 10e-9, so file of few bytes is enough to see an entry in the billing but you will be never really billing for so few.) – guillaume blaquiere May 27 '21 at 20:21
  • That's Great News!! Thanks a Lot ✨ – Ahmed Khaled May 27 '21 at 21:40