0

I have tried replicating this but it seems bit confusing still.

Scenario: I have 10 Hive tables, each pointing to different location within same GCS bucket.

For example, table1 points to gs://bucket/table1, table2 points to gs://bucket/table2 and so on. Overall say the bucket has 100 objects and each table has 10 objects.

Now, when someone queries for Select * from table1, the number of Class A operations (storage.objects.list) will be 10 or 100.

I believe it will be 100 every-time as listing all the objects inside that bucket will be 100. Wanted to know this fact from your end.

Thanks in advance!

dipayan
  • 128
  • 5
  • 18

1 Answers1

-1

In Google Cloud Platform (GCP), Class A operations, also known as metadata operations, are operations that involve listing or retrieving metadata about objects in a storage bucket, like the storage.objects.list operation you mentioned. These operations are separate from data retrieval or data transfer operations and are used to manage and organize objects within a bucket.

In your scenario, you have 10 Hive tables, each pointing to a different location within the same Google Cloud Storage (GCS) bucket. Each table has 10 objects, and the bucket as a whole has 100 objects.

When someone queries SELECT * FROM table1, GCS needs to retrieve the metadata for the objects in the gs://bucket/table1 path. The Class A operations (metadata operations) involved would be based on the number of objects in the table1 path, which is 10 objects. Therefore, in this specific query, you would have 10 Class A operations (storage.objects.list).

It's important to note that the metadata operations are based on the number of objects in the specific path being accessed, not the total number of objects in the bucket. So, querying a different table or path within the bucket would result in a different number of Class A operations based on the number of objects in that specific path.

In summary, for the scenario you described, querying SELECT * FROM table1 would result in 10 Class A operations (storage.objects.list), not 100.

Please refer to below links for better reference and official documentation on this: https://www.googlecloudcommunity.com/gc/Cloud-Hub/How-does-Cloud-Storage-pricing-with-operations-work/m-p/526788

https://cloud.google.com/storage/pricing#operations-pricing

Abdul Fahad
  • 264
  • 1
  • 5