1

I'm trying to find a way to automate the task of getting COS bucket sizes on IBM Cloud.

I have dozens of buckets over different accounts but still couldn't find a way to get this information using IBM Cloud COS CLI, just other information, like bucket names, etc.

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
Murilo Lang
  • 11
  • 2

2 Answers2

4

The COS S3 API does not return size information for buckets. Thus, the CLI, which is based on the API, won't return size information either.

But here's an indirect way to find the size of a bucket by looping through the sizes of the individual objects in the bucket

ibmcloud cos objects --bucket <BUCKET_NAME> --output JSON | jq 'reduce (.Contents[] | to_entries[]) as {$key,$value} ({}; .[$key] += $value) | .Size'

The output is in bytes

You may have to loop through the bucket names may be in a shell script. For all the buckets in an account + resource group, run the below command

ibmcloud cos buckets --output JSON

Note: Before running the above commands, remember to add the COS service CRN to the config with the below command

ibmcloud cos config crn --crn <SERVICE_CRN>
data_henrik
  • 16,724
  • 2
  • 28
  • 49
Vidyasagar Machupalli
  • 2,737
  • 1
  • 19
  • 29
  • Regarding that last Note, is there a way to get the CRN using the CLI? All I can find is how to list the items in the service catalog (`ibmcloud catalog service cloud-object-storage`). – Christy Dec 06 '21 at 17:12
  • found it: `ibmcloud resource service-instance ` – Christy Dec 06 '21 at 17:26
1

The answer that loops through the individual objects is indeed the only (and likely best) way to use the IBM Cloud CLI to find that information, but there are a few other ways worth mentioning for completion.

If you need to do this elegantly on the command line, the Minio Client provides a Linux-esque syntax:

mc du cos/$BUCKET

This returns the size of the bucket in MiB.

Additionally, the COS Resource Configuration API will directly return a bytes_used value, with no iterating over objects behind the scenes. While there's no official CLI implementation yet (although it's in the pipeline) it's relatively easy to use cURL or httpie to query the bucket.

curl "https://config.cloud-object-storage.cloud.ibm.com/v1/b/$BUCKET" \
     -H 'Authorization: bearer $TOKEN'
Nick Lange
  • 857
  • 6
  • 8