3

Following IBM documentation, I'm trying to list objects in my COS bucket using Python and ibm-cos-sdk.

import ibm_boto3
from ibm_botocore.client import Config, ClientError

cos = ibm_boto3.resource("s3",
      ibm_api_key_id=params['API_KEY'],
      ibm_service_instance_id=params['COS_TGT_INSTANCE_CRN'],
      ibm_auth_endpoint=params['IAM_ENDPOINT'],
      config=Config(signature_version="oauth"),
      endpoint_url=params['COS_ENDPOINT']
)

for obj in cos.Bucket('sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f').objects.all():
  print(obj.key)

Which results in:

ibm_botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist.

I'm pretty sure the bucket does exist, because I can clearly see it in the output from

>>> for b in cos.buckets.all():
...  print(b.name)
... 
sql-efbfb11b-fa01-4c49-8fe1-c70793be3f5f

What am I doing wrong here?

mustaccio
  • 18,234
  • 16
  • 48
  • 57

1 Answers1

4

The reason for the error is conceptual. You can see all buckets, but only can obtain details for buckets in the connected region. I ran into this a long time ago and solved it like this (tested then, not today):

def buckets_json():
   # Get a list of all bucket names from the response
   buckets = [bucket['Name'] for bucket in cos.list_buckets()['Buckets']]
   locs2=[{"name":name,"loc":loc} for name,loc in locations(buckets=buckets).iteritems()]
   return jsonify(buckets=locs2)

Another snippet which I found:

def locations(buckets):
   locs={}
   for b in buckets:
      try: 
         locs[b]=cos.get_bucket_location(Bucket=b)['LocationConstraint']
      except: 
         locs[b]=None
         pass
   return locs
data_henrik
  • 16,724
  • 2
  • 28
  • 49
  • Thanks Henrik, that was right on point. Your snippets apparently refer to the `Client` interface. I don't see how one can specify the bucket location using `Resource`, so I guess the approach here would be to simply specify the correct endpoint for each bucket. – mustaccio Mar 24 '20 at 16:36
  • Yes, I tried to extract the location info and connect to the right endpoint. – data_henrik Mar 24 '20 at 16:52