0

I'm trying to adapt the open-source project mmfashion on Amazon SageMaker that requires the CEPH module for backend. Unfortunately pip install ceph doesn't work. The only work-around was to build the ceph source-code manually by running in my container:

!git clone git://github.com/ceph/ceph 
!git submodule update --init --recursive

This does allow me to import ceph successfully. But it throws the following error when it comes to fecthing data from Amazon S3:

AttributeError: module 'ceph' has no attribute 'S3Client'

Has someone integrated CEPH with Amazon S3 Bucket or has suggestions in the same line on how to tackle this?

Sayon Kumar Saha
  • 1
  • 2

1 Answers1

0

you can use ceph S3 api to connect to AWS buckets , here is the simple python example script to connect to any S3 api :

import boto
import boto.s3.connection
access_key = 'put your access key here!'
secret_key = 'put your secret key here!'

conn = boto.connect_s3(
        aws_access_key_id = access_key,
        aws_secret_access_key = secret_key,
        host = 'objects.dreamhost.com',
        #is_secure=False,               # uncomment if you are not using ssl
        calling_format = boto.s3.connection.OrdinaryCallingFormat(),
        )

then you will be able to list the buckets :

for bucket in conn.get_all_buckets():
        print "{name}\t{created}".format(
                name = bucket.name,
                created = bucket.creation_date,
        )
Norbert_Cs
  • 116
  • 2