0

What I want to achieve is to download files from Box directly to my COS bucket without downloading it to my local computer. I want to run this code in IBM Cloud Functions so I need to establish this direct download somehow.

def main(args):
    config = JWTAuth.from_settings_file('test_config.json')
    client = Client(config)
    print(client.user('me').get())

    #set the path to folder
    folderId = "131511144823"
    #filename we need
    file_we_looking_for = "test.xls"

    files_to_download = {}
    #get all items in box folder
    items_in_folder = client.folder(folder_id=folderId).get_items()
    for item in items_in_folder:
        if file_we_looking_for == item.name:
            print(item.name)
            files_to_download[item.id] = item.name

    #upload the file straight to object storage
    cos_client_conn = get_cos_client_connection()
    upload_large_file(cos_client_conn,'test','exceltestfrombox', files_to_download.keys()[0])


def upload_large_file(cos_cli, bucket_name, item_name, file_path):
    print("Starting large file upload for {0} to bucket: {1}".format(item_name, bucket_name))

    # set the chunk size to 5 MB
    part_size = 1024 * 1024 * 5

    # set threadhold to 5 MB
    file_threshold = 1024 * 1024 * 5


    # set the transfer threshold and chunk size in config settings
    transfer_config = ibm_boto3.s3.transfer.TransferConfig(
        multipart_threshold=file_threshold,
        multipart_chunksize=part_size
    )

    # create transfer manager
    transfer_mgr = ibm_boto3.s3.transfer.TransferManager(cos_cli, config=transfer_config)

    try:
        # initiate file upload
        future = transfer_mgr.upload(file_path, bucket_name, item_name)

        # wait for upload to complete
        future.result()

        print ("Large file upload complete!")
    except Exception as e:
        print("Unable to complete large file upload: {0}".format(e))
    finally:
        transfer_mgr.shutdown()

The problem is the upload_large_file function is looking for an absolute path of the file downloaded to my computer. I also tried to generate a shared link of the file and paste that there but it did not work.

Looz
  • 377
  • 2
  • 14
  • When you download, you do it to the system where the code is executed, similar for upload. If you run it as Cloud Functions action, the action runtime environment is where the down- and upload is performed. – data_henrik Jun 16 '21 at 18:10
  • Hi! Thanks for your feedback, in that case I assume it is not possible in this format. I run my code on Cloud Functions in a way to first download the file and then upload it to box. But as CF is a FaaaS I have no idea about what happens to my downloaded file. The upload works, but I receive errors like : "ResourceWarning: Enable tracemalloc to get the object allocation traceback" or "ResourceWarning: unclosed – Looz Jun 17 '21 at 07:52

0 Answers0