How can I transfer a file from one bucket to another using flysystem?

Question

I have objects in one bucket that I occasionally need to transfer to a second bucket in Amazon S3. I'm using Laravel 5.3 with Flysystem to manage those buckets.

One solution is to download the images to my server and then upload it to the other bucket but this seems like a waste of time/bandwidth since the file exists in S3 and is getting moved within S3. Can this be done within Flysystem or will I need to directly use Amazon's API?

You could write a job for the files which has to be transferred and in the job, use `$contents = Storage::get('file.jpg');` to get the file and create separate new `Flysystem` instance with the connection to new server and write the contents which are read. — Cerlin, Jan 05 '17 at 08:41
@CerlinBoss wouldn't I be downloading and uploading at that point? That's what I'm trying to avoid. My goal is to transfer the file from bucket -> bucket, not bucket -> my server -> bucket. — Citizen, Jan 05 '17 at 08:58
i dont think thats something which you can do from your laravel server. U may need to leverage Amazon's API for that (if they provide any). — Cerlin, Jan 05 '17 at 09:36

score 8 · Answer 1 · edited Sep 06 '18 at 01:01

I have found the solution. Here is my code:

try {
    $s3 = Storage::disk('s3');
    $s3_temp = $s3->getDriver()->getAdapter()->getClient()->copy(
        env('S3_BUCKET_ORIGIN'),
        $file_path_origin,
        env('S3_BUCKET_DEST'),
        $file_path_dest,
        'public-read'
    );
} catch(\Exception $e) {
    dd($e->getMessage());
}

Remember S3 filesystem of Laravel uses the SDK aws-s3-v3 therefore, you search the libraries of aws-s3-v3 to see what functions the Laravel wrapper has.

So, in the example, I get the Client Class of aws-s3 so I found out in the documentation that, with this class, I can move a file from one bucket to another.

S3 Php Documentation - Client Class - Copy Method

After one day having headaches with this issue, finally a solution. Thank you! — cawecoy, Jul 20 '18 at 22:46

score 0 · Answer 2 · answered Sep 27 '19 at 07:41

0

For Laravel > 5.6

 try {
        // creat two disk s3 and s3new 
        $fs = Storage::disk('s3')->getDriver();
        $stream = $fs->readStream('file_path/1.jpg');
        $new_fs = Storage::disk('s3new')->getDriver();
        //will create new folder damages if not available 
        $new_fs->writeStream(
            'damages/newfile.jpg',$stream
        );
    } catch(\Exception $e) {
        dd($e->getMessage());
    }

answered Sep 27 '19 at 07:41

Furquan

1,542
16
20

This is not desirable as it results in the server acting as the conduit for all the bytes. Not only can it be slow, but it will also result in additional ingress and egress and thus be more costly. – Alexander Trauzzi Jan 27 '22 at 18:37

score -3 · Accepted Answer · answered Jan 05 '17 at 15:25

-3

You can use the FilesystemAdapter move function to move a file:

$disk = Storage::disk('s3');
if (!$disk->move('bucketOne/testFile.jpg', 'bucketTwo/testFile.jpg')) {
   throw new \Exception('File could not be moved.');
}

answered Jan 05 '17 at 15:25

Link.de

353
1
4

1

Thanks. I'm trying to find documentation that shows that it won't passthrough my server and will be direct from bucket to bucket, but I don't see that method in the Flysystem API documentation. I'm not sure how I would go about testing to make sure this is the case otherwise. – Citizen Jan 05 '17 at 16:58
1

The FilesystemAdapter is using the flysystem-aws-s3-v3 Library. The move functions calls two functions: copy and delete (first copy the object and then delete the old). Copy uses copyObject by the php S3Client written by AWS. I don't know what they are doing internally but I think its an API-Call to S3 which does not download the file and upload it again. – Link.de Jan 13 '17 at 08:25
3

This only allows to move files within a bucket but not between them – Poma Sep 10 '17 at 07:58

How can I transfer a file from one bucket to another using flysystem?

3 Answers3