26

I would like to know how to copy the object from s3 to s3 using node.js With the aws s3 command, It could be executed as follows.

s3 cp --recursive s3://xx/yy  s3://zz/aa

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#copyObject-property

I refer to above link,But I don't know how to do it using node.js

any Idea?

negabaro
  • 3,127
  • 7
  • 21
  • 33
  • This might help - https://stackoverflow.com/questions/30959251/how-to-copy-move-all-objects-in-amazon-s3-from-one-prefix-to-other-using-the-aws – kawadhiya21 Oct 26 '17 at 06:33

5 Answers5

34

If you just want to copy one object copyObject API.

 var params = {
  Bucket: "destinationbucket", 
  CopySource: "/sourcebucket/sourceKeyName", 
  Key: "targetKeyName"
 };
 s3.copyObject(params, function(err, data) {
   if (err) console.log(err, err.stack); // an error occurred
   else     console.log(data);           // successful response
 });

If you want to perform recursively for all objects in a bucket, then

  1. List all the object keys in a bucket using the listObjectsV2 API.

  2. If versioning is enabled in the source-bucket and you want to copy a specific version of the key, Invoke the listObjectVersions API as well and get the Version-Id for each S3 Key.

    NOTE: If versioning is not enabled, then you can ignore STEP-2.

  3. Call copyObject for each S3 Key and the Version-Id obtained in Step-1 and Step-2 respectively. Version-id is optional.

Madhukar Mohanraju
  • 2,793
  • 11
  • 28
31

Note that encodeURI is required if the filename has special chars

    await S3.copyObject({
        Bucket: bucketName,
        CopySource: encodeURI(`/${sourceBucketName}/${filename}`),
        Key: filename,
    }).promise()
jox
  • 2,218
  • 22
  • 32
radzserg
  • 1,258
  • 1
  • 13
  • 22
  • 7
    This is true and important! I just stumbled into this pitfall. I guess it's good practice to simply always `encodeURI` the CopySource. Otherwise, it might hit on you anytime. A german umlaut was enough to slam a `NetworkingError: Invalid character in header content ["x-amz-copy-source"]` at me. – jox Feb 19 '20 at 21:13
  • That's incorrect. What should be used is `encodeURIComponent` as it encodes all the characters that AWS expects to be encoded. `encodeURI` would make it impossible to access file keys that contain e.g. `+` sign. – biphobe Aug 23 '23 at 13:57
24

If you want to really move (so not just copy, but in addition remove the source file) 

const moveAndDeleteFile = async (file,inputfolder,targetfolder) => {
    const s3 = new AWS.S3();

    const copyparams = {
        Bucket : bucketname,
        CopySource : bucketname + "/" + inputfolder + "/" + file, 
        Key : targetfolder + "/" + file
    };

    await s3.copyObject(copyparams).promise();

    const deleteparams = {
        Bucket : bucketname,
        Key : inputfolder + "/" + file
    };

    await s3.deleteObject(deleteparams).promise();
    ....
}
Tobi
  • 1,702
  • 2
  • 23
  • 42
7
s3.copyObject({
  Bucket,
  CopySource: `/${Bucket}/${Key}?versionId=${versionId}`,
  Key,
}).promise()

if you do pass a versionId this is what it looks like.

took me longer than i'd care to admit to figure out.

Manuel Spigolon
  • 11,003
  • 5
  • 50
  • 73
user1664072
  • 201
  • 2
  • 6
-1

Try this:

const headCode = await S3.headObject(params).promise();

if (headCode) {
  const res = await S3.copyObject({
    CopySource: sourceBucket + '/' + sourceKey,
    Bucket: destinationBucket,
    Key: destinationKey
  }).promise();
}
Tyler2P
  • 2,324
  • 26
  • 22
  • 31
Abbu Turab
  • 1
  • 1