0

I want to perform some lambda function to upload files to s3. I found that using aws-sdk taking too much space for me. so i try to upload with pure node without any libary that not build in.

this is the code that run on aws lambda function:

//My file:
const buffer = await downloadFile(url); // file is gz file downloaded as buffer

    const endpoint = `https://${myBucket}.s3.${myRegion}.amazonaws.com/${store}/${file_name}.gz`;
    const options = {
      method: "PUT",
      headers: {
        "Content-Encoding": "gzip",
        "Content-Length": Buffer.byteLength(buffer),
        "Content-Type": "application/octet-stream",
      },
    };

const data = await uploadFile(endpoint, options, buffer);

and the upload function is :

const uploadFile = (endpoint, options, buffer) => {
  const s3UploadRequest = https.request(endpoint, options, (res) => {
    console.log(`Status code: ${res.statusCode}`);
    console.log(`Status message: ${res.statusMessage}`);
    res.on("data", (chunk) => {
      console.log(`Response: ${chunk}`);
    });
    res.on("end", () => {
      if (res.statusCode > 400 && res) console.log("Upload FAILED");
      else console.log("Upload complete");
    });
  });

  s3UploadRequest.on("error", (error) => {
    console.error(`Error uploading to S3: ${error}`);
  });

  s3UploadRequest.write(buffer);
  s3UploadRequest.end();
};

and im getting "AccessDeniedAccess DeniedsomeIdidLc/largeStringWithManyChars="

The issue is when i run this code using aws-sdk it works fine , but i dont know how to make a simple upload without this libary. (also used documention aws_doc but didnt help me)

Bennyh961
  • 85
  • 1
  • 7
  • 1
    What do you mean by "taking too much space for me"? – John Rotenstein Mar 12 '23 at 22:12
  • You need to get the [role](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) and [sign](https://docs.aws.amazon.com/AmazonS3/latest/userguide/RESTAuthentication.html) the call to S3, depending on the size of the upload, you may need to call [CreateMultipartUpload](https://docs.aws.amazon.com/AmazonS3/latest/API/API_CreateMultipartUpload.html) and related APIs. Or, just use the SDK. – Anon Coward Mar 12 '23 at 22:25
  • @JohnRotenstein , the size of the packge is 85mb . if just take the relvant script it will be around 5kb (with all other files) . i know it is not the memory size for payment calculation but its crucial for me. – benny hassan Mar 13 '23 at 19:39
  • @AnonCoward i cant see how it answere what i ask. can u provide me an example ? – benny hassan Mar 13 '23 at 19:40
  • My attempt was to guide to a possible solution, I'm not going to provide a working example, perhaps someone else can. It's a non-trivial amount of work. Any solution I write for production purposes would use the SDK instead of recreating it. – Anon Coward Mar 13 '23 at 20:20
  • If you want to write it yourself, see: [Authenticating Requests (AWS Signature Version 4) - Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) However, it is much better to use the AWS SDK that is already available. – John Rotenstein Mar 13 '23 at 22:55
  • is not a simpler way ? for simple uses we need to use all the libary? i have some lambda functions that each use different single function from aws-sdk. uploading the entire packges seems not so efficent. – Bennyh961 Mar 17 '23 at 08:51

0 Answers0