AWS C# cant figure out how to upload zip stream to S3 bucket

Question

I am currently in the process of trying to put 3rd party API data into a S3 bucket via a C# AWS Lambda function. The data from the 3rd party comes in a zip file that I have been able to save locally fine so I am assuming that is working as I need it to. The next step is to upload this data to a S3 bucket but via a stream rather than saving the file locally and uploading etc.

public static async void PutObjectAsync()
    {
        try
        {
            using (var client = new AmazonS3Client(Amazon.RegionEndpoint.EUWest2))
            {

                var ms = new MemoryStream();
                GetStream().CopyTo(ms);    //GetStream() returns the api data stream (Stream object).

                var request = new PutObjectRequest()
                {
                    BucketName = "*BUCKETNAME*",
                    Key = "data.zip",
                    InputStream = ms

                };

                var response = await client.PutObjectAsync(request);                  

            }
        }
        catch (AmazonS3Exception e)
        {
            Console.WriteLine(
                    "Error encountered ***. Message:'{0}' when writing an object"
                    , e.Message);
        }
        catch (Exception e)
        {
            Console.WriteLine(
                "Unknown encountered on server. Message:'{0}' when writing an object"
                , e.Message);
        }

        Console.ReadKey();
    }
 }

I get a object reference not set to reference of an object error on the PutObjectAsync function call. I have inspected the request and the memory stream appears to be loaded as expected from the CopyTo method. I am aware that buckets have issues with files over 5MB but the zip file is a lot smaller than 5MB.

This is my first lambda function so any advice would be welcomed. Thanks in advance.

Which line of code gives the exception? How this code of lambda is being invoked? — Chetan, Aug 26 '18 at 19:57
@ChetanRanpariya This is not currently in a lambda function but in a console app, just to get it running. So PutObjectAsync() is called from Main. Thanks var response = await client.PutObjectAsync(request); throws the exception and it is caught in the Exception e catch not the AmazonS3 — William James, Aug 26 '18 at 19:59
Have you got the AWS Credentials in the right place? https://stackoverflow.com/a/43243290/784908 — berkeleybross, Aug 26 '18 at 20:07
Are you able to debug the code. Debugging should tell you which line caused the exception to be thrown to `catch(Exception e)` . It would be also good if you can share code of main function and how are you actually uploading the file etc. — Chetan, Aug 26 '18 at 20:08
@berkeleybross Credentials was the issue I created created a BasicAWSCredentials object with my keys and now it works! thanks you so much. — William James, Aug 26 '18 at 20:35

score 0 · Accepted Answer · answered Aug 26 '18 at 21:09

As diagnosed in the comments, AWS requires credentials to allow access to the S3 Bucket. Unfortunately it doesnt mention in the documentation that these are required, and are by default looked up from disk as described at https://stackoverflow.com/a/43243290/784908.

AWS C# cant figure out how to upload zip stream to S3 bucket

1 Answers1