How to download a list of files from AWS S3 with C# / .Net to my device?

Question

I wish to download multiple images stored in S3. But for now it would be enough if I could download only one. I have the information of the paths of objects.

When I run the following code, I get this error:

Error encountered ***. Message:'Access Denied' when reading object ...

I first make an AmazonS3Client object based on my keys and access-config to connect to the server and then make a request based on storage path of my object and then somehow read the data and save it in my device.

By the way, in our AWS S3 we have “role”s, meaning that for downloading directly from S3 from the web platform my user in AWS is not enough to access to those objects, I should also change the role to the one which has access to those images after I log in.

using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
using System;
using System.IO;
using System.Threading.Tasks;

namespace Amazon.DocSamples.S3
{
    class GetObjectTest
    {
        private const string bucketName = "My_Bucket_Name";
        private const string keyName = "123/456/789.jpg";
        // Specify your bucket region (an example region is shown).
        private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2;
        private static IAmazonS3 client;

        static string accessKey = "***AccessKey***";   // "YourAccessKeyHere";
        static string secretKey = "***SecretKey***"; 

        public static void Main()
        {
            client = new AmazonS3Client(accessKey,secretKey,bucketRegion);
            ReadObjectDataAsync().Wait();
        }

        static async Task ReadObjectDataAsync()
        {
            string responseBody = "";
            GetObjectRequest request = new GetObjectRequest
            {
                 BucketName = bucketName,
                 Key = keyName
            };
            using (GetObjectResponse response =     awaitclient.GetObjectAsync(request))
            using (Stream responseStream = response.ResponseStream)
            using (StreamReader reader = new StreamReader(responseStream))
            {
                string title = response.Metadata["x-amz-meta-title"]; // Assume you have "title" as medata added to the object.
                string contentType = response.Headers["Content-Type"];
                Console.WriteLine("Object metadata, Title: {0}", title);
                Console.WriteLine("Content type: {0}", contentType);

                responseBody = reader.ReadToEnd(); // Now you process the response body.
             }
        }
    }
}

I found the following link related to access problems: https://aws.amazon.com/premiumsupport/knowledge-center/s3-troubleshoot-403/

the issue that this code doesn't work and I can't download the image. — Iraj, Jun 16 '21 at 12:39
What is the error you are seeing? Could it be a permissions related issue within the S3 bucket? — CB Hoffman, Jun 16 '21 at 16:23
I did a major modification to the post and added what exact error I get. The problem is now more clear. — Iraj, Jun 17 '21 at 14:48

Prayag Sagar · Answer 1 · 2021-06-26T10:34:32.257

I have modified your code a bit and was able to successfully execute a GetObjectAsync request.

using Amazon;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.SecurityToken;
using Amazon.SecurityToken.Model;
using System.Drawing;
using System.Threading.Tasks;

namespace DownloadFromS3
{
    class GetObjectTest
    {
        private static IAmazonS3 client;

        private const string regionName = "us-west-2";
        private const string accessKey = "accessKey";
        private const string secretKey = "secretKey";
        private const string roleSessionName = "roleSessionName";
        private const string roleArn = "arn:aws:iam::123456789:role/yourRoleName";
        private const string bucketName = "bucketName";
        private const string keyName = "folder/keyName.jpg";

        public static void Main(string[] args)
        {
            var assumeRoleCredentials = FetchAssumeRoleCredentials();
            client = new AmazonS3Client(assumeRoleCredentials, RegionEndpoint.GetBySystemName(regionName));
            ReadObjectDataAsync().Wait();
        }

        static async Task ReadObjectDataAsync()
        {
            var request = new GetObjectRequest()
            {
                BucketName = bucketName,
                Key = keyName,
                ResponseHeaderOverrides = new ResponseHeaderOverrides
                {
                    ContentType = "image/jpg"
                }
            };

            using (var response = await client.GetObjectAsync(request))
            {
                using (var bitmap = new Bitmap(response.ResponseStream))
                {
                    bitmap.Save(@"C:\Image.jpg");
                }
            }
        }

        static Credentials FetchAssumeRoleCredentials()
        {
            var assumeRoleRequest = new AssumeRoleRequest
            {
                RoleSessionName = roleSessionName,
                RoleArn = roleArn
            };

            var securityTokenServiceClient = new AmazonSecurityTokenServiceClient(accessKey, secretKey, RegionEndpoint.GetBySystemName(regionName));
            var assumeRoleResponse = securityTokenServiceClient.AssumeRoleAsync(assumeRoleRequest).Result;
            return assumeRoleResponse.Credentials;
        }
    }
}

The code is more or less the same. What I suspect is

the regionName value being used AWS region code
AWS credentials being used not having enough permissions to read data from S3

Edit :

Updated code to save the image locally. (Make sure to set the necessary write permissions)
Updated code to create S3 Client using assumed role credentials. Your access key and secret key should have access to assume role and the assumed role should have access to S3 bucket.

Thank you for your answer. But I get error for the line of defining client. ```client = new AmazonS3Client(... ```. and the error is ```Exception Unhandled -- System.TypeInitializationException: The type initializer for Amazon.AWSConfigs threw an exception. --- Inner Exception FileNotFoundException: Could not load file or assembly System.Configuration.ConfigurationManager,``` Also I don't see where you have attempted to save the file in the local device. — Iraj, Jun 17 '21 at 10:22
for the exception you are receiving, try this. https://stackoverflow.com/a/59935998/6344875 I will update my answer to write contents to a file — Prayag Sagar, Jun 17 '21 at 14:20
Dear @PrayagSagar, I made a major modification in my post. Previous problems seem to be solved by fixing the ```client``` object. Now I don't get runtime error and I get ```access denied``` error as I wrote in the main post. — Iraj, Jun 17 '21 at 14:51
Since you are getting access denied, its because the aws credentials which you are using to access the resource is not having enough permissions to read the file. Can you ensure you use credentials which has read access for S3. One way to check this is by configuring AWS profile locally and using AWS CLI to read the S3 file — Prayag Sagar, Jun 17 '21 at 15:15
Thank you @PrayagSagar. I tried AWS CLI and implemented this command successfully ```aws s3api list-objects --bucket DOC-EXAMPLE-BUCKET --prefix exampleprefix```. But I run it when chosen my particular "role" to access the console. I wish there was a command or property to choose "role" when you make the ```client``` object. — Iraj, Jun 21 '21 at 14:48
@Iraj If you really want to access S3 objects using roles, you can consider AssumeRole. See https://docs.aws.amazon.com/sdkfornet/v3/apidocs/index.html?page=SecurityToken/MSecurityTokenServiceAssumeRoleAssumeRoleRequest.html — Prayag Sagar, Jun 21 '21 at 16:52
thank you. I read that article and more about AssumeRole but Ii don't know how I can use it when I have only a role that has has access but I can't switch to that role in my .NET app. Those explanations seem more about how to create a role or tag or token. — Iraj, Jun 22 '21 at 12:25

How to download a list of files from AWS S3 with C# / .Net to my device?

1 Answers1

Linked