Mock AWS s3 service in C#

Question

I'm trying to mock AWS s3 service (more specifically the upload of files to s3), for that I'm using https://docs.getmoto.org/en/latest/docs/server_mode.html , the problem is I'm getting

 error during upload","@i":"4c86b2a2","@l":"Fatal","@x":"Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.\n   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.FetchCredentials()\n   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.GetCredentials()\n   at Amazon.Runtime.DefaultInstanceProfileAWSCredentials.GetCredentialsAsync()\n   at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.S3.Internal.AmazonS3ExceptionHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext)\n   at ...,"ExceptionDetail":{"HResult":-2146233088,"Message":"Unable to get IAM security credentials from EC2 Instance Metadata Service.","Source":"AWSSDK.Core","TargetSite":"CredentialsRefreshState FetchCredentials()","ErrorType":"Sender","ErrorCode":null,"RequestId":null,"StatusCode":"0","Retryable":null,"Type":"Amazon.Runtime.AmazonServiceException"}}

I'm using the following configuration

  "AWS": {
    "Profile": "default",
    "Region": "eu-central-1",
    "ServiceURL": "http://127.0.0.1:5000/"
  },

Obs.: looks like Moto is indeed running at http://127.0.0.1:5000/

registration

        builder.Register(componentContext => 
        {
            var configuration = componentContext.Resolve<IConfiguration>();
            return configuration.GetAWSOptions();
        }).As<AWSOptions>().SingleInstance();
    }

        builder.Register(componentContext =>
            {
                var options = componentContext.Resolve<AWSOptions>();
                var client = options.CreateServiceClient<IAmazonS3>();
                return client;
            })
            .As<IAmazonS3>()
            .InstancePerDependency();

usage:

        var s3PutRequest = new PutObjectRequest
        {
            BucketName = _settings.S3BucketName,
            Key = key,
            InputStream = stream,
            CalculateContentMD5Header = true
        };

        try
        {
            var response = await _s3Client.PutObjectAsync(s3PutRequest);
            _logger.LogInformation(" Uploaded {File} - {StatusCode}", s3PutRequest.Key, response.HttpStatusCode);
        }
        catch (Exception ex)
        {
            _logger.LogCritical(ex, " error during upload");
            throw;
        }

I have moto running at: http://127.0.0.1:5000/moto-api/, There is not much documentation so I'm not sure what I should do.

Please feel free to suggest a possible easier solution for my problem.

Where are you actually creating a mock object? Share your test code. — Daniel Mann, Aug 22 '22 at 21:07
I’m not sure if I understood your comment, maybe I did not understand the library, but I wanted to run the app using the mock not for integration/unit tests, I’m actually using docker for running the moto, so no reference in the code for moto — Andre, Aug 22 '22 at 22:14
Potentially related: https://stackoverflow.com/questions/60815037/aws-dotnet-sdk-error-unable-to-get-iam-security-credentials-from-ec2-instance-m It may be that the #C SDK never tries to make the call to http://localhost, if it cannot find credentials — Bert Blommers, Aug 23 '22 at 09:24
Note that for Moto, any credentials will work - they do not have to be valid — Bert Blommers, Aug 23 '22 at 09:25

score 1 · Accepted Answer · answered Aug 22 '22 at 21:03

1

An alternate solution to Mock AWS s3 service in C# is by using MinIO. With MinIO you can use same AWS S3 client that you would use in your production application. You would just change the URL pointing to the server. That means you would use exactly the same code but just with different URLs for different environments. Please take a look at the example from docs

answered Aug 22 '22 at 21:03

Syed

96
8

That is a nice option, unfortunately looks like it does not support Athena, what is a further function needed – Andre Aug 23 '22 at 13:11

Mock AWS s3 service in C#

1 Answers1