AWS SDK S3 node.js connect to local MinIO server

Question

I have application server written in Node.js which upload files to AWS S3 storage. For that I'm using https://www.npmjs.com/package/aws-sdk and when I'm connecting to and uploading to my AWS production storage it is working fine. However during development I want to upload files to local MinIO server (create from docker image https://hub.docker.com/r/minio/minio/):

docker run -p 9000:9000 -p 9001:9001 \
  quay.io/minio/minio server /minio --console-address ":9001"

MinIO server itself is working fine, I'm able to login there locally, create buckets, etc. However when I tried to upload file to my local MinIO storage using AWS SDK for node.js I'm unable to do so:

import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3';

#s3Client = new S3Client({
    credentials: {
        accessKeyId: config.getAwsStorageAccessKeyId(), // from env variables
        secretAccessKey: config.getAwsStorageSecretAccessKey(),
    },
    ...(globalConfig.isReleaseVersion() && {
        endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001"
    }),
});

And later I'm trying to upload object:

await this.#s3Client.send(new PutObjectCommand({
    Bucket: config.getAwsStorageWebflowBucketName(), // from env "test" value
    Key: 'example.json',
    Body: body, // a buffer
}));

I'm receiving error:

ERROR EndpointError: Custom endpoint `test.127.0.0.1://9001/` was not a valid URI

However I'm able to reach address http://test.localhost:9001/ from my browser. When I try to pass directly value 'http://localhost:9001/' as an endpoint to S3Client configuration, I'm getting error:

ERROR Error: getaddrinfo ENOTFOUND test.localhost

Looks like port value is ignored my SDK. How can this be fixed?

<Yes, I know there is separate MinIO SDK - I want to use one provided by AWS, existence of endpoint parameter in configuration implies that it should be possible>

score 3 · Answer 1 · edited Jun 22 '23 at 17:56

All you need to do is set s3ForcePathStyle config option to true. Alternatively (in sdk V3) it can have name forcePathStyle. In my code I do it this way:

const awsConfig: S3.Types.ClientConfiguration = {
  s3ForcePathStyle: true
};

I understand that there's a similar answer but it did resolve my issue.

Additional options that I also supplied:

sslEnabled: false
endpoint: http://localhost:port/
accessKeyId: dummy
secretAccessKey: dummydummy

score 0 · Answer 2 · answered Dec 08 '22 at 15:20

0

Have you tried the option "s3ForcePathStyle" ? I had to use this config to make it work on a local environment

  options.s3ForcePathStyle = true

See https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Config.html#s3ForcePathStyle-property

So in your code

...(globalConfig.isReleaseVersion() && {
    endpoint: config.getAwsStorageEndpoint(), // from env "http://localhost:9001",
    s3ForcePathStyle: true,
}),

answered Dec 08 '22 at 15:20

gladix

302
2
11

Yes I'm still getting `ERROR Error: getaddrinfo ENOTFOUND test.localhost` - port is still missing – Furman Dec 12 '22 at 08:06

score 0 · Answer 3 · answered Dec 15 '22 at 03:08

0

the listener endpoint for S3 API is port 9000, 9001 is for dashboard, also remove region when using Minio, because I think Minio doesn't need it. by so, sdk will not give a prefix 'test' for the endpoint.

answered Dec 15 '22 at 03:08

V.Andri

12

score 0 · Accepted Answer · answered Jun 03 '23 at 10:18

http://localhost:9001 is not the correct URL

9001 is the one where you access the minio console
9000 is the one where S3 API is served

Console output when running minio

Try following & it should work

const s3Client = new S3Client({
  region: "region_set_in_minio",
  credentials: {
    accessKeyId: "minio_access_key",
    secretAccessKey: "minio_secret_access_key",
  },
  endpoint: "http://127.0.0.1:9000",
});

You can set minio region from minio console.

Go to Administrator -> Settings -> Region

Change minio region

score 0 · Answer 5 · answered Jun 22 '23 at 18:10

I saw here correct answers (@gladix and @p2t2p - I have upvoted, of course). But I think we should make it a bit clearer:

AWS SDK V3

In the question I see:

import { S3Client } from '@aws-sdk/client-s3';

It means he uses V3 of the AWS SDK. This is important, because in V3 the config passed to the constructor of the client is of type S3ClientConfig.

import { S3ClientConfig } from "@aws-sdk/client-s3";

And there, there is a member forcePathStyle: boolean.

AWS SDK V2

However, if user uses V2 (like import * as AWS from 'aws-sdk'), then the config passed to the constructor of the client is of a different type, ClientConfiguration.

import { ClientConfiguration } from "aws-sdk/clients/s3";

And there, there is a member s3ForcePathStyle: boolean.

Hope that makes things clearer :)

AWS SDK S3 node.js connect to local MinIO server

5 Answers5

Linked