3

I'm using aws-sdk module in my node app,
specifically the S3 library, to write to a ceph cluster.

The application upload fixed size files periodically.
The ceph perform sharding sometimes, which cause the http upload request to stall for 500 seconds.
How can I cancel those requests for buckets that performing sharding?
How can I set a timeout for, say 2 seconds, for all the uploads?

natdev
  • 507
  • 1
  • 8
  • 19

2 Answers2

4

After searching the web for a long time, I found this abort function.

So, if it help anyone, this is how I implemented it:

const upload = (bucket, key, body, timeout = 2000) => {
  const request = this.s3.upload({Bucket: bucket, Key: key, Body: body})
  setTimeout(request.abort.bind(request), timeout)
  return request.promise()
}

If timeout passed, then an Error is thrown with RequestAbortedError code.

natdev
  • 507
  • 1
  • 8
  • 19
1

You can use timeout in httpOptions. example:

var s3 = new AWS.S3({apiVersion: '2006-03-01', httpOptions: {timeout: 2000}});
Seena Fallah
  • 560
  • 4
  • 12
  • I did it, it didn't work... as documented, it states that it closes `idle` connection where there is no traffic, but as it turns out, the client doesn't close the connection upon waiting for the response – natdev Aug 19 '20 at 06:11
  • @natdev Maybe there is a chunk transferring. If you want to timeout on the whole request I think you should implement it by node js and aws sdk doesn’t have this option. – Seena Fallah Aug 19 '20 at 15:37
  • I found a way to do it, check my answer – natdev Aug 20 '20 at 08:42