3

The title basically explains itself.

I have a REST endpoint with VertX. Upon hitting it, I have some logic which results in an AWS-S3 object.

My previous logic was not to upload to S3, but to save it locally. So, I can do this at the response routerCxt.response().sendFile(file_path...).

Now that the file is in S3, I have to download it locally before I could call the above code.

That is slow and inefficient. I would like to stream S3 object directly to the response object.

In Express, it's something like this. s3.getObject(params).createReadStream().pipe(res);.

I read a little bit, and saw that VertX has a class called Pump. But it is used by vertx.fileSystem() in the examples.

I am not sure how to plug the InputStream from S3'sgetObjectContent() to the vertx.fileSystem() to use Pump.

I am not even sure Pump is the correct way because I tried to use Pump to return a local file, and it didn't work. 

    router.get("/api/test_download").handler(rc -> {
        rc.response().setChunked(true).endHandler(endHandlr -> rc.response().end());
        vertx.fileSystem().open("/Users/EmptyFiles/empty.json", new OpenOptions(), ares -> {
            AsyncFile file = ares.result();
            Pump pump = Pump.pump(file, rc.response());
            pump.start();
        });
    });

Is there any example for me to do that?

Thanks

william
  • 7,284
  • 19
  • 66
  • 106

1 Answers1

6

It can be done if you use the Vert.x WebClient to communicate with S3 instead of the Amazon Java Client.

The WebClient can pipe the content to the HTTP server response:

webClient = WebClient.create(vertx, new WebClientOptions().setDefaultHost("s3-us-west-2.amazonaws.com"));

router.get("/api/test_download").handler(rc -> {
HttpServerResponse response = rc.response();
response.setChunked(true);
webClient.get("/my_bucket/test_download")
 .as(BodyCodec.pipe(response))
  .send(ar -> {
    if (ar.failed()) {
      rc.fail(ar.cause());
    } else {
      // Nothing to do the content has been sent to the client and response.end() called
    }
  });
});

The trick is to use the pipe body codec.

tsegismont
  • 8,591
  • 1
  • 17
  • 27
  • Thanks. That took me a step in the right direction. Is there any header that I should add to the `response`'s `headers`? When I didn't add any extra `header`, it doesn't download it, when I added `Content-Type: binary/octet-stream`, it is downloading a blank file. I tested the temp url beforehand, and it does download it. So, that's probably not a cause. – william Aug 06 '18 at 18:15
  • Seems like a separate problem. It would be better to post a complete reproducer on the Vert.x forum. – tsegismont Aug 07 '18 at 07:46
  • hey.. I figured it out.. it was a novice mistake, nth important. thanks for the help. – william Aug 09 '18 at 01:07
  • Thanks for the heads-up! – tsegismont Aug 09 '18 at 07:56