Upgrading S3Client to S3AsyncClient how to get Publisher?

Question

I am upgrading S3Client in aws cloud service to S3AsyncClient.

I have this function to convert to async:

public PutObjectResponse uploadFileByUrl(String fileUrl, String builderId, PbModel category, String categoryId)

    URL url = new URL(fileUrl);
    String[] fileNameArray = url.getFile().split("\\.");
    var uniqueFileName = prepareFileName(fileNameArray[fileNameArray.length -1]);

    URLConnection connection = url.openConnection();

    long contentSize = connection.getContentLengthLong();
    InputStream inputStream = connection.getInputStream();

    return s3Client.putObject(myObjectRequestBuild, RequestBody.fromInputStream(inputStream, contentSize));
}

I have this function to convert to async:

public CompletableFuture<PutObjectResponse> uploadFileByUrl(String fileUrl, String builderId, PbModel category, String categoryId)

    URL url = new URL(fileUrl);
    String[] fileNameArray = url.getFile().split("\\.");
    var uniqueFileName = prepareFileName(fileNameArray[fileNameArray.length -1]);

    URLConnection connection = url.openConnection();

    long contentSize = connection.getContentLengthLong();
    InputStream inputStream = connection.getInputStream();

    return asyncClient.putObject(myObjectRequestBuild, AsyncRequestBody.fromPublisher(???));
}

As you can see in second method above when I convert the first function above to async I need to use AsyncRequestBody instead of RequestBody. AsyncRequestBody doesn't have fromInputStream method but it have fromPublisher method that I want to use, the fromPublisher method get as parameter type of Publisher.

So my question is how to convert my inputStream into an Publisher?

Consider to use `fromFile()` instead: https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/async/AsyncRequestBody.html#fromFile-java.io.File- — Artem Bilan, Oct 12 '21 at 19:28

Ermiya Eskandary · Accepted Answer · 2021-10-17T12:31:22.527

AsyncRequestBody doesn't have fromInputStream method

Correct however, it has a plethora of other ways to create an AsyncRequestBody:

Considering the above, you have a few solutions:

Convert the InputStream to a byte array using IOUtils.toByteArray(inputStream) (or in Java 9+, inputStream.readAllBytes()) and then use fromBytes directly
As above but then convert the byte[] to a ByteBuffer using ByteBuffer.wrap(byteArray) and then use fromByteBuffer
Create a new File object specifying a filename, copy the contents of the InputStream to the file's FileOutputStream using IOUtils.copy(), and then use fromFile(File file)
As above but instead of providing the File object, provide its path to fromFile(Path path) after you've written to it's FileOutputStream
Convert the InputSteam to a Publisher<ByteArray> using DataBufferUtils.readByteChannel from the Spring Framework, Akka StreamConverters etc. and then use fromPublisher
Convert the InputStream to a UTF-8 encoded String then use fromString(String string) (no need to specify the Charset if it is UTF-8 encoded)
Convert the InputStream to a non-UTF-8 encoded String then use fromString(String string, Charset cs), specifying the CharSet

Of course, some of the above are plain redundant in your case e.g. fromFile(Path path) is for files that you've already stored & converting the InputSteam to a Publisher<ByteArray> is going to be a pain but I've included all possible solutions for completeness.

I would approach this using solution #1, resulting in the cleanest, simplest code out of the above.

Convert the InputStream to byte[] using inputStream.readAllBytes() and then use AsyncRequestBody.fromBytes(...).

This should work:

public CompletableFuture<PutObjectResponse> uploadFileByUrl(String fileUrl, String builderId, PbModel category, String categoryId)
    URL url = new URL(fileUrl);
    String[] fileNameArray = url.getFile().split("\\.");
    var uniqueFileName = prepareFileName(fileNameArray[fileNameArray.length -1]);

    URLConnection connection = url.openConnection();

    long contentSize = connection.getContentLengthLong();
    InputStream inputStream = connection.getInputStream();

    byte[] fileByteArray = inputStream.readAllBytes();

    return asyncClient.putObject(myObjectRequestBuild, AsyncRequestBody.fromBytes(fileByteArray));
}

Upgrading S3Client to S3AsyncClient how to get Publisher?

1 Answers1