In Ktor, how can I stream an InputStream into a HttpClient Request's body?

Question

I am using Ktor 1.2.2 and I have an InputStream object that I want to use as the body for an HttpClient request I make down the line. Up until Ktor 0.95 there was this InputStreamContent object that seemed to do just that but it has been removed from Ktor at version 1.0.0 (couldn't figure out why unfortunately).

I can make it work using a ByteArrayContent (see code below) but I'd rather find a solution that does not require loading the entire InputStream into memory...

ByteArrayContent(input.readAllBytes())

This code is a simple test case that emulate what I'm trying to achieve:

val file = File("c:\\tmp\\foo.pdf")
val inputStream = file.inputStream()
val client = HttpClient(CIO)
client.call(url) {
      method = HttpMethod.Post
      body = inputStream // TODO: Make this work :(
    }
// [... other code that uses the response below]

Let me know if I missed any relevant information,

Thanks!

score 5 · Answer 1 · edited Jan 14 '23 at 11:01

One way to achieve that is to create a subclass of OutgoingContent.WriteChannelContent, and set it to the body of your post request.

An example could look like this:

class StreamContent(private val pdfFile:File): OutgoingContent.WriteChannelContent() {
    override suspend fun writeTo(channel: ByteWriteChannel) {
        pdfFile.inputStream().copyTo(channel, 1024)
    }
    override val contentType = ContentType.Application.Pdf
    override val contentLength: Long = pdfFile.length()
}


// in suspend function
val pdfFile = File("c:\\tmp\\foo.pdf")
val client = HttpClient()
val result = client.post<HttpResponse>("http://upload.url") {
    body = StreamContent(pdfFile)
}

@Peter and Stefan that saved my day. Thx! – otembajelle Aug 02 '22 at 08:26 — otembajelle, Aug 02 '22 at 08:26

Benjamin Charais · Accepted Answer · 2019-06-21T16:54:19.310

The only API (that I have found...) in Ktor 1.2.2 is potentially sending a multi-part request, which would require your receiving server to be able to handle this, but it does support a direct InputStream.

From their docs:

val data: List<PartData> = formData {
    // Can append: String, Number, ByteArray and Input.
    append("hello", "world")
    append("number", 10)
    append("ba", byteArrayOf(1, 2, 3, 4))
    append("input", inputStream.asInput())
    // Allow to set headers to the part:
    append("hello", "world", headersOf("X-My-Header" to "MyValue"))
}

This being said, I don't know how it works internally and likely still loads to memory the entirety of the stream.

The readBytes method is buffered, so wont take up the entirety of memory.

inputStream.readBytes()
inputStream.close()

As a note, you are still required to close the inputStream with most methods on InputStreams

Ktor Source: https://ktor.io/clients/http-client/call/requests.html#the-submitform-and-submitformwithbinarydata-methods

Kotlin Source: https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.io/java.io.-input-stream/index.html

Currently, `inputStream.asInput()` returns an implementation of `Input`, that seems to handle stream closing — Peter, Sep 17 '21 at 12:11

score 0 · Answer 3 · answered Feb 23 '20 at 14:31

0

This is what works for me on Ktor 1.3.0 to upload files to GCP:

client.put<Unit> {
    url(url)
    method = HttpMethod.Put
    body = ByteArrayContent(file.readBytes(), ContentType.Application.OctetStream)
}

answered Feb 23 '20 at 14:31

Sir Codesalot

7,045
2
50
56

2

You are smart to read the whole thing in memory. – Mihai Nov 30 '20 at 04:12

score 0 · Answer 4 · answered Jan 16 '23 at 10:52

I couldn't manage to get the solution from @stefan to work. Here's my alternative example (targeting Ktor 2.2.2):

class StreamContent(private val pdfFile: File) : OutgoingContent.ReadChannelContent() {
    override fun readFrom(): ByteReadChannel = pdfFile.readChannel()
    override val contentType = ContentType.Application.Pdf
    override val contentLength: Long = pdfFile.length()
}

// in suspend function
val pdfFile = File("c:\\tmp\\foo.pdf")
val client = HttpClient()
val result = client.post<HttpResponse>("http://upload.url") {
    setBody(StreamContent(pdfFile))
}

In Ktor, how can I stream an InputStream into a HttpClient Request's body?

4 Answers4