2

I've created two sample applications for testing file upload, one with MVC and one with WebFlux (Spring Boot 2.4.0-M2 because they have implemented zero-copy non-blocking transferTo in that version). I did not configure anything on the servers (Netty and Tomcat).

The WebFlux code:

public Mono<Void> uploadFile(Mono<FilePart> filePartMono) {
    return filePartMono.flatMap(
        filePart -> filePart.transferTo(
            STORAGE_PATH.resolve(UUID.randomUUID().toString())
        )
    );
}

The MVC code:

public void uploadFile(MultipartFile multipartFile) throws IOException {
    multipartFile.transferTo(
        STORAGE_PATH.resolve(UUID.randomUUID().toString())
    );
}

I've created two JMeter tests, one is testing one big file upload (2GB ISO), other is testing with 1000 threads using small files (1MB documents).

MVC file upload outperforms WebFlux around 10 times in speed, and never throws errors. WebFlux on the other hand sometimes never completes the file upload but does not throw exception either (some kind of deadlock?), or in other cases I get a no more space left on device error. I investigated the latter and it turned out that the files are duplicated/temporarly stored(?) in /tmp/spring-multipart folder and that drive gets full fast. Even if the file upload manages to complete successfully, these files are not deleted. I couldn't find anything about this issue by Googling. (I also couldn't find where these files are written in Windows so I'm missing 30GB space right now. :))

As I've observed, MVC writes a dozen files at a time, while WebFlux writes little chunks to all. Should I publish on an other Scheduler to reduce the context switches or it will introduce an other bottleneck?

What am I missing here? What performance should I be expecting to be normal at all? On localhost I presumed (nearly) OS copy speed.

Edit: removed the publishOn(Schedulers.boundedElastic()) from the WebFlux code snippet.

Edit 2: I also tried to use FilePart's content() method to read the DataBuffers, it filled up my /tmp folder as well, but this time the temp files were in nio-file-upload. Memory consumption was way more than MVC.

Peter
  • 323
  • 1
  • 16
  • 46
  • Did you try it without the `publishOn`? What was the reason for putting the request on a separate thread? The reactor framework is meant to do concurrent processing on limited or a single thread. Nice feature on Spring Boot 2.4.0-M2. – K.Nicholas Aug 21 '20 at 17:54
  • @K.Nicholas yes, same results. I also tried to switch [this](https://github.com/spring-projects/spring-framework/blob/51cc719c004da241c5a29f63d81c1203c2d00d26/spring-web/src/main/java/org/springframework/http/codec/multipart/DefaultPartHttpMessageReader.java#L187) flag on but doesn't seem to do anything at all. – Peter Aug 21 '20 at 18:06
  • Hmm, well, I can't say I played around with WebFlux in this manner. I suspect it was not a major priority for the WebFlux team. It's tempting to suggest that you call `multipartFile.transferTo` from the WebFlux request but who knows what sort of dependency issues will be caused by it. So, make a separate URL to handle file uploads and service it with a MVC process for now. – K.Nicholas Aug 21 '20 at 18:18
  • Made a simple web flux controller spring boot version 2.2.4.RELEASE `@PostMapping("/upload")` `public Mono process(@RequestPart("files") Flux filePartFlux) {` `return filePartFlux.flatMap(it -> it.transferTo(Paths.get("/tmp/test/"+ UUID.randomUUID())))` `.then(Mono.just("OK"));` `}` Contrary to your post, this performs surprisingly well. I am not sure about the changes in `2.4.x` but hope this info helps. – Himanshu Chaudhary Aug 28 '20 at 20:39
  • @HimanshuChaudhary thank you for testing it! The reason why it performs well is because it's blocking, see [here](https://github.com/spring-projects/spring-framework/blob/b16f6fa4564658615363f9206edf726a492b6083/spring-web/src/main/java/org/springframework/http/codec/multipart/SynchronossPartHttpMessageReader.java#L467), 2.4.0 is the first non-blocking version. – Peter Aug 28 '20 at 20:40
  • 1
    @Peter you are right, I tried to hit the upload endpoint using postman for 1000 requests. Went fine for 85 requests on 86th it just stopped doing anything... – Himanshu Chaudhary Aug 28 '20 at 21:20

0 Answers0