Wiremock withBodyFile OutOfMemoryError for large files

Question

In the following stub request, I use withBodyFile which loads large files (>300MB) from specific locations on local disk using streams:

public void mockGetUrlContent(String url) {
    stubFor(get(urlEqualTo(url))
            .willReturn(ok()
                    .withBodyFile(FilenameUtils.getName(url))));
}

Once the stub is being called, wiremock tries to complete a served event and in order to log the response it uses the from method below:

  public static LoggedResponse from(Response response) {
    return new LoggedResponse(
        response.getStatus(),
        response.getHeaders() == null || response.getHeaders().all().isEmpty()
            ? null
            : response.getHeaders(),
        response.getBody(),
        response.getFault());
  }

When reached to wiremock's Response getBody method it converts the stream in to byte[] and in that points it blows on OutOfMemoryError (Java heap space).:

  public byte[] getBody() {
    try (InputStream stream = bodyStreamSource == null ? null : getBodyStream()) {
      return stream == null ? null : ByteStreams.toByteArray(stream);
    } catch (IOException e) {
      throw new RuntimeException(e);
    }
  }

Any ideas about what am I doing wrong?

Your help is highly appreciated!

What you are doing wrong is trying to convert an input stream for a >300MB file into a `byte[]`. Solution: don't. Your `getBody()` method should return the `InputStream`. — user207421, Jun 15 '22 at 08:24
man, I don't try to do it , its wiremock framework that does it, maybe its a bug in their framework or wrong configuration — DeJaVo, Jun 15 '22 at 08:25
This is hard-coded behaviour in WireMock at the moment unfortunately - it reads the entire response body into the request journal so that it's present for later verification, but with very large bodies like this that's not useful. If you open a GH issue I'll work on adding a --truncate-large-response-bodies parameter or something similar so that you can limit the amount it keeps in memory. — Tom, Jun 16 '22 at 09:49
@Tom I opened a GH issue following your request: https://github.com/wiremock/wiremock/issues/1905 — DeJaVo, Jun 16 '22 at 13:43
@Tom I added a workaround for this issue until fixed, see in the question's body — DeJaVo, Jun 16 '22 at 13:54
@Tom solved it and closed the issue. https://github.com/wiremock/wiremock/commit/34761d64009534d951a24bd0aa4b152b93ead00c — DeJaVo, Sep 15 '22 at 19:05
Please do not add answers to the question body itself. Instead, you should add it as an answer. [Answering your own question is allowed and even encouraged](https://stackoverflow.com/help/self-answer). I've rolled back the edit, feel free to add it as an answer though! — Adriaan, Sep 16 '22 at 09:48

score 1 · Accepted Answer · answered Sep 16 '22 at 09:43

1

WireMock 2.34.0 has a solution to this.

To ensure that logged response bodies are limited to a particular size you need to start WireMock with a new configuration parameter specifying the max size in bytes

When running standalone add the CLI parameter:

--logged-response-body-size-limit=1000000

When running in Java add the following to the configuration builder:

wireMockConfig().maxLoggedResponseSize(1000000)

answered Sep 16 '22 at 09:43

Tom

3,471
21
14

first of all thank you for a quick fix. I've one question about the new behavior : what will happen if I have a file weigh about 300mb and I set the max logged response to 20m max, will it trunct the rest 280m and the file body will be limited to 20m? – DeJaVo Sep 17 '22 at 13:32
1

Yes, it'll still serve the whole body back to the client but it'll only capture the first 20m in the log. – Tom Sep 19 '22 at 15:24
Tom, again thank you for the quick address, i"ve just used your fix and works perfectly. – DeJaVo Sep 21 '22 at 07:49

Wiremock withBodyFile OutOfMemoryError for large files

1 Answers1