Binary data in body is corrupted when sending HTTP POST in JMeter

Question

I have a local web app with an HTTP POST endpoint. This endpoint receives emails sent from SendGrid.

I have saved the body of a previous email with attachments which I know works, as requestBody.bin.

I'm trying to replicate this HTTP POST request using JMeter.

I have this Beanshell Preprocessor script:

FileInputStream in = new FileInputStream("C:\\Projects\\centaurjmeter\\src\\InboundEmails\\requestBody.bin");
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] buffer = new byte[999024]; //Value of Content-Length from when I sent the actual email. So this should be more than enough just for the HTTP POST body.
for (int i; (i = in.read(buffer)) != -1; ) {
    bos.write(buffer, 0, i);
}
in.close();
byte[] requestBody = bos.toByteArray();
bos.close();

vars.put("requestBody", new String(requestBody, "ISO-8859-1"));

and I'm using ${requestBody} in the body of the HTTP POST request:

This is what my JMeter test looks like:

In my HTTP POST action method, I'm saving the attachments of the email to file.

The pdfs seem to be slightly corrupted, only part of the text on each page is showing. The .png files do not even open.

I've tried using:

vars.put("requestBody", new String(requestBody));
vars.put("requestBody", new String(requestBody, "Windows-1252")); //ANSI
vars.put("requestBody", new String(requestBody, "ISO-8859-1")); //default encoding

But none of them work. "ISO-8859-1" results in the least amount of corruption, in that at least some text appears in the pdfs (but the .png files don't work at all).

I can't use a HTTP Raw Request Sampler because that doesn't work with https.

How can I get the bytes from requestBody.bin and send them in my HTTP POST body correctly?

UPDATE

I read https://stackoverflow.com/a/41892324/2063755 and tried sending requestBody.bin as a file with the request in the "Files Upload" tab, but I get the same result as using the Beanshell Preprocessor script.

UPDATE

The above link actually did help me solve the problem. I just had to add this extra TE header:

Answer 1

You need to ensure that the same encoding is being applied everywhere, I would recommend sticking to UTF-8

1. Run JMeter providing file.encoding=UTF-8 property, it can be added to system.properties file (lives in "bin" folder of your JMeter installation)

2. Since JMeter 3.1 you should be using JSR223 Test Elements and Groovy language for scripting so you can use the following __groovy() function directly in the "Body Data" tab of your HTTP Request sampler:

   ${__groovy(new File('C:\\Projects\\centaurjmeter\\src\\InboundEmails\\requestBody.bin').getText('UTF-8'),)}
   

   More information on Groovy scripting in JMeter: Apache Groovy - Why and How You Should Use It

3. Assuming everything goes well your "local web app" should see the identical file, it can be double-checked using a 3rd-party sniffer tool like Wireshark. Make sure to use UTF-8 for parsing the file in your app as well.