Spring boot Invalid UTF-8 middle byte on specific machine when sending request with Hebrew

Question

I am using Spring boot mvc and I have an issue that is reproduced only on my staging machine, but works fine locally.

I am sending the following JSON reqeust (notice Hebrew chars under "whatever" field):

{
 "messageInitiaterId":"0542258808",
 "destinationId":"0544556543",
 "whatever":"משהו",
 "pushToken":"e2eeb632-8c2c-4ad2-a163-cfeb671d1955",
 "androidVersion":"7.1.1",
 "deviceModel":"huawei nexus 6p",
 "appVersion":"1.42"
}

Locally, it returns a response and all is well. On the staging machine, this is the response returned:

{
  "timestamp": 1485591099485,
  "status": 400,
  "error": "Bad Request",
  "exception":          

    "org.springframework.http.converter.HttpMessageNotReadableException",
  "message": "Could not read document: Invalid UTF-8 middle byte 0x3f\n at [Source: java.io.PushbackInputStream@6996e7d; line: 4, column: 17]; nested exception is com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0x3f\n at [Source: java.io.PushbackInputStream@6996e7d; line: 4, column: 17]",
  "path": "/v1/IsRegistered"

}

Currently using Postman to reproduce this. The header on the request is:

"Content-Type":"application/json; charset=UTF-8"

I also tried to add the Hebrew language to the machine itself, which is running Windows 8, but it did not seem to make a difference.

Thanks in advance.

EDIT:

This is the controller, adding the suggested MediaType but still getting the same error:

@Controller
public class IsRegisteredController extends AbstractController {

private final IsRegisteredLogic logic;

@Autowired
public IsRegisteredController(IsRegisteredLogic logic) {
    this.logic = logic;
}

@ResponseBody
@RequestMapping(value = "/v1/IsRegistered", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_UTF8_VALUE)
public Response<UserDTO> isRegistered(@Valid @RequestBody IsRegisteredRequest request, HttpServletResponse response) {
    return logic.execute(request, response);
}
}

EDIT2:

Tried the suggestions below, as well as others I found online, all of which did not make a difference. This seems to be a more difficult problem than I thought.

Added to application.properties the following fields:

spring.http.encoding.charset=UTF-8
spring.http.encoding.enabled=true
spring.http.encoding.force=true
spring.datasource.connectionproperties=useUnicode=true;characterEncoding=utf-8;

As well as the following beans:

@Bean
public TomcatConnectorCustomizer tomcatConnectorCustomizer() {
    return connector -> connector.setURIEncoding("UTF-8");
}

@Bean
public Filter characterEncodingFilter() {
    CharacterEncodingFilter characterEncodingFilter = new CharacterEncodingFilter();
    characterEncodingFilter.setEncoding("UTF-8");
    characterEncodingFilter.setForceEncoding(true);
    return characterEncodingFilter;
}

All is to no avail. What's going on here? Surely Spring boot can handle Hebrew chars in requests...

Answer 1

The difference between your local development machine and the staging machine is quite obviously that they are configured with different language and text encoding settings. To work around it, explicitly specify the text encoding when starting the JVM for your application:

java -Dfile.encoding=UTF-8 -jar target/myproject-0.0.1-SNAPSHOT.jar

You can even set the following properties in application.properties to prevent Tomcat from starting if the encoding isn't configured properly:

spring.mandatory_file_encoding=UTF-8

Answer 2

In your controller method @RequestMapping, explicitly set consumes to application/json;charset=UTF-8

Sample:

@RequestMapping(value = "your_url", method = RequestMethod.POST, consumes = MediaType.APPLICATION_JSON_UTF8_VALUE)

Answer 3

Add the following lines to the file application.properties:

spring.http.encoding.charset=UTF-8 # Charset of HTTP requests and responses. Added to the "Content-Type" header if not set explicitly.
spring.http.encoding.enabled=true # Enable http encoding support.
spring.http.encoding.force=true # Force the encoding to the configured charset on HTTP requests and responses.

The last line might not be needed.

Answer 4

Are you using tomcat? If yes, you have to make sure that your tomcat's server.xml's file connector element has URIEncoding attribute which value is UTF-8.

<Connector port="8080" maxHttpHeaderSize="8192"
 maxThreads="150" minSpareThreads="25" maxSpareThreads="75"
 enableLookups="false" redirectPort="8443" acceptCount="100"
 connectionTimeout="20000" disableUploadTimeout="true" 
 compression="on" 
 compressionMinSize="128" 
 noCompressionUserAgents="gozilla, traviata" 
 compressableMimeType="text/html,text/xml,text/plain,text/css,text/ javascript,application/x-javascript,application/javascript"
 URIEncoding="UTF-8"
/>