MockMvc no longer handles UTF-8 characters with Spring Boot 2.2.0.RELEASE

Question

After I upgraded to the newly released 2.2.0.RELEASE version of Spring Boot some of my tests failed. It appears that the MediaType.APPLICATION_JSON_UTF8 has been deprecated and is no longer returned as default content type from controller methods that do not specify the content type explicitly.

Test code like

String content = mockMvc.perform(get("/some-api")
            .contentType(MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
            .andReturn()
            .getResponse()
            .getContentAsString();

suddenly did not work anymore as the content type was mismatched as shown below

java.lang.AssertionError: Content type 
Expected :application/json;charset=UTF-8
Actual   :application/json

Changing the code to .andExpect(content().contentType(MediaType.APPLICATION_JSON)) resolved the issue for now.

But now when comparing content to the expected serialized object there is still a mismatch if there are any special characters in the object. It appears that the .getContentAsString() method does not make use of the UTF-8 character encoding by default (any more).

java.lang.AssertionError: Response content expected:<[{"description":"Er hörte leise Schritte hinter sich."}]> but was:<[{"description":"Er hörte leise Schritte hinter sich."}]>
Expected :[{"description":"Er hörte leise Schritte hinter sich."}]
Actual   :[{"description":"Er hörte leise Schritte hinter sich."}]

How can I get content in UTF-8 encoding?

Answer 1

Yes. This is problem from 2.2.0 spring-boot. They set deprecation for default charset encoding.

.getContentAsString(StandardCharsets.UTF_8) - good but in any response would be populated ISO 8859-1 by default.

In my project I updated current created converter:

@Configuration
public class SpringConfig implements WebMvcConfigurer {

    @Override
    public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
        converters.stream()
            .filter(converter -> converter instanceof MappingJackson2HttpMessageConverter)
            .findFirst()
            .ifPresent(converter -> ((MappingJackson2HttpMessageConverter) converter).setDefaultCharset(UTF_8));
    }
...

Answer 2

Using .getContentAsString(StandardCharsets.UTF_8) instead of .getContentAsString() resolves the problem.

Answer 3

The default encoding character is no longer UTF-8 since the 5.2.0 version of spring.

To continue using UTF-8, you must set it in the ServletResponse of the MockMvc result. To set the default character encoding to UTF-8, do something like this in your setup method:

@Before
public void setUp() {
   mockMvc = webAppContextSetup(wac).addFilter(((request, response, chain) -> {
                response.setCharacterEncoding("UTF-8");
                chain.doFilter(request, response);
            })).build();
}

Then you can use the mockMvc instance to perform your request.

Hope this help.

Answer 4

Following on black4bird's answer, you can override character encoding for all your tests by placing a following MockMvcBuilderCustomizer implementation in your test Spring context:

@Component
class MockMvcCharacterEncodingCustomizer implements MockMvcBuilderCustomizer {
    @Override
    public void customize(ConfigurableMockMvcBuilder<?> builder) {
        builder.alwaysDo(result -> result.response.characterEncoding = "UTF-8");
    }
}

That might help if you don't want to explicitly setup MockMvc, and just use @AutoconfigureMockMvc.

Answer 5

I am using Spring Boot 1.5.15.RELEASE and faced the same issue when writing tests.

The first solution that helped me was to add .characterEncoding("UTF-8")) like this:

String content = mockMvc.perform(get("/some-api")
            .contentType(MediaType.APPLICATION_JSON)
            .characterEncoding("UTF-8"))
            .andExpect(status().isOk())
            .andExpect(content().contentType(MediaType.APPLICATION_JSON_UTF8))
            .andReturn()
            .getResponse()
            .getContentAsString();

I am using a StandaloneMockMvcBuilder in my test class, so the second solution that helped me was to create a filter e.g.:

private static class Utf8Filter extends OncePerRequestFilter {

    @Override
    protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain)
        throws ServletException, IOException {
        response.setCharacterEncoding(StandardCharsets.UTF_8.toString());
        filterChain.doFilter(request, response);
    }
}

and later add it to the standaloneSetup method in my test class like this:

@Before
public void setup() {
    MockitoAnnotations.initMocks(this);
    final SomeResource someResource = new SomeResource(someService);
    this.restLankMockMvc = MockMvcBuilders.standaloneSetup(someResource)
        .setCustomArgumentResolvers(pageableArgumentResolver)
        .setControllerAdvice(exceptionTranslator)
        .setConversionService(createFormattingConversionService())
        .setMessageConverters(jacksonMessageConverter)
        .addFilter(new Utf8Filter())
        .build();
}

Answer 6

To restore the original behavior (Content-Type=application/json;charset=UTF-8) and allow your tests to pass as is, you could do the following:

@Configuration
public class MyWebConfig implements WebMvcConfigurer {

    @Override
    public void configureContentNegotiation(ContentNegotiationConfigurer configurer) {
        configurer.defaultContentType(MediaType.APPLICATION_JSON_UTF8);
    }
...

However, since APPLICATION_JSON_UTF8 is deprecated and Spring's intention is not to include the charset, it might be better going forward to modify your tests. black4bird's solution worked for me.

Answer 7

According to this pull request from the spring developers the UTF-8 header is not required anymore and therefore it's deprecated. If you're using the UTF-8 header in your application you might consider removing it from your application instead of trying fix your test. Just make sure you're using the Content-Type: application/json header and you should be fine.

Answer 8

Additional setting to MockMvc, .accept(MediaType.APPLICATION_JSON_UTF8_VALUE):

    String content = mockMvc.perform(get("/some-api")
        .contentType(MediaType.APPLICATION_JSON)
        .accept(MediaType.APPLICATION_JSON_UTF8_VALUE))
        .andExpect(status().isOk())
        .andExpect(content().contentType(MediaType.APPLICATION_JSON))
        .andReturn()
        .getResponse()
        .getContentAsString();

This problem is not Spring Boot, but MockMvc specific one, I guess. So, a workaround must be applied to MockMvc only. (JSON must be encoded using UTF-8.)

related issue: Improper UTF-8 handling in MockMvc for JSON response · Issue #23622 · spring-projects/spring-framework

Answer 9

if you are using maven, you can use the maven-surefire-plugin and enable UTF-8 encoding this way:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <argLine>-Dfile.encoding=UTF-8</argLine>
    </configuration>
</plugin>

Answer 10

This works in Spring Boot 2.6.6

@Autowired
private WebApplicationContext wac;

private MockMvc mockMvc;

@PostConstruct
public void init() {
   mockMvc = MockMvcBuilders.webAppContextSetup(wac)
                            .defaultResponseCharacterEncoding(StandardCharsets.UTF_8)
                            .build();
}