JsonParseException: Invalid UTF-8 middle byte 0x20 with MockMVC and Maven

Question

EDIT: If I remove the spring-restdoc code, the test passes with maven.

I'm trying to write an integration test with MockMVC in a Spring Boot web app.

I have only one test at the moment (steps and code below) that pass if I run it in the IDE but fails with the following stacktrace if executed by Maven:

org.springframework.restdocs.payload.PayloadHandlingException: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0x20
 at [Source: [B@3793896e; line: 30, column: 23]
        at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1419)
        at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:508)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3223)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3230)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeUtf8_3fast(UTF8StreamJsonParser.java:3036)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2216)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2165)
        at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:279)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:441)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(UntypedObjectDeserializer.java:574)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:423)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapArray(UntypedObjectDeserializer.java:510)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:437)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.mapObject(UntypedObjectDeserializer.java:552)
        at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializer$Vanilla.deserialize(UntypedObjectDeserializer.java:423)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3066)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2230)
        at org.springframework.restdocs.payload.JsonContentHandler.readContent(JsonContentHandler.java:81)
        at org.springframework.restdocs.payload.JsonContentHandler.findMissingFields(JsonContentHandler.java:49)
        at org.springframework.restdocs.payload.AbstractFieldsSnippet.validateFieldDocumentation(AbstractFieldsSnippet.java:112)
        at org.springframework.restdocs.payload.AbstractFieldsSnippet.createModel(AbstractFieldsSnippet.java:73)
        at org.springframework.restdocs.snippet.TemplatedSnippet.document(TemplatedSnippet.java:64)
        at org.springframework.restdocs.mockmvc.RestDocumentationResultHandler.handle(RestDocumentationResultHandler.java:101)
        at org.springframework.test.web.servlet.MockMvc$1.andDo(MockMvc.java:158)

Code:

I added the starter-test dependency and I created a super class for all the future tests:

@Ignore
@MappedSuperclass
@RunWith(SpringJUnit4ClassRunner.class)
@SpringApplicationConfiguration(classes = Application.class)
@WebAppConfiguration
@ActiveProfiles(value = {"test"})
public class IntegrationTest
{
    @Autowired
    WebApplicationContext context;

    MockMvc mockMvc;

    @Rule
    public RestDocumentation restDocumentation = new RestDocumentation("target/generated-snippets");

    @Before
    public void setUp()
    {
        this.mockMvc = MockMvcBuilders.webAppContextSetup(this.context).apply(documentationConfiguration(this.restDocumentation)).build();
    }
}

Then I created this test:

public class PostsTest extends IntegrationTest
{
    @Test
    public void getAll() throws Exception
    {
        mockMvc.perform(get("/api/v1/posts").accept(MediaType.APPLICATION_JSON_VALUE)
            .andExpect(status().isOk())
            .andDo(document("posts-index", preprocessResponse(prettyPrint()), responseFields(
                fieldWithPath("data").description("The response payload"),
                fieldWithPath("data.[].id").description("ID"),
                fieldWithPath("data.[].title").description("Title"),
                fieldWithPath("data.[].body").description("Content"),
                fieldWithPath("data.[].type").description("Type"),
                fieldWithPath("data.[].createdAt").description("Creation date"),
                fieldWithPath("data.[].modifiedAt").description("Last edit date"),
                fieldWithPath("data.[].entityType").description("Entity type"),
                fieldWithPath("meta").description("The response metadata")
            )));
    }
}

Francesco Papagno · Accepted Answer · 2016-02-24T14:26:59.337

3

I found out that removing preprocessResponse(prettyPrint()) from the restdocs code, the test passes.

EDIT: As reported by Andy Wilkinson in the comments, this bug is fixed with the version 1.0.2.BUILD-SNAPSHOT of spring-restdocs.

edited Feb 24 '16 at 14:26

answered Feb 24 '16 at 09:31

Francesco Papagno

617
10
29

1

That sounds like [this bug](https://github.com/spring-projects/spring-restdocs/issues/202). It's been fixed in 1.0.2 snapshots. They're available from https://repo.spring.io/libs-snapshot if you'd like to give it a try. Use `1.0.2.BUILD-SNAPSHOT` as the version. – Andy Wilkinson Feb 24 '16 at 12:42
That's correct. It works with the snapshot version. Thanks! – Francesco Papagno Feb 24 '16 at 14:25

score -1 · Answer 2 · answered Feb 24 '16 at 09:10

-1

You are parsing json at some point, I suspect it is being read from a file.

That file seems to be saved in the wrong encoding, such as ISO-8859-1. Make sure it is saved in either UTF-8, UTF-16 or UTF-32. Most JSON parsers are guessing the encoding from the first four bytes and fail if they find a unsupported encoding.

answered Feb 24 '16 at 09:10

f1sh

11,489
3
25
51

The data is pulled from a MySQL database. What I don't understand is the difference between maven and the IDE. – Francesco Papagno Feb 24 '16 at 09:12
The IDE probably uses UTF-8 while maven, when run in the command prompt, uses the platform encoding (which is still ISO-8859-1 or Latin1 on windows machines). Try to add ``-Dfile.encoding="UTF-8"`` to your command line statement and see if it changes anything. – f1sh Feb 24 '16 at 09:15
Nothing changed, and I already had `UTF-8` in my pom. – Francesco Papagno Feb 24 '16 at 09:20
The problem is reading from a database. The option I suggested and your setting in the pom.xml don't affect the database's contents... Can you post the code that's involved in reading the json from the database? – f1sh Feb 24 '16 at 09:24
I use Spring Data and Spring MVC, so the models are serialized into JSON by Jackson. (I edited the question because if I remove the spring-restdoc code, the test passes with maven.) – Francesco Papagno Feb 24 '16 at 09:27

JsonParseException: Invalid UTF-8 middle byte 0x20 with MockMVC and Maven

2 Answers2

Linked