0

I am calling an endpoint via RestTemplate as follows:

   SSLContext sslContext = SSLContexts.createSystemDefault();
   SSLConnectionSocketFactory socketFactory = new SSLConnectionSocketFactory(sslContext);
   HttpClient client = HttpClientBuilder.create().setSSLSocketFactory(socketFactory).build();
   ClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(client);

   HttpEntity<String> entity = new HttpEntity<>(requestJson, headers());
   Object[] response = restTemplate.postForObject(uri, entity, Object[].class);

I have verified that the JSON String in the entity object is valid by copying it and using it in a cURL request to the same endpoint without any error. The same headers and authorization token were used in this request too.

When I execute the POST, I get back the following error:

Error while extracting response for type [class [Ljava.lang.Object;] and content type [application/json;charset=utf-16]

org.springframework.http.converter.HttpMessageNotReadableException: JSON parse error: Unrecognized token '嬀崀': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false'); nested exception is com.fasterxml.jackson.core.JsonParseException: Unrecognized token '嬀崀': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
     at [Source: (InputStreamReader); line: 1, column: 3]

My accept and content-type headers are both set to application/json. From examining the output from cURL, I see that there are no Chinese characters in the response body.

The response headers are as follows:

Connection →keep-alive
Content-Encoding →gzip
Content-Type →application/json; charset=utf-8
Date → xxx
Transfer-Encoding →chunked
Vary →Accept-Encoding

When I make the request with the responseType set to String.class or Object.class, the response is Chinese characters:

restTemplate.postForObject(uri, entity, String.class);
嬀崀

I am expecting that this call should return an empty array [].

When I change the requestJson String to one which should serve back a non-empty array, I get back hundreds of Chinese characters instead of just two.

How can I decode the response to get valid data like when I use cURL?

edit:

I'm not sure how this is related, but the bytecode for the chars in an empty array [] are 91 and 93, and the bytecode for the two Chinese characters is 0, 91, 0, 93.

Queue
  • 446
  • 7
  • 23

3 Answers3

0

It seems like the response from restTemplate is returned as a String of Chinese characters not an array. The first error you posted seems to indicate that the issue is with extracting the response into an Object[]. If the response from the restTemplate is actually a String, then that would explain the second error as well. The RestTemplate was expecting to parse an array but instead received the String 嬀崀. That's why changing the response type to String.class seems to work.

If you're expecting a JSON array back from the api you're calling, then I would double check the response from the api you're calling. Otherwise, I would suggest using String.class instead.

Edited: It's possible that the restTemplate is parsing the response using the utf-16 charset and the server is encoding the response using the utf-8 charset. Like you've posted in the description, those characters seem to have the same bytecode. Maybe changing the expected charset in the restTemplate to utf-8 will resolve your problem.

jcarey
  • 1
  • 2
  • The response from the endpoint is expected to be an empty array `[]` but it's being decoded in my code (incorrectly) as Chinese characters. – Queue Apr 13 '21 at 14:33
  • The response is valid JSON. I was just using String.class to cast as wide a net as possible to ensure I didn't throw a mapping exception to a DTO. – Queue Apr 13 '21 at 14:36
  • oh! sorry, I thought you were expecting Chinese characters in the array or something. I don't know the values of the character encodings of utf-8 vs utf-16, but it looks like the charset on the response from curl is utf-8 and the charset on the response from the restTemplate is utf-16. Maybe the representation of an empty array is translated into those chinese characters in the other charset? – jcarey Apr 13 '21 at 14:44
0

Do not use UTF16. HTTP Spec says ASCII, and many use UTF8. The error points this out with charset=utf-16 in it. Try setting the encoding header on the request.

What you're seeing, as you noted with the char codes, is exactly the result expected when using UTF16 because each char is 2 bytes.

Richard Barker
  • 1,161
  • 2
  • 12
  • 30
  • Ok I set these request headers: ACCEPT, CONTENT_TYPE, and CONTENT_ENCODING to the value of `"application/json; charset=utf-8"`, however the error message served back is still `Error while extracting response for type [class java.lang.Object] and content type [application/json;charset=utf-16]` – Queue Apr 13 '21 at 15:11
  • Assuming you have control of the API server, make sure it's not forcing UTF16. – Richard Barker Apr 13 '21 at 18:37
  • When I hit the exact same API via curl, I see that the response header is: `Content-Type →application/json; charset=utf-8` – Queue Apr 13 '21 at 20:02
  • We'd need more info..but that's where you should be looking: why are you getting a UTF16 encoded response, when headers seem to be set right. – Richard Barker Apr 14 '21 at 02:05
0

The API is serving back UTF-8 content when I call it from cURL, but when I call the API programmatically it is serving back UTF-16LE despite my request headers asking for UTF-8 explicitly.

I will need to do some de-/en-coding games, but the following allows me to observe the valid, expected response JSON:

ResponseEntity<byte[]> responseEntity = restTemplate.postForEntity(uri, entity, byte[].class);
   
byte[] bytes = responseEntity.getBody();

String json = new String(bytes, StandardCharsets.UTF_16LE);

FooBar[] fooBar = objectMapper.readValue(json, FooBar[].class);
Queue
  • 446
  • 7
  • 23