1

Calling an API that returns french sentences, all the accented characters are displayed like <?> in my java code, even if the charset is well defined (application/json;charset=iso-8859-1).

Using postman or my web browser, I don't face any problem. I also tried to call the API with a Content-Type header with the value application/json;charset=UTF-8 or application/json;charset=iso-8859-1 but the problem remains the same. Any idea ?

response.getBody() gives:

{"sentences":[{"fr_value":"il �tait loin","dz_value":"kaan b3id","additional_information":{"personal_prounoun":"HE","verb":"�tre","adjective":"loin","tense":"pass�"}}],"count":1}

new String(response.getBody().getBytes(StandardCharsets.UTF_8)) gives exactly the same.

enter image description here

I'm using scribejava.

Edit: even saving the response in a file and opening it with NotePad++, the result is similar: enter image description here

Redouane B.
  • 93
  • 10
  • `application/json;charset=iso-8859-1` is invalid, AFAIK. JSON always uses an UTF encoding (8, 16 or 32) - see https://www.rfc-editor.org/rfc/rfc8259 – Gereon Apr 03 '22 at 12:43
  • 1
    What encoding is set in that IDE/editor you're viewing that in? – g00se Apr 03 '22 at 13:03
  • @g00se in UTF-8, but the problem is not only on the IDE but even elsewhere (I actually tweet the value and I still have the > problem) – Redouane B. Apr 03 '22 at 13:10
  • @Gereon just updated the encoding to `"application/json;charset=UTF-8"` but still the same. – Redouane B. Apr 03 '22 at 13:14
  • Might be worth saving it to a file so we can examine it. Might be worth asking whether the the server *already* has it wrong – g00se Apr 03 '22 at 13:40
  • @g00se just tried saving the file and opening it with NotePadd++, encoding is UTF-8 but the problem is the same one. – Redouane B. Apr 03 '22 at 13:44
  • Can we see it? How do you know it's not like that at the server? – g00se Apr 03 '22 at 13:48
  • @g00se just added the screenshot in the initial post. When I call the API through postman, the answer is OK. From my browser using http://localhost:8080/ , the API response is also correct. – Redouane B. Apr 03 '22 at 13:52
  • 1
    Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/243558/discussion-between-g00se-and-redouane-b). – g00se Apr 03 '22 at 13:54

2 Answers2

0

You need to read it as ISO-8859-1. Not sure what then as I don't know what you're doing. My https://technojeeves.com/index.php/aliasjava1/51-transcode-in-java is helpful. With wget: wget -O - us-central1-dz-dialect-api.cl… | xcode -ie Latin1 (I made 'xcode' to invoke that Java app)

g00se
  • 3,207
  • 2
  • 5
  • 9
0

Problem solved using the following code :

      httpResponse.setContentType("application/json;charset=UTF-8");
      mapper.getFactory().configure(JsonGenerator.Feature.ESCAPE_NON_ASCII, true);
Redouane B.
  • 93
  • 10