Java code converting special char to junk char

Question

I am facing an issue that when I post a json message with special char from Java program, that special character ( say ’ in O’Reilly) is replaced with ?. If I post same message from POSTMAN, I get correct response. Could you please suggest how to handle this issue ?

Thank You.

Below is the code and json I am using:

{ "lastName":"O’Reilly", "firstName":"Shaun"}

package rest_apis;

import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.client.utils.URIBuilder;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.util.EntityUtils;

public class IDQ_tranformation_cd {

    public static void main(String args[]){
        String json_req="{"lastName":"O’Reilly","firstName":"Shaun"}";

        try{
          CloseableHttpClient httpclient = HttpClients.createDefault();
          String instanceUrl = "https://my-end-point";
          URIBuilder builder = new URIBuilder(instanceUrl);

          StringEntity params =new StringEntity(json_req);
          HttpPost post_data = new HttpPost(builder.build());

          post_data.addHeader("Content-Type","application/json");
          post_data.addHeader("Accept", "application/json");

          post_data.setEntity(params);

          HttpResponse queryResponse = httpclient.execute(post_data);
                    int Out_RespCd = queryResponse.getStatusLine().getStatusCode();
          HttpEntity httpEntity = queryResponse.getEntity();

          System.out.println(Out_RespCd +":"+EntityUtils.toString(httpEntity));
        } catch (Exception e) {
          System.out.println("Exception :"+e.toString());
        } 
    }
}

I expect correct output as (I am able to get this when I use postman):

{"lastName":"O’Reilly","firstName":"Shaun"}

But I am getting this when I use java code (’ replaced with ?)

{"lastName":"O?Reilly","firstName":"Shaun"}

If you are printing this in your console its possible that it doesn't support `’` encoding — Saharsh, Oct 03 '19 at 07:35
What happens when you just print `json_req` on to `System.out`? That should help identify if the problem is in the output, as Saharsh suggests. — Joachim Sauer, Oct 03 '19 at 07:53
look this question: https://stackoverflow.com/questions/19445797/java-httpurlconnection-post-request-special-characters-strange-behavior — Vishwa Ratna, Oct 03 '19 at 08:08
@JoachimSauer, it print correctly inside Syste.out : "lastName":"O’Reilly" — Sourabh, Oct 03 '19 at 10:10

score 0 · Answer 1 · answered Oct 03 '19 at 10:14

Try to change your line:

post_data.addHeader("Content-Type","application/json");

to

post_data.addHeader("Content-Type","application/json; charset=utf-8");

Also, it may be that you get the data OK and only the display is incorrect. What helped me numerous times to diagnose similar problems is to convert the "problematic" string into a sequence of unicodes. There you can see if the data itself is correct and only the display is corrupted or the data itself is corrupted. There is an Open Source java library MgntUtils that has a Utility that converts Strings to unicode sequence and vise versa:

result = "Hello World";
result = StringUnicodeEncoderDecoder.encodeStringToUnicodeSequence(result);
System.out.println(result);
result = StringUnicodeEncoderDecoder.decodeUnicodeSequenceToString(result);
System.out.println(result);

The output of this code is:

\u0048\u0065\u006c\u006c\u006f\u0020\u0057\u006f\u0072\u006c\u0064
Hello World

The library can be found at Maven Central or at Github It comes as maven artifact and with sources and JavaDoc

Here is JavaDoc for the class StringUnicodeEncoderDecoder. Also in that library, there is a very simple Http client.

Thank You Michael for valuable suggestion. it worked when I change in below line : StringEntity params =new StringEntity(json_req,"UTF-8"); — Sourabh, Oct 04 '19 at 05:07

score 0 · Answer 2 · answered Oct 04 '19 at 05:05

Thanks All for your help. It worked after changing below line in code:

StringEntity params =new StringEntity(json_req,"UTF-8");

The below article was very helpful.

HttpPost with StringEntity having special characters like ®, seeing ¿½` instead of ®

Java code converting special char to junk char

2 Answers2