Can't get character encoding to work from android client with resttemplate and json

Question

Ok, so ive tried everything but asking at stackoverflow...

I'm trying to perform a REST call with some http params from an Android using httpclient and resttemplate to a server-side Spring controller. All my Swedish chars end up on the server as '\u001A'...

setting up httpclient and resttemplate code:

HttpClient httpClient = new HttpClient();

Credentials defaultcreds = new UsernamePasswordCredentials(msisdn, password);
httpClient.getState().setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT, AuthScope.ANY_REALM), defaultcreds);

//httpClient.getParams().setContentCharset(prefs.());
httpClient.getParams().setCredentialCharset(prefs.getCredentialsEncoding());

CommonsClientHttpRequestFactory requestFactory = new CommonsClientHttpRequestFactory(httpClient);

RestTemplate restTemplate = new RestTemplate(requestFactory);

// Add message converters
List<HttpMessageConverter<?>> mc = restTemplate.getMessageConverters();
MappingJacksonHttpMessageConverter json = new MappingJacksonHttpMessageConverter();

List<MediaType> supportedMediaTypes = new ArrayList<MediaType>();
supportedMediaTypes.add(MediaType.APPLICATION_JSON);
json.setSupportedMediaTypes(supportedMediaTypes);
mc.add(json);
restTemplate.setMessageConverters(mc);

return restTemplate;

I then prepare a httpentity with my parameter:

HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.add("myparam", "" + "å");
HttpEntity<String> entity = new HttpEntity<String>(headers);

I finally make the rest call:

ResponseEntity<UserStatusTO> response = rest.exchange(url, HttpMethod.GET, entity,
            MyResponseClass.class);

On the server, i have a jackson deserializer and my spring controller method looks like:

@RequestMapping(method = RequestMethod.GET, value = "/event/out", headers = "Accept=application/json", produces = {"application/xml;charset=UTF-8", "application/json;charset=UTF-8"})
public
@ResponseBody
UserStatusTO out(Authentication auth, @RequestHeader(value="myparam", required =  false) String myparam) {

All the special chars end up as \u001a ! I've tried tons of stuff, re-encoding the strings manually client AND server side, none worked. I've tried fiddling with the httpClient.getParams().setContentCharset(); httpClient.getParams().setUriCharset();

None worked as far as i could tell.

I'm out of ideas! If anybody has any input, i'd be much obliged. Thanks!

Except for the "myparam" header, where are you cramming in the å, ä:s and ö:s? — Jens, Feb 02 '12 at 11:22
Well, there's only one parameter that has swedish characters. I shortened my example to get make it easier to read, but in my app there's a field where the user can enter a note that i'm trying to send to the server... But the place where i have the "å" is where i in my code put in the text from the textfield — Mathias, Feb 02 '12 at 11:59

score 4 · Answer 1 · answered Mar 28 '12 at 12:32

I had a similar issue and I fixed it by setting the proper content-type AND character set manually instead of using MediaType.APPLICATION_JSON:

HttpHeaders headers = new HttpHeaders();
Charset utf8 = Charset.forName("UTF-8");
MediaType mediaType = new MediaType("application", "json", utf8);
headers.setContentType(mediaType);

score 1 · Answer 2 · answered Feb 02 '12 at 12:20

This sounds like an issue on the server side since the Apache HTTP client will gladly output Latin-1 characters in headers, as you can test with this simple test:

HttpGet get = new HttpGet("http://www.riksdagen.se");
get.addHeader("SwedishHeader", "Mona Sahlin är ett nötdjur");

ByteArrayOutputStream out = new ByteArrayOutputStream();
SessionOutputBufferImpl buffer = new SessionOutputBufferImpl(out, get.getParams());
HttpRequestWriter writer = new HttpRequestWriter(buffer, BasicLineFormatter.DEFAULT, get.getParams());

writer.write(get);
buffer.flush();
System.out.println(out.toString("ISO-8859-1"));

SessionOutputBufferImpl is a little hack to use the AbstractSessionOutputBuffer

class SessionOutputBufferImpl extends AbstractSessionOutputBuffer {
    SessionOutputBufferImpl(OutputStream out, HttpParams params) {
        init(out, 1024, params);
    }
}

You should probably concentrate on your server backend - what are you using? If it's "fixed" - you should consider trying to format your header according to RFC 2407 - some servers support, others fail miserably.

I'm sorry, but i can confirm that it's not the server. I spent the day updating my IOS application with the same functionality and the texts arrive fine, encoded properly and all characters look ok in the database where they end up... — Mathias, Feb 03 '12 at 00:54
Have you wiresharked your server input & checked for encoding differences? — Jens, Feb 03 '12 at 07:54

Can't get character encoding to work from android client with resttemplate and json

2 Answers2