Delphi XE8 idHttp Erromessage text coding is wrong

Question

I am using Delphi XE8 and I am sendig a PUT message via idHttp.

  Http.Request.CustomHeaders.Clear;
  Http.Request.BasicAuthentication := false;
  http.Request.Method := 'PUT';
  Http.Request.Accept := '*/*';
  Http.Request.ContentType := 'application/json';
  http.Request.CustomHeaders.AddValue('apiKey','T_API23207_169');
  http.Request.CustomHeaders.AddValue('transactionId','20200924_015');
  http.Request.CustomHeaders.AddValue('usziID','1');
  Http.Request.AcceptEncoding := '*';
  http.Request.CharSet := 'utf-8';

  kuldes_header.Text := http.Request.CustomHeaders.Text;

  http.Intercept := IdLogEvent1;
  IdLogEvent1.Active := true;

  jsonToSend := TStringStream.create(json_adat.Text,system.sysUtils.TEncoding.UTF8);

  kuldes_body.Lines.LoadFromStream(jsonToSend);

  try
    try
        send_text := http.Put('http://10.109.132.24:8090/rest/usziIroda/1',jsonToSend);
        resp := http.ResponseText;
        code := http.ResponseCode;
        jsonToSend.Position := 0;
    except
        on E: EIdHTTPProtocolException do
          begin
          code := e.ErrorCode;
          error_message := e.ErrorMessage;
          end;
    end;

    hiba_kod.Lines.Add(IntToStr(code));
    valasz_uzenet.Text := send_text;
    hiba_uzenet.Text := error_message;   enter code here

The returned error message has strange characters: "Megadott tranzakció azonosítóval már történt API hívás"

But it should be like this: "Megadott tranzakció azonosí­tóval már történt API hí­vás"

How can I convert the returned message to normal string?

Thank you!

Answer 1

The result you showed - Megadott tranzakció azonosítóval már történt API hívás - is the UTF-8 encoded form of Megadott tranzakció azonosí­tóval már történt API hí­vás being misinterpreted in Latin-1/ISO-8859-1. Which most likely means that the response did not specify a UTF-8 charset in its Content-Type header (since you have an Intercept assigned, you can easily verify this for yourself), so Indy would fall back to a default charset instead.

The original UTF-8 bytes have been decoded and lost before you can access the response data in send_text or error_message. However, since ISO-8859-1 basically has a 1:1 relationship between byte values and Unicode codepoint values, what you can try doing in this specific situation is copy the ErrorMessage's Char values as-is to a RawByteString(65001) or UTF8String, and then let the RTL decode that as UTF-8 back into a proper UTF-16 (Unicode)String, eg:

function DecodeISO88591AsUTF8(const S: string): string;
var
  utf8: UTF8String;
  I: Integer;
begin
  SetLength(utf8, Length(S));
  for I := Low(S) to High(S) do
    utf8[I] := AnsiChar(S[I]);
  Result := string(utf8);
end;

...

error_message := e.ErrorMessage;
//if not TextIsSame(http.Response.CharSet, 'utf-8') then
if TextIsSame(http.Response.CharSet, 'ISO-8859-1') then
  error_message := DecodeISO88591AsUTF8(error_message);

Alternatively, you can instead call the overloaded version of TIdHTTP.Put() that fills a response TStream instead of returning a decoded String, and then you can decode the original raw bytes however you want. Just be sure to enable the hoNoProtocolErrorException and hoWantProtocolErrorContent flags in the TIdHTTP.HTTPOptions property so that any error response is stored in the TStream, then you don't need a try/except to handle the EIdHTTPProtocolException separately.

http.HTTPOptions := http.HTTPOptions + [hoNoProtocolErrorException, hoWantProtocolErrorContent];

...

RespStrm := TMemoryStream.Create;
try
  http.Put('http://10.109.132.24:8090/rest/usziIroda/1', jsonToSend, RespStrm);
  resp := http.ResponseText;
  code := http.ResponseCode;
  jsonToSend.Position := 0;
  RespStrm.Position := 0;

  if (code div 100) = 2 then
  begin
    send_text := decode RespStrm as needed...;
  end else
  begin
    error_message := decode RespStrm as needed...;
  end;
finally
  RespStrm.Free;
end;

...