Delphi XE2: UTF16LE -> UTF8

Question

URL : http://www.gagalive.kr/livechat1.swf?chatroom=~~~BBQ

[1]-------------------------------------------------------------------

procedure TForm1.FormCreate(Sender: TObject);

begin

 IdTCPClient.Host := '61.97.246.131';

  IdTCPClient.Port := 8080;

  IdTCPClient.Connect;

  IdTCPClient.IOHandler.Write('Ag8m' + Char(0));

  IdTCPClient.IOHandler.Write('LShady|###BBQ' + Char(0));

  IdTCPClient.IOHandler.Write('#' + 'Some Text'  + Char(0));

  IdTCPClient.Disconnect;

end;

[2]-------------------------------------------------------------------

function UTF8FromUTF16_2(sUTF16: UnicodeString): UTF8String; 

begin   

  Result := sUTF16; 

end; 

procedure TForm1.FormCreate(Sender: TObject);

begin

 IdTCPClient.Host := '61.97.246.131';

  IdTCPClient.Port := 8080;

  IdTCPClient.Connect;

  IdTCPClient.IOHandler.Write('Ag8m' + Char(0));

  IdTCPClient.IOHandler.Write('LShady|###BBQ' + Char(0));

  IdTCPClient.IOHandler.Write(UTF8FromUTF16_2('#' + '안녕'  + Char(0)));

  IdTCPClient.Disconnect;

end;

[1] : working

[2] : not working (string broken > Shady: ??)

See : http://www.gagalive.kr/livechat1.swf?chatroom=~~~BBQ

UTF8FromUTF16 function Original VB Code : VB 6.0 -> Delphi XE2 Conversion

Help me.. :(

2 not working? Use 1. Or describe what you'd expect and what you got. — GolezTrol, May 14 '12 at 06:57
1. char(0)<>ansichar(0) you will want to attach anischar(0) to a onebyte type 2. is write() really rawbytestring? If it is declared as string it will convert back to UTF16 anyway. — Marco van de Voort, May 14 '12 at 07:01
Yeah, "not working" is pretty hopeless. Which Write overload are you using? The string one? That's going to convert back to UTF-16. Probably need to write bytes directly to retain UTF-8. But I'm guessing because I don't actually know what's wrong. — David Heffernan, May 14 '12 at 07:01
IdTCPClient.IOHandler.Write('#' + '안녕' + Char(0)); < Same String Broken — user1390537, May 14 '12 at 07:53
Those comments don't help. Be precise and explain it clearly. Don't do it in comments, add some explanation to the question. Also, please format your code properly and remove all the spurious new lines. These things matter. — David Heffernan, May 14 '12 at 08:05
Without debugging it's hard to tell, but I would bet that the Korean characters are being corrupted during the concatenation before the UTF8FromUTF16_2 conversion. Would something like this work better? "Write('#' + UTF8FromUTF16_2('(anyoung)') + char(0)));" Or will this place the # before the Unicode header for the sting? — James L., May 14 '12 at 15:26
Using `UTF8FromUTF16_2()` is the completely wrong approach to use in this situation. See my answer. — Remy Lebeau, May 15 '12 at 00:05

score 3 · Answer 1 · answered May 15 '12 at 00:02

3

Strings are UTF-16 encoded in XE2. #2 does not work because you are passing a UTF8String where a UnicodeString is expected, so the RTL will perform a UTF-8 -> UTF-16 conversion before Indy sees the data, and then Indy will convert the UTF-16 data to its own default encoding during the transmission.

If you want to transmit a UnicodeString as UTF-8, you have to tell Indy to do that, via its TIdTextEncoding class, eg:

procedure TForm1.FormCreate(Sender: TObject); 
begin 
  IdTCPClient.Host := '61.97.246.131'; 
  IdTCPClient.Port := 8080; 
  IdTCPClient.Connect; 
  IdTCPClient.IOHandler.Write('Ag8m'#0); 
  IdTCPClient.IOHandler.Write('LShady|###BBQ'#0); 
  IdTCPClient.IOHandler.Write('#안녕'#0, TIdTextEncoding.UTF8); // <-- here
  IdTCPClient.Disconnect; 
end;

Or:

procedure TForm1.FormCreate(Sender: TObject); 
begin 
  IdTCPClient.Host := '61.97.246.131'; 
  IdTCPClient.Port := 8080; 
  IdTCPClient.Connect; 
  IdTCPClient.IOHandler.DefStringEncoding := TIdTextEncoding.UTF8; // <-- here
  IdTCPClient.IOHandler.Write('Ag8m'#0); 
  IdTCPClient.IOHandler.Write('LShady|###BBQ'#0); 
  IdTCPClient.IOHandler.Write('#안녕'#0);
  IdTCPClient.Disconnect; 
end;

If you do not tell Indy which encoding to use, it will default to ASCII, which will corrupt/lose your Unicode data.

answered May 15 '12 at 00:02

Remy Lebeau

555,201
31
458
770

Winsock.SendData('#' + Edit1.Text + Char(0)); < TIdTextEncoding.UTF8 how to use in Winsock? – – user1390537 May 15 '12 at 09:20
You would have to use the `TIdTextEncoding.GetBytes()` method to convert a string to an encoded byte array, then you can send the bytes. – Remy Lebeau May 15 '12 at 16:00
Thank you but I do not understand the answer. I am very sorry.. Can you give a example code..? – user1390537 May 15 '12 at 17:02
`GetBytes()` returns a `TBytes`, which is a dynamic array of bytes. Simply pass your desired string to `GetBytes()`, and then pass the byte array to WinSock. What is hard to understand about that? – Remy Lebeau May 15 '12 at 18:01
var S : String; B : TBytes; begin B := TEncoding.UTF8.GetBytes('안녕'); Winsock.SendData('#' + B + #0); ///////// It is difficult for me........ Error :( – user1390537 May 15 '12 at 18:57
You cannot mix strings and bytes like that. If `SendData()` expects a `String` as input, you will have to send the strings and bytes separately: `Winsock.SendData('#'); Winsock.Send(B); Winsock.SendData(#0);`, otherwise you have to encode all of the strings together into a single set of bytes and send it as one call: `B := TEncoding.UTF8.GetBytes('#안녕'#0); Winsock.Send(B);`. How you actually send an array of bytes depends on what methods your Winsock class actually exposes (why you are moving away from `TIdTCPClient`?). I am assuming it has some kind of method for sending raw bytes. – Remy Lebeau May 15 '12 at 19:48
i don't know receive data in IdTCPClient. IdTCPClient.Socket.ReadLn < Not Working IdTCPClient.IOHandleor < Not Working / Thank you very much. God bless you – user1390537 May 15 '12 at 20:07
`ReadLn` expects a `LF` terminator by default. Is the server sending a `LF` character? If not, is the server sending any terminator character at all? Your outgoing messages are null-terminated, are the server's inbound messages also null-terminated? `ReadLn()` has an optional `ATerminator` parameter that you can pass `#0` to for reading null-terminated strings. – Remy Lebeau May 15 '12 at 20:40

Delphi XE2: UTF16LE -> UTF8

1 Answers1