How to handle UTF-8 and ANSI conversion before Delphi 2009?

Question

In Delphi 2009 we have:

RichEdit1.Lines.LoadFromFile(OpenDialog1.FileName,TEncoding.UTF8);
RichEdit1.Lines.SaveToFile(OpenDialog2.FileName,TEncoding.Unicode);

How do I do that on Delphi 2006 if I do not have TEconding yet?

Is there someway to transport that newer library back there? or is there a solution hidden in the Web?

I believe that `UTF8Encode` and `UTF8Decode` were present before Delphi 2009. So you can decode/encode byte strings manually. — Andreas Rejbrand, Mar 01 '13 at 13:13
Wow! Man you're fast!! Post your answer, cause that is right!!! Problem solved. — NaN, Mar 01 '13 at 13:18
Those functions won't help replicate `TEncoding.Unicode` since that is UTF-16. — David Heffernan, Mar 01 '13 at 13:26
A standard RTF file can consist of only 7-bit ASCII characters (http://en.wikipedia.org/wiki/Rich_Text_Format#Character_encoding), why do you use UTF8? — mjn, Mar 01 '13 at 15:42
I made an android application that creates text files. But those come in UTF8. I made another application in delphi that read those txt files and do something, but they have to be in the default delphi encoding (I think it's ANSI) and for that I need to decode when I open them. — NaN, Mar 02 '13 at 01:55

score 6 · Accepted Answer · answered Mar 01 '13 at 13:20

6

I believe that UTF8Encode and UTF8Decode were present even before Delphi 2009. So you can decode/encode byte strings manually. (I have done that myself.)

answered Mar 01 '13 at 13:20

Andreas Rejbrand

105,602
8
282
384

score 3 · Answer 2 · answered Mar 01 '13 at 19:16

Here is a snippet from a Delphi 7 project of mine:

function LoadFile(const fn: string): WideString;
var
  f:TFileStream;
  src:AnsiString;
  wx:word;
  i,j:integer;
begin
  if FileExists(fn) then
   begin
    f:=TFileStream.Create(fn,fmOpenRead or fmShareDenyNone);
    try
      f.Read(wx,2);
      if wx=$FEFF then
       begin
        //UTF16
        i:=(f.Size div 2)-1;
        SetLength(Result,i);
        f.Read(Result[1],i*2);
        //detect NULL's
        for j:=1 to i do if Result[j]=#0 then Result[j]:=' ';//?
       end
      else
       begin
        i:=0;
        if wx=$BBEF then f.Read(i,1);
        if (wx=$BBEF) and (i=$BF) then
         begin
          //UTF-8
          i:=f.Size-3;
          SetLength(src,i);
          f.Read(src[1],i);
          //detect NULL's
          for j:=1 to i do if src[j]=#0 then src[j]:=' ';//?
          Result:=UTF8Decode(src);
         end
        else
         begin
          //assume current encoding
          f.Position:=0;
          i:=f.Size;
          SetLength(src,i);
          f.Read(src[1],i);
          //detect NULL's
          for j:=1 to i do if src[j]=#0 then src[j]:=' ';//?
          Result:=src;
         end;
       end;
    finally
      f.Free;
    end;
   end
  else
    Result:='';
end;

How to handle UTF-8 and ANSI conversion before Delphi 2009?

2 Answers2

Linked