Best and simply way to store unicode record (array) into a file

Question

Yesterday I finally moved from Delphi 7 to Delphi-XE5. And I am re-writing whole application to Unicode and have some troubles.

Lets says, I have:

TCountry = record
  Name: WideString;
  Extension: WideString;
  Region: WideString;
end;

and have dynamic array of this class, which I want to store in a file on close and then load when application is started. Before (in Delphi 7) I stored it in INI file. But now I want to use Unicode (UTF-16LE) and I don't want to store it in the INI file; maybe I can save it in some binary format.

I tried couple methods, and just want to ask, what is the best way (method) to store a dynamic array with Unicode strings into a file?

First question. Why are you using `WideString`? More broadly, have you considered JSON or XML. The former is probably better, especially if you want humans to read the file. — David Heffernan, Mar 04 '14 at 23:01
You can still use INI, just use `TMemIniFile` instead of `TIniFile`. `TMemIniFile` supports `SysUtils.TEncoding`, so you can save Unicode data as UTF-8, for instance. — Remy Lebeau, Mar 05 '14 at 00:04
I was thinking about json, it will be not a problem. But for now, I saved it as XML in UTF-8, and after loading it I convert it back to Unicode. I will try some more tests, which will be the best for me... — Legionar, Mar 05 '14 at 08:59
After reading more about Delphi XE-5, I consider to use String (= UnicodeString). — Legionar, Mar 05 '14 at 09:00
@Legionar No XML library that I know of requires you to perform manual Unicode conversions. The library will receive strings and internally encode them (typically to UTF-8). In reverse, the library will decode the bytes in the XML file, and hand you Delphi strings. Which are `UnicodeString`. And don't consider using `string` instead of `WideString`. Do it. Do not use `WideString` unless you are doing COM interop. And then only do it at the COM boundary. — David Heffernan, Mar 05 '14 at 10:15

HeartWare · Accepted Answer · 2014-03-05T19:25:04.297

Try this code:

TYPE
  TCountry    = RECORD
                  Name        : STRING;
                  Extension   : STRING;
                  Region      : STRING
                END;
  TCountryArr = ARRAY OF TCountry;

VAR
  Countries   : TCountryArr;

{$DEFINE UTF8 }

PROCEDURE SaveCountries(CONST FileName : STRING ; CONST ARR : TCountryArr);
  VAR
    S       : TStream;
    I       : INTEGER;

  PROCEDURE WriteString(S : TStream ; CONST STR : STRING);
    VAR
      LEN           : INTEGER;
      {$IFDEF UTF8 }
        UTF8        : UTF8String;
      {$ENDIF }

    BEGIN
      {$IFDEF UTF8 }
        UTF8:=STR; LEN:=LENGTH(UTF8);
        S.Write(LEN,SizeOf(INTEGER));
        IF LEN>0 THEN S.Write(UTF8[LOW(UTF8)],LEN*SizeOf(AnsiChar))
      {$ELSE }
        LEN:=LENGTH(STR);
        S.Write(LEN,SizeOf(INTEGER));
        IF LEN>0 THEN S.Write(STR[LOW(STR)],LEN*SizeOf(CHAR))
      {$ENDIF }
    END;

  BEGIN
    S:=TFileStream.Create(FileName,fmCreate);
    TRY
      TRY
        I:=LENGTH(ARR);
        S.Write(I,SizeOf(INTEGER));
        FOR I:=LOW(ARR) TO HIGH(ARR) DO WITH ARR[I] DO BEGIN
          WriteString(S,Name);
          WriteString(S,Extension);
          WriteString(S,Region)
        END
      FINALLY
        S.Free
      END
    EXCEPT
      DeleteFile(FileName);
      RAISE
    END
  END;

PROCEDURE LoadCountries(CONST FileName : STRING ; VAR ARR : TCountryArr);
  VAR
    S       : TStream;
    I       : INTEGER;

  PROCEDURE ReadString(S : TStream ; VAR STR : STRING);
    VAR
      LEN           : INTEGER;
      {$IFDEF UTF8 }
        UTF8        : UTF8String;
      {$ENDIF }

    BEGIN
      S.Read(LEN,SizeOf(INTEGER));
      {$IFDEF UTF8 }
        SetLength(UTF8,LEN);
        IF LEN>0 THEN S.Read(UTF8[LOW(UTF8)],LEN*SizeOf(AnsiChar));
        STR:=UTF8
      {$ELSE }
        SetLength(STR,LEN);
        IF LEN>0 THEN S.Read(STR[LOW(STR)],LEN*SizeOf(CHAR))
      {$ENDIF }
    END;

  BEGIN
    S:=TFileStream.Create(FileName,fmOpenRead);
    TRY
      TRY
        S.Read(I,SizeOf(INTEGER));
        SetLength(ARR,I);
        FOR I:=LOW(ARR) TO HIGH(ARR) DO WITH ARR[I] DO BEGIN
          ReadString(S,Name);
          ReadString(S,Extension);
          ReadString(S,Region)
        END
      FINALLY
        S.Free
      END
    EXCEPT
      SetLength(ARR,0);
      RAISE
    END
  END;

It'll save (and reload) an array of TCountry to a file. It'll work in both Unicode and non-Unicode (but the files are not interchangable between the two, ie. a UniCode program can save and reload a UniCode file, but a non-Unicode program cannot reload a file saved by a Unicode-program.

EDIT: If you leave the UTF8 symbol defined, it'll use UTF8 encoding in the binary file. As it stands, it'll only be avilable in the Unicode-enabled Delphi versions (2009 or 2010 and upwards), due to the use of UTF8String

If you are starting afresh then something like JSON will life easier in the long run. — David Heffernan, Mar 05 '14 at 06:28
I was thinking about json, it will be not a problem. But for now, I saved it as XML in UTF-8, and after loading it I convert it back to Unicode. I will try some more tests, which will be the best for me... — Legionar, Mar 05 '14 at 08:58
It's actually quite feasible to write UTF8 even in the older Delphi versions. — David Heffernan, Mar 05 '14 at 19:28
I know - but it takes a bit more code, as there isn't built-in support to do the conversion by simple assignment. I'll leave that excercise up to the user :-). — HeartWare, Mar 05 '14 at 19:29

Best and simply way to store unicode record (array) into a file

1 Answers1