TMemo is painfuly slow when working with large number of lines

Question

I have 100000 lines in a TMemo. I want to do something like:

 for i:= 0 to Memo.Lines.Count-1 do
  Memo.Lines[i]:= SomeTrim(Memo.Lines[i]);

but the speed is 0.5 lines per second!!

After adding BeginUpdate/EndUpdate I don't see any speed improvement.

 Memo.Lines.BeginUpdate;
 for i:= 0 to Memo.Lines.Count-1 do
  Memo.Lines[i]:= SomeTrim(Memo.Lines[i]);
 Memo.Lines.EndUpdate;

My question is why BeginUpdate/EndUpdate won't help?

PS: The current solution is to assign the lines to a TStringList, process them, and put them back into the memo. But I am still curious why BeginUpdate does not work. — Gabriel, Oct 28 '17 at 14:55
@Victoria - The user will put those lines there. Usually, I expect under 100 lines. I wanted to test to see what happens with 100000. And this is what happens. — Gabriel, Oct 28 '17 at 14:58

Uwe Raabe · Accepted Answer · 2017-10-29T15:19:15.043

14

TStrings.BeginUpdate/EndUpdate will only prohibit the OnChanging and OnChanged events. It has no influence on the internal handling of the changes to the content itself.

TMemo.Lines is implemented by TMemoStrings which stores the text content in the Window control itself. Thus BeginUpdate/EndUpdate is pretty useless here.

You might get better results by using a local TStringList instance, and using the Text property to copy the data from TMemo to TStringList and back. The Text property is the most efficient way to access the whole content of a TMemo at once.

  lst := TStringList.Create;
  try
    lst.Text := Memo1.Lines.Text;
    for I := 0 to lst.Count - 1 do begin
      lst[I] := SomeTrim(lst[I]);
    end;
    Memo1.Lines.Text := lst.Text;
  finally
    lst.Free;
  end;

Note: Some comments mention to use Assign instead of the Text property when copying the content from and to the Memo: Assign is significantly slower in this case due to an internal optimization of the Text property for TMemoLines. The Getter and Setter of this property accesses directly the Windows control with a single WM_GETTEXT/WM_SETTEXT message, while Assign uses one EM_GETLINE message per line for reading and a sequence of EM_LINEINDEX, EM_SETSEL, EM_LINELENGTH and EM_REPLACESEL per line for writing. A simple timing test shows that the above code needs about 600 ms while replacing the Text assignments with Assign calls needs more than 11 seconds!

edited Oct 29 '17 at 15:19

answered Oct 28 '17 at 15:01

Uwe Raabe

45,288
3
82
130

6

Use the `Assign()` method instead of the `Text` property: `lst.Assign(Memo1.Lines); ... Memo1.Lines.Assign(lst);` Since the lines are already separate, just copy them as-is, don't concatenate them just to re-parse them, that is a waste of memory and processing – Remy Lebeau Oct 28 '17 at 15:46
@RemyLebeau, getting and setting each line separately is actually what the original code did. – Uwe Raabe Oct 28 '17 at 15:53
10

I'm aware of that. I'm referring to copying the Memo content to the StringList and then back. Your answer says the `Text` property is the "most efficient" way to do that, but that is not even remotely true, especially for 100000 lines. The `Text` getter performs a 2-pass scan to allocate and copy memory, then the `Text` setter parses the input to perform a LOT of allocatations. `Assign()`, on the other hand, performs far fewer allocations and uses a 1-pass scan. Profile it for yourself, `Assign()` is more efficient than `Text` – Remy Lebeau Oct 28 '17 at 16:02
@UweRaabe - Can you give an example where begin/end update makes a difference. – Gabriel Oct 28 '17 at 19:13
6

@RemyLebeau, TStrings.Text property uses the virtual GetTextStr and SetTextStr methods, which are overriden in TMemoLines. These implementations access the controls text content through a single WM_GETTTEXT or WM_SETTEXT message call. I tested it and using Text is way faster (like 18x). – Uwe Raabe Oct 28 '17 at 20:23
@UweRaabe - Nice to know!!! This applies only to TMemo? Do you know other classes to which this applies too? – Gabriel Oct 30 '17 at 08:57
That applies to TMemo and it may as well apply to other classes, although I am currently not aware of any (mostly because I din't investigate yet). – Uwe Raabe Oct 30 '17 at 13:24
Uwe is right, see test below. – Gabriel Nov 23 '22 at 15:28

Gabriel · Answer 2 · 2022-11-23T15:28:05.933

Test and results:

{-------------------------------------------------------------------------------------------------------------
   Conclusion 1:
       BeginUpdate has (a positive) effect ONLY if you add items one by one in a visual control (TMemo, TListBox)

   Conclusion 2:
       If you want to transfer the items from a TStringList to a TMemo, .Text is much faster than .Assign
-------------------------------------------------------------------------------------------------------------}



{ ListBox, adding 10000 items:
  61ms with BeginUpdate 
  1340ms without BeginUpdate }
procedure TfrmMain.btnInsertClick(Sender: TObject);
var
  I: Integer;
begin
  TimerStart;
  ListBox1.Items.BeginUpdate;
  TRY
    for I := 1 to StrToInt(Edit1.Text) do
      ListBox1.Items.Add(IntToStr(I));
  FINALLY
    ListBox1.Items.EndUpdate;
  END;

  Caption:= 'Inserting: '+ TimerElapsedS;
  Label3.Caption := 'Items : ' + IntToStr(ListBox1.Count);
end;


{ Memo.Lines: 1800ms 
  BeginUpdate makes no difference  }
procedure TfrmMain.btnLinesClick(Sender: TObject);
begin
  btnClearMemoClick(Sender);
  TimerStart;

  Memo1.Lines.BeginUpdate;
  try
    Memo1.Lines := ListBox1.Items;
  finally
    Memo1.Lines.EndUpdate;
  end;

  Caption:= TimerElapsedS;
end;



{ Memo.Lines.Add: 1900ms 
  BeginUpdate makes no difference }
procedure TfrmMain.btnLinesAddClick(Sender: TObject);
begin
  btnClearMemoClick(Sender);
  TimerStart;
  Memo1.Lines.BeginUpdate;
  try
    for VAR I := 0 to ListBox1.Items.Count - 1 do
      Memo1.Lines.Add(ListBox1.Items.Strings[I])
  finally
    Memo1.Lines.EndUpdate;
  end;
  Caption:= TimerElapsedS;
end;


{ 1900ms | BeginUpdate makes no difference }
procedure TfrmMain.btnAssignClick(Sender: TObject);
begin
  btnClearMemoClick(Sender);
  TimerStart;

  Memo1.Lines.BeginUpdate;
  try
    Memo1.Lines.Assign(ListBox1.Items);
  finally
    Memo1.Lines.EndUpdate;
  end;

  Caption:= TimerElapsedS;
end;


{ Fill a TStringList and assign it to the Memo }
procedure TfrmMain.btnTSLClick(Sender: TObject);
begin
  Caption:= '';

  { 0ms }
  btnClearMemoClick(Sender);
  TimerStart;
  VAR TSL:= TStringList.Create;
  for VAR I := 1 to 10000 do
    TSL.Add(IntToStr(i));
  Caption:= 'Create TSL: '+ TimerElapsedS;

  { TEXT: 64ms with or without BeginUpdate }
  TimerStart;
  Memo1.Lines.BeginUpdate;
  Memo1.Text:= TSL.Text;
  Memo1.Lines.EndUpdate;
  Caption:= Caption+ '.    Text: '+ TimerElapsedS;

  { ASSIGN: 1960ms | BeginUpdate makes no difference }
  btnClearMemoClick(Sender);
  TimerStart;
  Memo1.Lines.BeginUpdate;
  Memo1.Lines.Assign(TSL);
  Memo1.Lines.EndUpdate;
  Caption:= Caption+ '.    Assign: '+ TimerElapsedS;

  FreeAndNil(TSL);
end;

So, Uwe is right.

TMemo is painfuly slow when working with large number of lines

2 Answers2

Linked