7

First of all I am answering my own question Q/A style, so I don't necessarily need anyone to answer this. It's something I've learned and many can make use of this.

I have a tree view which consists of many different nodes. Each node has an object behind it in its Data property, and the objects refer to different levels of hierarchy from one master list of objects, which is quite large (many thousands of items). One node represents a specific property on this main listed object, where the tree allows the user to select a node to view those items which fall under that specific selected category.

When the tree is being populated, it becomes extremely time consuming (in some cases 2 minutes), because each node needs to iterate through every item in this large list and find each item in this list which falls under any given node. Therefore, if there are to be 500 nodes in this tree, then it iterates through this large list 500 times. There are a total of 3 levels of hierarchy - the performance choke comes in when loading the second and third levels, but the first level is simple and quick.

Now there aren't any options to improve the performance of iterating through this list hundreds of times. What I'm wondering is are there any known tricks to improve the performance of populating the tree view?

Here's how it currently works:

var
  X: Integer;
  N: TTreeNode;
  O: TMyObject;
begin
  for X := 0 to MyObjectList.Count - 1 do begin
    O:= TMyObject(MyObjectList[X]); //Object which Node represents
    N:= TreeView.Items.AddChild(nil, O.Caption);
    N.Data:= O;
    LoadNextLevel(N); //Populates child nodes by iterating through master list again
  end;
end;

And a similar method for each additional level.

PS - The first level of hierarchy is populated from a separate list of its own (about 50 objects) whereas the second and third levels are populated from the properties of these many thousands of objects in a master list. This is why the first level loads quickly, and the rest is slow.

Community
  • 1
  • 1
Jerry Dodge
  • 26,858
  • 31
  • 155
  • 327
  • How about pre-sorting the data so it can be looped through sequentially so you only have to scan through it once? Like the data in your other question: [How to populate a tree view based on a flat list with “levels”?](http://stackoverflow.com/questions/19772367/). – Remy Lebeau Nov 13 '13 at 03:43
  • @RemyLebeau That question was a completely separate topic from a different project, this project is quite a bit different. – Jerry Dodge Nov 13 '13 at 03:53
  • @JerryDodge: that does not negate my question. If sorting the tree data is an option, that could help you load the data into the tree easier. – Remy Lebeau Nov 13 '13 at 05:55
  • @RemyLebeau Honestly, the data in this massive list is SKU's of a Vendor. and I'm sorting it already by the Vendor's SKU code. Most of the time, the SKU code is structured in a way to be automatically sorted in its hierarchy, but not always. – Jerry Dodge Nov 13 '13 at 06:04

2 Answers2

15

If you really care about speed of populating a massive treeview, you should look into virtualTreeView (http://code.google.com/p/virtual-treeview/).
It's an open source treeview specifically designed to be virtual and maximize speed/memory for large treeviews.
It's an amazing component.

Prashant Gaur
  • 9,540
  • 10
  • 49
  • 71
Dave Novo
  • 693
  • 3
  • 9
  • But while everyone admits it, the VTV project itself is dying without contributors and maintainers... – Arioch 'The Nov 13 '13 at 07:49
  • 3
    @Arioch, no way. There is still Joachim Marder, which currently maintains the project. And he's doing it right. You [`report a problem`](http://stackoverflow.com/q/19030376/960757) on StackOverflow and you have a fix in 4 days :-) I don't know where you heard this, but there are even people who offer contracts for porting to FMX, etc. VTV never dies! And even if that would gonna happen, VTV is stable enough even for tasks, which you can't even think about with the system tree view. – TLama Nov 13 '13 at 09:17
  • Yes, and many thanks to him. But he is alone. "Bus factor" = 1. // They can offer whatever they wish, that does not matter. The question is whether those contracts were fulfilled, whether VTV was portedto FMX or .Net or whatever but VCL. To me it means, that feature request for VTV was not fulfilled even for money offer. // Again, i agree that VTV is far more advanced than TTreeView. But that tells only about its past, not about its current state. – Arioch 'The Nov 13 '13 at 11:27
  • VTV is indeed a great performing alternative, but the problem I was addressing is not a problem with the tree view but rather the method required to load each node. Even using VTV it would be slow. – Jerry Dodge Nov 13 '13 at 12:34
  • @Jerry Why would VTV be slow? Why would VTV have to load up front? – David Heffernan Nov 13 '13 at 14:19
  • @David I'm not saying VTV its self would be slow, but in my question I explain how for every node it needs to add, it needs to iterate through a massive list of objects and compare properties of each item, so for a list of let's say 10,000 items, and a tree view of 500 nodes, that's 5 million total iterations. – Jerry Dodge Nov 13 '13 at 14:22
  • 1
    Yes. But why would VTV do all that up front? It's virtual. The whole point is that you only do things when you need to. And 10,000 items can be processed trivially. Because that happens outside the GUI. – David Heffernan Nov 13 '13 at 14:28
  • Well thanks for hijacking my Q/A, of course VTV is the ideal solution to tree view performance, but again, it all boils down to the fact that when using the VCL Tree View control, there is a trick to improve performance in scenarios such as mine. In fact, this qualifies for any and all programming languages which might have to do with a tree view, even in HTML. – Jerry Dodge Nov 14 '13 at 02:58
6

There is a common trick in tree views to improve the performance in this type of situation. When you refresh this tree view, only load the first level of hierarchy, and don't worry about any further levels. Instead, you can load each additional level at the time of expanding each node. Here's how to do it.

When you populate the first level, rather than continuing with loading each of its child nodes, instead just create 1 "dummy" child node with a nil pointer in its Data property - since each node is expected to have an object in the Data property anyway. Then, monitor the OnExpanding event of the tree view. When a node is expanded, it will check to see if this "dummy" child node exists or not. If so, then it knows it needs to load the child nodes.

When the first level of hierarchy is loaded...

var
  X: Integer;
  N, N2: TTreeNode;
  O: TMyObject;
begin
  for X := 0 to MyObjectList.Count - 1 do begin
    O:= TMyObject(MyObjectList[X]); //Object which Node represents
    N:= TreeView.Items.AddChild(nil, O.Caption);
    N.Data:= O;
    N2:= TreeView.Items.AddChild(N, '');
    N2.Data:= nil; //To emphasize that there is no object on this node
  end;
end;

Then, create an event handler for OnExpanding...

procedure TForm1.TreeViewExpanding(Sender: TObject; Node: TTreeNode;
  var AllowExpansion: Boolean);
var
  N: TTreeNode;
begin
  N:= Node.getFirstChild;
  if N.Data = nil then begin
    //Now we know this is a "dummy" node and needs to be populated with child nodes
    N.Delete; //Delete this dummy node
    LoadNextLevel(N); //Populates child nodes by iterating through master list
  end;
end;

The only disadvantage to this trick is that all nodes which have not yet been expanded will have a + next to them, even though there may not be any child nodes. If this is the case, then when the user clicks the + to expand a node, the child node gets deleted and the + disappears so the user knows there are no child nodes within that node.

Also, using BeginUpdate and EndUpdate in TreeView.Items improves performance by not performing GUI updates until it's all done...

TreeView.Items.BeginUpdate;
try
  //Refresh the tree
finally
  TreeView.Items.EndUpdate;
end;
Jerry Dodge
  • 26,858
  • 31
  • 155
  • 327