0

I have to build a small viewer application that can import a large archive table from a USB stick, several million row with 4 fields (two Uint64 ID's, a timestamp and an Int32 ID). The application needs to simply show the data, allow for sorting by any column and/or filtering the data by any of the fields. No grouping, no images or tree-like structures. I am at the point where I have a List in place that is filled with objects as described above.

I'm more used to server backends with web frontends where all is about lazy loading, but now I have a giant list of objects already in place (in memory), but when I tried to use a DatagridView, the performance was not acceptable at all.

Before I reinvent the wheel and implement lazy loading from my large List: how can I get a fast and responsive table/grid for my data? This must be something that is needed very often, I suppose.

Rob
  • 11,492
  • 14
  • 59
  • 94
  • 'This must be something that is needed very often, I suppose.' no not particularly - why would you think that? – sommmen May 14 '20 at 08:50
  • 1
    I guess that you shouldn't upload all data to ram as once – Daniel Haish May 14 '20 at 08:53
  • also read this; https://learn.microsoft.com/en-us/dotnet/framework/winforms/controls/implementing-virtual-mode-wf-datagridview-control im used to wpf so i cannot advise on winforms. I'd check out any libraries online because there should be something there already. – sommmen May 14 '20 at 08:54
  • @sommmen Maybe not in your field of work, but having a large pile of data from which you want to extract relevant information seems not that exotic to me. – Rob May 14 '20 at 08:58
  • Does this answer your question? [C# Virtual List View in WinForms](https://stackoverflow.com/questions/24195868/c-sharp-virtual-list-view-in-winforms). Win32 has had this for [sometime](https://learn.microsoft.com/en-us/windows/win32/controls/use-virtual-list-view-controls) now and the feature was included in WinForms. Essentially you tell the control it is in _virtual_ mode showing millions; you maintain a cache, and Windows simply pings you when it needs to show a visible row. It's very memory-efficient –  May 14 '20 at 09:00
  • ...https://learn.microsoft.com/en-us/dotnet/api/system.windows.forms.listview.virtualmode?view=netcore-3.1 –  May 14 '20 at 09:03
  • Thanks, MickyD, that sounds promising. Seems like virtual mode is lazy loading in Winforms speech. – Rob May 14 '20 at 09:04
  • No problem good sir. Hehe indeed. –  May 14 '20 at 09:05
  • I suggest to check out ObjectListView. http://objectlistview.sourceforge.net/cs/recipes.html Specifically the `FastObjectListView` or the `VirtualObjectListView` which are designed for speed and loading of large datasets. The documentation talks of 10m rows! – jason.kaisersmith May 14 '20 at 09:06
  • 1
    @Alex now you're talking about extracting relevant data, whilst first you were talking about displaying that data. Im saying its rare that you need to have a list that contains huge amounts of items to display. For example the app im working on handles thousands of db entities and displays them in a way, however we show results by day, or group based on a data query so the output always is manageble. The larges display we have at on time could be like 1000 or so items - but most ui fx handle that okay oout of the box. Just saying its rare that you actually display large amounts of data – sommmen May 14 '20 at 09:15

0 Answers0