Huge Data in Listbox(Table)

Question

I want to show a listbox(Table) with nearly 20 Million rows. How can I do so, with lower memory usage and not letting my server die(stop responding) while doing so.

Even you have any Theoretical idea please do share(I will try to implement). Need solution very urgently.

I know I cannot load all the rows at once. I need to ask new rows from server every time I scroll. I have tried it but my scroll is not smooth enough.

Thanks & Regards, Aman

in all seriousness with the amount of data you are talking about i'd look for a prebuilt component to do the heavy lifting here otherwise you'll spend a long time doing to optomisation that some of the component vendors have spend months and months doing. — krystan honour, Apr 03 '12 at 10:17
A listbox or table loaded with 20m *anything* at once is a bit crazy when relating to any of the tags in your question, what are you exactly trying to do? — Alex K., Apr 03 '12 at 10:17
@krystanhonour can suggest any such Open Source component which I can got through? — , Apr 03 '12 at 10:33
Nope for that sort of information you're going to have to stream it in on scroll or its going to suffer from major issues performance wise @Erik down below alludes to the same thing. Do not know any open source thing to do this unfortunately I'd (meaning what I would do) use a major pay-for vendor for this. — krystan honour, Apr 03 '12 at 10:42

score 3 · Accepted Answer · answered Apr 03 '12 at 10:16

3

Why not just retrieve the first 100 entries and then once the client scrolls to the bottom you append another 100 entries and so on.

answered Apr 03 '12 at 10:16

erikvimz

5,256
6
44
60

next? can u pls explain with more details. – Apr 03 '12 at 10:18
Load 100 entries and check the position of the scroll bar, if the scroll bar is at the bottom of the page (all entries have been viewed) you send an ajax request and add another 100 entries after the already existing ones. – erikvimz Apr 03 '12 at 10:20
In that case I need to know the height of each row so that I can set the height of my scroll accordingly. but I have data in my rows such that they cannot fit in one line they are wrapped to next line – Apr 03 '12 at 10:24
1

Not necessarily. Just check weather the scroll bar is at the bottom. This might help you: http://groups.google.com/group/comp.lang.javascript/browse_thread/thread/bf9941d696f229f9?pli=1 – erikvimz Apr 03 '12 at 10:30

score 1 · Answer 2 · answered Apr 05 '12 at 04:03

1

Maybe you could wait for ZK's new feature. Reference http://books.zkoss.org/wiki/Small_Talks/2012/March/Handling_a_Trillion_Data_Using_ZK

answered Apr 05 '12 at 04:03

TonyQ

657
7
8

score 0 · Answer 3 · answered Apr 04 '12 at 08:20

You could use http://books.zkoss.org/wiki/ZK Developer's Reference/MVC/View/Renderer/Listbox Renderer.

public void render(Listitem listitem, Object data, int index)

To start, you can implement render in way so that you get element to render from datasource at hand by index from render method. You can use standard cache (if Hibernate is in place) or custom-written one if otherwise (look also at EhCache).

@Erik solution is really fast to implement. To add you could make a button, so that user would be aware that loading more records would cost some time and would think if one really needs to load more. Scrolling can make you Listbox just hang up for a moment.

And always make an upper constraint on maximal number of records you will show at one time - don't pollute your server's memory.

score 0 · Answer 4 · answered Apr 05 '12 at 05:29

0

Make paging for the table values and retrieve specific number of records on demand.

Use can use dataTable pluging to make pagination for data records.

Notice that you can retrieve data in synchronous and asynchronous way using this library

answered Apr 05 '12 at 05:29

palAlaa

9,500
33
107
166

Huge Data in Listbox(Table)

4 Answers4