0

Have written an application in Python that handles a large amount of data used to generate graphics. I need help finding a technique to allow the user to scroll through the data, in both directions, while displaying a segment of the data in a graphic. Ideally only data which would be visible would be read from the database.

I am currently using a combination of Pandas, Sqlalchemy and bqplot, but if there are better packages for implementing the desired functionality I am willing to change.

Sqlalchemy seems like a good bet for handling the database but I have found the documentation hard to understand and need a nudge in the right direction. Any help or advice will be appreciated.

Thanks, Steve

Stephen Sackett
  • 107
  • 2

1 Answers1

0

You can store your data in a grid whose each rectangle is the size of the user's display:

enter image description here

This way, you need to load at most 4 rectangles.

As to the database, sqlalchemy is great, but it does fancy ORM stuff you don't need. In fact, as a rule of thumb, if you're not going to do table joins (which, by your description you're not), you don't need an SQL database at all, but rather a large key-value store. Here is a pymongo tutorial, for example, but just choose any key-value store that works for you.

Ami Tavory
  • 74,578
  • 11
  • 141
  • 185