1

I have a mongo collection with ~20 millions documents and for each document a timestamp will be obtain from ObjectID(_id) and around 10 other values. In each document is a identifier key that is used to later group them, I have ~200 unique identifiers. For a given identifier I will collect the 10 values+timestamp from each document marked with that identifier and then plot it in various combinations (eg: timestamp + values:1,3,4,5,6 for a graph).

So far I have created a flask app that serves as a backed and angularjs for the frontend. I can send the identifier from angular and using pymongo to numpy arrays and matplotlib or seaborne on the python side I can create and serve static ephemeral png to be presented on the frontend.

I want to create interactive graphs to be zoomed, I've tried with dygraph on client side and I've tried with bokeh on the serverside. Both without success as the data is too big (cumulating ~1 mil datapoints for a page). I need a free application/library, I can have a load time of no more then ~3 minutes. I need to keep it as simple as I can (no prerendering with saved images, if possible auto or no downsampling).

What can I try next, can Datashader be usefull in my case?

Arkon-v
  • 126
  • 1
  • 8
  • Yes, datashader is a good target for you here. Also look at the web backend for mpl (it runs it's own tornado server) but you can capture mouse/key events on the browser side, send them back to the server and re-render the figure as needed (this is how webagg and the notebook backends work). – tacaswell Nov 09 '16 at 17:20
  • Thak you, I'll look into mpl backend. Does someone have a link to any resource explaining a datashader and angularjs or plain html integration? – Arkon-v Nov 09 '16 at 22:52

0 Answers0