I currently use a local Tomcat/JSP service/webpage to get data with sql and display it on the webbrowser with plotly (locally rendered with this javascript library). I want to do more data wrangling by the user like filtering and cannot do it with sql since the sql query takes several seconds and several sql queries for one webpage would take too much time. Within JSP data filtering is quite complicated since you have to write you own filter functions like https://blog.pavelsklenar.com/using-filteredrowset-simple-example/ or http://www.java2s.com/Code/Java/Database-SQL-JDBC/FilteredRowSetDemo.htm . If you know Pythons Pandas or sql this seems too complicated.
I wonder if Zeppelin is a good solution with these steps:
- Get the data directly via sql (already works) but the integrated display possibilities are nice but not sufficient for me.
- A user can adjust several filters (e.g. column1 between 1 and 3 and so) using the AngularJS view (https://zeppelin.apache.org/docs/0.7.3/displaysystem/front-end-angular.html).
- Read the filter ranges from the last paragraph and do data wrangling with python’s pandas with some filters and save it a new dataframe.
- Use this filtered dataframe and plot the data with a local plotly.js file (I have the pure plotly JS code). In general, it could work with an %angular paragraph as Using d3.js with Apache Zeppelin (even without the code from “else”, so d3 seem to be useable within zeppelin out of the box). But I wonder
- how to pass the dataframe from Python to the Angular node. According to https://zeppelin.apache.org/docs/0.7.3/displaysystem/back-end-angular.html I have to use Spark to use the variables; so it seems that I have to integrate a spark wrapping paragraph. After that do I have to write code that converts the dataframe to JSON or similar?
- how to use plotly or other local javascript files? How can I specify the location?
What do you think or do you have other/better ideas?
