0

I am working on a kinesis POC. I have a small python script running to generate stream data. The streaming data is passed via a kinesis agent, kinesis firehose, and finally the data is inserted into Redshift table at regular interval of 5 minutes.

I wanted to view the updated records by way of a graphical representation so as to see the trend of data as the data comes in.

I tried with AWS Quicksight, but the issue with Quicksight is that the scheuled refresh interval is daily/weekly/monthly, so it doesn't solve my purpose to view update data every 5 minutes.

Is there any visualisation tool that i can connect to redshift? Please help me with suggestions.

Thanks

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
Yuva
  • 2,831
  • 7
  • 36
  • 60
  • 1
    Maybe you can try Geckoboard ? Don't have much experience but it's good. – Varun Chandak Apr 13 '18 at 10:18
  • Thanks, but am looking for an open source tool, so I can try out for my POC (development). Kibana is one open source but i believe it works with ElasticSearch/Logstash. So looking for some other open source tools. – Yuva Apr 13 '18 at 10:44

1 Answers1

1

After some googling, I found Metabase to be a good tool to satisfy my requirement (to view real time data updates on a dashboard, with auto refresh scheduled), and I find it awesome for the following reasons:

  • Open source, for Windows/Mac, download the jar file and run it using “java -jar metabase.jar” and you are ready to go. (JRE is required)
  • Can be run on Docker, AWS EBS, and other environments as well. Can be connected to many databases such as AWS Redshift, BQ, Druid, Google Analytics, MongoDB, PostgreSql, MySql, SqlServer, etc.
  • Dashboard auto refresh at 1,5,10,15,30,60 minutes. Can be integrated with Slack, Mails for sharing updates/alerts, etc.
  • Interactive queries, many report formats, dashboard (day /night mode).

  • LDAP integration, and provisions for SSH based db connections.

  • Ease of use

https://www.metabase.com/start/

I was able run my kinesis agent running on a EC2, capture the streaming data to a Redshift table via Kinesis Firehose. The I download metabase.jar file on my windows and configured redshift database in my metabase, ooh, am able to view streaming data on metabase dashboard. We can schedule autorefresh every 1 minute,5 minute, or 15, 30, 60 minutes.

Yuva
  • 2,831
  • 7
  • 36
  • 60