-1

I need to create an end-to-end platform:

  1. Input data collection and storage - Data will be periodically collected via FTP and stored in cloud.
  2. Data Analysis - The data will be analyzed (using Tableau/ any other analytics software)
  3. Reports - Daily and Monthly reports to be generated based on changing input data.

I need to decide which tool/technology I can use for this solution and how do i host the scripts/processes online for them to run 24*7.

One thought I had was to use AWS & Tableau:

  1. For data collection - Schedule cron jobs in Amazon ECS console
  2. For storage - Store on AWS - probably Amazon S3 storage) But will I have an option to format/change the data before loading to Amazon S3?
  3. Use Tableau to create dashboards. But this will require multiple Tableau sessions, as data source will be different for different reports. Also, I need to send these reports/dashboard in email format to users regularly. Can it be done in AWS?

But, I believe that AWS solution will be costly for company. What other workarounds are possible?

My apologies, I am new to this field and having a hard time trying to figure out a cost-effective solution.

priya
  • 73
  • 1
  • 1
  • 9

1 Answers1

1

Look at building AWS serverless architecture. S3 > lambda > Athena > quicksight

Use lambda function to transform the data coming into s3 in the required format.

Use Athena to partition and query the data and create tables and database.

Use tableau extracts or even better use quicksight.

Charges depend upon the volume of data. This solution will cost less than 10$ a month and support 10 users but it is a vague estimate without any variables.

Animesh Kansal
  • 113
  • 10