-3

Hello geniuses (dare I say, Minkus'?)

A bit of background. I work for a small, non-tech company that currently does not have a data warehouse. All data is manually pulled from a bunch of sources (let's say different platforms like Facebook and twitter, Client side excel data), and is then stored in excel spreadsheets all over the server.

I'm looking for ways of consolidating this data, and storing it somewhere that will allow me to access the history (currently I have to either hunt down all of the excel sheets and merry them up manually, or go back to Facebook and re-pull the data all over again)

So here is the breakdown:

I have multiple data sources

Ideally something that connects to API's to pull data manually from platforms like Facebook and instagram, but if this doesn't exists, no biggie. Just looking for storage right now.

I'm very time-poor. So no real time to learn a new programming language etc.

I'm happy with something "off-the-shelf" (does that exist?)

Something that will connect to R? I have basic SQL skills, mid level R skills.

Something that will eventually be able to connect to a BI solution like Looker, or to a custom built dashboard.

Is there a business out there that can set something like this up for me, without having to hire a consultant? Is Snowflake one of them?

Sincere apologies (really, I mean it) for the very basic and messy question. I know everyone that will read this will be much more advanced that I am, so I appreciate you all humouring me! (please please keep the tech jargon to a minimum. #idiotproof).

Kat
  • 13
  • 3
  • 1
    From the [help/on-topic]: *Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam.* – Ken White Aug 19 '16 at 02:05
  • Amazon, Azure offer cloud solutions that will probably let you get up to speed pretty quickly. – Nick.Mc Aug 19 '16 at 04:14
  • I have looked into both options. Azure data lake has enterprise level features. Amazon data lake is very ordinary. – Srini Sydney Jul 02 '18 at 07:08

2 Answers2

1

If you are unable to buy or develop a data warehouse/business intelligence system right now, then I would look into some of the Self-Service options that are out there. I know there are alot of good vendors in the area right. However, my background is in the Microsoft On-Premise & Cloud BI Solutions, so I would suggest looking at Microsoft PowerBI or Microsoft PowerQuery/PowerView/PowerPivot Excel pluggins. Both options have some extract, transform and load capabilities as well as data modeling and visualization features. In newer releases they have tried to incorporate a variety of data source capabilities to easily pull data into your workspace.

I have used both tools for POC projects in the past and once a BI program was on budget the knowledge gained and dimensional models produced by the tools aided in the data warehouse design.

SQL Data Ninja
  • 78
  • 1
  • 5
0

If you do have some programming skills, I would recommend you to check python. It is really easy, but managing data transfer will be a small script in the end .It should be understandable to you in just a couple of days. Otherwise, if you have resources, I would recommend to go for corporate solutions since it will save your time.