0

I want to build a software solution for datafeed aggregation. I dont want to use datafeed aggregation services like Rakuten PopShops or Datafeedr. I am looking for a guidance on how to build such solution from a software architecture point of view.

Which architectural design patterns should I use ? Which technologies should I use ? How to face the problem of non-normalized data formats and APIs from affiliate networks and merchants ? Do you know some book on such topic ?

Since I am comming from a Java Enterprise world, these technologies seems to be a usable components of the solution for me:

Apache Camel, ElasticSearch, NoSQL database (MongoDB), Akka, ZeroMQ, RabbitMQ, Spring Framework, Typesafe Reactive Platform and a lot of other tools.

michal.kartak
  • 1
  • 1

1 Answers1

1

Unfortunately there is no single answer to your question. I would say stop thinking about solutions/patterns at this point. Try to figure out what exactly you want to do from a requirements perspective. What kind of data you want to aggregate, where to get the data from, data scrubbing rules, legal issues etc. Once you have that nailed down, take the easiest path to implement it. Technologies you have used in the past/comfortable with. Then add in other technologies once you find out that your existing solutions will not work.

Mahesh Guruswamy
  • 769
  • 5
  • 15