0

I've got a web app where I use plain file system for my custom logs - a lot of small files, I don't want to put that into db, that works for me quite well. But now I need to scale my app by using a load balancer in front, so I also need to keep those logs in sync between servers. Is there any reliable solution for such cases ? I know I could sync it by some OS means, or by scripting, but I'm thinking if there is any better solution for such scenarios? Is it the case for MongoDB usage or something more modern or is it better to keep it on file system as plain files ?

Kubber
  • 367
  • 3
  • 13

1 Answers1

0

This questions is going to get you some heat since essentially your asking for our opinion. Ill be frank tho and wont argue with anyone since its just MY opinion. With web apps in my humble opinion, its always better to keep your data in a DB for scalability but also for analytical research. I know little about what your app does but its easier to write third party data apps that tell you how many of X or Y etc when its centrally stored in a DB. Since the app that gets said data can be anywhere. I know I probably wasted time with an argument but hey, hope I helped a bit.

Scott Johnson
  • 369
  • 2
  • 15
  • Thx Scott, yes I agree with you, these are log files but too often I have to write some scripts to find some text inside for various conditions. So – Kubber May 21 '15 at 06:03
  • Thx Scott, yes I agree with you, these are log files but too often I have to write some scripts to find some text inside for various conditions. So I will narrow my question a bit - which database would suit best for such data? I mean up to 100Mb daily from small log files - just text data, no relations - table would look like : id,created_at, content . I would need it to be easily searchable by text comparison, preferably data stored compressed (if that's possible), and I should be able to delete old data to free space if necessary i.e. afaik with mysql deleting might not free space – Kubber May 21 '15 at 06:09
  • Well if your looking for a good logging setup I reccomend Logstash, Elasticsearch, and Kibana4. Logstash can log anything, elasticsearch makes it searchable to an extreme, and kibana adds graphing and gui to it all. Check out this tut and see if its something that meetes your needs.https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-4-on-ubuntu-14-04 – Scott Johnson May 22 '15 at 15:35