I have an application with a web-app, which basically accepts upload of some log files and there are some processes which are supposed to be acting on the received log files sequentially. In its current implementation, every process receives the files from the file system and performs some processing and outputs the processed file to some other folder and same is deleted from the input folder. Now, I want to know if it's the best possible approach, though, I doubt that. What should be the best possible approach to follow here? Should I be using some messaging system like RabbitMQ for this? Should I pass the entire file in the queue or only a directory / file name and keep the file on some other common(eg, NAS) or distributed file system?
Asked
Active
Viewed 121 times
1 Answers
0
It depends a lot on your requirements and setup. For example, if by processes you mean physically independent nodes, then yes a Message Oriented Middleware would make sense in terms of enabling the communication between publisher/subscriber nodes. Also, another advantage is that you offload the node which accepts the input files and distribute the business logic on other nodes.
Andrei
- 7,509
- 7
- 32
- 63