1

I have a production system use-case where my controller code depends on some external files (Metadata information of some relevant business logic; 3-5 JSON files which in total would amount to 1GB of data) which gets updated frequently to create features for my data science model.

I'm trying to think about what would be the best approach to copy these external files on all machines serving my request. Imagine I've 50 EC2 machines running right now serving requests from the load balancer and I receive a notification from SNS that the external files have been updated. What would be the best way to fetch new metadata file in all 50 EC2 machines? I received a comment from Senior that I should use an event-based mechanism instead of polling.

Controller code is written in Java - Spring Boot, Cloud technology in use - AWS

here_to_learn
  • 179
  • 2
  • 11
  • To update your EC2 instances, SSM Run Command, Automation, State Manager or Maintenance Windows can be used. There is a lot of possibilities. Unfortunately, I don't understand the comment about "event-based mechanism instead of polling". Can you explain what you mean? "Event-based" like in using CloudWatch Events or EventBridge to trigger the updates? The 1GB config files can be stored in S3. – Marcin Apr 25 '20 at 07:42
  • Since the config files can be dropped in S3, this could be a trigger/event that could alert the production system to fetch the file (Event-based). Polling in my case was checking s3 bucket if new files are added to be fetched. – here_to_learn Apr 25 '20 at 08:00
  • I see. Yes, so you can set S3 event notification to trigger for example SNS or Lambda. And the rest depends how the 50 instances are setup? Your question does not provide any details. For example, if they are using Systems Manger (SSM), then you can just trigger SSM Run Command to update all 50 instances at once. – Marcin Apr 25 '20 at 08:15

0 Answers0