0

Is there a light weight open source tool for dispatching jobs under distributed system? I'd like to dispatch config files to slave servers，one config file stands for one job.

Managed server maintains all config files, it also maintains a mapping table which records mapping between config file and slave server.

The requirement seems simple:

For example, There are 10 slave servers and 5000 config files(jobs). First, 10 slave servers got 500 config files separately, just put the config files under config folders in slave servers. If some of config files are modified or deleted. Managed server will search mapping table and notify the slave servers to sync the change. If some new config files are added. Managed server will dispatch these config file to some of the slave servers.

Seems I can achieve this via RMI, is it safe?

stealthjong
  • 10,858
  • 13
  • 45
  • 84
Bruce_Han
  • 1
  • 2

1 Answers1

0

See if Spring batch is useful for this case. It is useful for execution of a series of jobs. It consists of READ-PROCESS-WRITE, i.e. read data from resources (xml, csv, db), process it and write to db. It processes these in batches and also maintains which batch processed successfully and which did not so that the unsuccessful could be restarted again. It solves many use case, read files from folder X, process the data and write to folder Y, etc..

Sandeep B
  • 765
  • 1
  • 6
  • 19