I want to read the unread mails of 10 mail accounts in a multi-thread way.
But if the thread pool size is 5, then 5 threads will be used from the thread pool. Each thread will read one mail account. So once the Thread_1 has read the first mail box, it should read mailbox_6. Then thread 2 will read mailbox_7.
When all mail account have been read once, the cycle will start from 1st mail account.
How can we do this in java?
This should be pretty easy. You create a fixed-thread pool with 5 threads and then you submit the 10 jobs to the pool -- 1 per each user email account:
// create a thread pool with 5 workers
ExecutorService threadPool = Executors.newFixedThreadPool(5);
// submit all 10 user email accounts to the pool to be processed in turn
for (UserEmail userEmail : userEmailsToProcess) {
threadPool.submit(new EmailProcessor(userEmail));
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
...
// here's our runnable class which does the actual work
public class EmailProcessor implements Runnable {
private UserEmail userEmail;
public MyJobProcessor(UserEmail userEmail) {
this.userEmail = userEmail;
}
public void run() {
// read the user email
...
}
}
The UserEmail class could hold the filename of the email to "read" or maybe the account name or something. That will be up to you to figure out how to represent the mail account and mail to be read.
[[ From the comments: ]]
i have 10 mailboxes like.. mailbox1...mailbox10, now i have 5 thread from thread pool, so thread1 will fetch any mailbox so lets assume it will pick mailbox1 , then thread2 will pick mailbox2, thread3 will pick mailbox3, thread4 will pick mailbox4 and thread5 will pick mailbox5, now when thread1 (with specific time period which is already defined) will free then it should be pick mailbox6 - mailbox10, anyone which is yet not read it should not pick any mailbox from mailbox1 - mailbox5, up to all mailbox which yet not read.
Oh I see. One solution is to have a dispatch thread that is sleeping and waking up every so often to see if the mailbox's have any mail. If they do then it subjects a job to the thread-pool for that mailbox to be read. Once the mailbox has been read the thread goes back and asks for the next mailbox to process. The dispatch thread will continue to add mailboxes to the thread-pool until it is told to stop.
If there is a lot of context in an EmailProcessor then you could have your thread-pool but they could consume from a BlockingQueue<File> or something which tells them what mailbox needs attention.
How about simply creating a concurrent (or synchronized, doesn't matter much in this case) collection (maybe a queue) of the tasks. Each task will load the needed data from the email accounts. Then let each of the threads take a task from the collections, until it is empty.
Each of the threads will have a reference to this collection and will do a loop on it. While the collection is not empty, take a task from it and handle it.
Keep track of the email accounts that are read. For example define something like this,
//total number of email accounts that need to be read.
private int noOfEmails=10;
//the thread pool that is used to read the emails
private ExecutorService threadPool = Executors.newFixedThreadPool(5);
//the tracker array that keeps track of the emails that
//are already read. This array is cleared only after all 10
//emails are read.
private ArrayList<String> emailTracker=new ArrayList<String>(noOfEmails);
//any changes to emailTracker should be synchronized as
//this is a common data shared between all 5 threads.
private Object syncObject=new Object();
In the Runnable implementation, check if the emailTracker contains your email account identifier, if so that implies it is already read, so return and wait till the emailTracker is cleared. It will be cleared when all 10 email accounts are read.
if(emailTracker.contains(email.identifier))
{
return;
}
//read email.
email.read();
//simple synchronization.
synchronized (syncObject)
{
//read email
emailTracker.add(email.identifier);
//if all emails are read, clear the tracker
//This will allow reading of all the emails
//once again.
if(emailTracker.size()==noOfEmails)
{
emailTracker.clear();
}
}
Executors may not be the optimal solution here but for simplicity I'll use a variation on Gray's code. In order to continuously scan your 10 mailboxes, you can do the following although you'll have to add some code to handle clean termination:
// Create one semaphore per mailbox
Semaphore semaphores[] = new Semaphore[10]
for (int s = 0; s < semaphores.length; s ++) {
semaphores[s] = new Semaphore(1);
}
// create a thread pool with 5 workers
ExecutorService threadPool = Executors.newFixedThreadPool(5);
// submit all 10 user email accounts to the pool to be processed in turn
for (int i = 0; i < 5; i ++) {
threadPool.submit(userEmailsToProcess, semaphores);
}
// once we have submitted all jobs to the thread pool, it should be shutdown
threadPool.shutdown();
...
// here's our runnable class which does the actual work
public class EmailProcessor implements Runnable {
private UserEmail userEmailToProcess[];
private Semaphore semaphores[];
public MyJobProcessor(UserEmail userEmailToProcess[], Semaphore semaphores[]) {
this.userEmailsToProcess = userEmailToProcess;
this.semaphores = semaphores;
}
public void run() {
while (true) { // you could use a semaphore here to test program termination instead
for (int s = 0; s < semaphores.size; s ++) {
if (semaphores[s].tryAcquire()) {
UserEmail email = userEmailToProcess[s];
// read the user email
…
semaphores[s].release();
}
}
}
}
}
This is a quick and dirty solution that is not 100% fair but it will work for any number of threads and mailboxes. In your special case with 10 emails and 5 workers you could have each thread continuously scan a subset of mailboxes, i.e., thread1 checks mailbox1 then mailbox2, thread 2 checks mailbox3 then mailbox4, ... This way you could do without the semaphores since there is no contention
Sanju I believe you want to execute them in the following round-robin fashion:
Thread1: 1,6 and so on Thread2: 2,7 Thread3: 3,8 Thread4: 4,9 Thread5: 5,10
First of all this is not what threads are meant for if you want to execute them in a sequence like that. Second I dont think it is possible with thread pool. If you still want them then here is my solution for executing threads in round-robin fashion which you may alter based on your need:
public class EmailRoundRobin {
public Object[] locks;
private static class EmailProcessor implements Runnable {
private final Object currentLock;
private final Object nextLock;
private UserEmail userEmail;
public EmailProcessor (UserEmail userEmail,, Object currentLock, Object nextLock) {
this.userEmail = userEmail;
this.currentLock = currentLock;
this.nextLock = nextLock;
}
@Override
public void run() {
try {
work = //reading email
while ( work != null) {
try {
currentLock.wait();
// Do your work here.
}
catch(InterruptedException e) {}
synchronized(nextLock) {
nextLock.notify();
}
}//while ends
} catch (IOException e) {
e.printStackTrace();
}
synchronized(nextLock) {
nextLock.notify(); /// Ensures all threads exit at the end
}
}
}
public EmailRoundRobin(int numberOfAccountsToRead) {
locks = new Object[numberOfAccountsToRead];
//Initialize lock instances in array.
for(i = 0; i < numberOfAccountsToRead; ++i) locks[i] = new Object();
//Create threads
int j;
for(j=0; j<(numberOfAccountsToRead-1); j++ ){
Thread linePrinterThread = new Thread(new EmailProcessor(emailInfo + "Temp" + j,locks[j],locks[j+1]));
linePrinterThread.start();
}
Thread lastLinePrinterThread = new Thread(new EmailProcessor(emailInfo + "Temp" + j,locks[numberOfFilesToRead-1],locks[0]));
lastLinePrinterThread.start();
}
public void startProcessing() {
synchronized (locks[0]) {
locks[0].notify();
}
}
public static void main(String[] args) {
EmailRoundRobin emailRoundRobin = new EmailRoundRobin(4);
emailRoundRobin.startProcessing();
}
}
I have taken this from my answer to this question! which had similar requirement. There is an option using Phaser also mentioned in that answer as well.
I had got an answer myself, its very simple, i had use ExecutorService and it managed by its only.