0

I have a problem I'm trying to decide how to approach and solve it, I have an application where the user selects certain files and the files is need to be added to my ListView but before I need to check this files (via another class), my problem is to know exactly this operation finishes in order to update my GUI

Please see this class:

public class ProducerConsumer : IDisposable
{
    public delegate void OnFileAddDelegate(string file);
    public event OnFileAddDelegate OnFileAddEventHandler;
    BlockingCollection<string> _taskQ = new BlockingCollection<string>();
    public ProducerConsumer(int workerCount)
    {
        // Create and start a separate Task for each consumer:
        for (int i = 0; i < workerCount; i++)
            Task.Factory.StartNew(Consumer);
    }

    public void Dispose()
    {
        _taskQ.CompleteAdding();
    }

    public void Enqueue(string action)
    {
        _taskQ.Add(action);
    }

    private void Consumer()
    {
        // This sequence that we’re enumerating will block when no elements
        // are available and will end when CompleteAdding is called. 
        FileChecker fileChecker = new FileChecker();
        foreach (string item in _taskQ.GetConsumingEnumerable())
        {
            string file = item;
            string result = fileChecker.Check(file);
            if (result != null && OnFileAddEventHandler != null)
                OnFileAddEventHandler(result);
        }
    }
}

I am using this class in my Winforms application, after the user choose files that need to be add into my ListView this class put this files (PDF files) into Queue and the consumer take this files and with another class (FileChecker) check this each file (simple search) and if the file is OK raise up an Event to the main form in order to add the file. now while search/add this files i am lock all my controllers and at the end of this operation i want to unlock this controllers but i cannot know the specific moment that the class has finish it's job, i try to put break-point at the end of the Consumer foreach loop but it seems that it didn't get there so i wonder how could i know when this operation has finish

UPDATE:

This is form the main form:

string[] files // All the files that the user choose

ProducerConsumer producerConsumer = new ProducerConsumer(5);
producerConsumer.OnFileAddEventHandler += pq_OnFileAddEventHandler;
foreach (string item in files)
{
    string filename = item;
    producerConsumer.Enqueue(filename);
}
may danit
  • 25
  • 6
  • Doesn't Consumer only run when your _taskQ will be empty? I can't figure how this is supposed to work... – Schwarzie2478 Aug 30 '14 at 07:12
  • 1
    It doesn't work, the most likely outcome is that all consumer threads quit before you had a chance to add any items to the queue. Leaving a queue that never gets emptied. An accidental threading race *might* keep some alive to get the job done. It is just the wrong abstraction for what you want to do. A starting point is to rewrite the constructor to `ProducerConsumer(int workers, List jobs)`. Simply counting the jobs with InterlockedDecrement lets you fire an event. – Hans Passant Aug 30 '14 at 07:24
  • Can i have an example how to do that ? (i am a new developer..) – may danit Aug 30 '14 at 08:51
  • BTW i am add items into the Queue before the Enqueue - please see my update – may danit Aug 30 '14 at 09:07
  • in your case completion of `OnFileAddEventHandler` for last item in `GetConsumingEnumerable` loop after `CompleteAdding` called is perhaps the end of the producer consumer job. but it is even harder to determine if there are async tasks in the consumer loop / handler. typically joining/waiting the Consumer task with the main thread will ensure processing of all the jobs. – pushpraj Aug 30 '14 at 09:38

1 Answers1

2

If you really want to spawn them and kill them when they are done, than you should not use blocking queue, but normal queue with synchronization:

  1. Fill the queue
  2. Spawn consumers
  3. Lock, try to pop, unlock in consumers
  4. Exit consumer if there was nothing to pop
  5. Use the original asnwer to signal that last consumer finished it's job

Full Example

public class ProducerConsumer {
    public delegate void FileAddedDelegate(string file);
    public delegate void AllFilesProcessedDelegate();
    public event FileAddedDelegate FileAdded;
    public event AllFilesProcessedDelegate AllFilesProcessed
    readonly Queue<string> queue; int counter;
    public ProducerConsumer(int workerCount, IEnumerable<string> list) {
        queue = new Queue<string>(list); // fill the queue
        counter = queue.Count; // set up counter
    }
    public void Start() {// split this to be able to add event hooks in between
        for (int i = 0; i < workerCount; i++)
            Task.Factory.StartNew(Consumer);
    }
    private void Consumer() {
        FileChecker fileChecker = new FileChecker();
        for(;;) {
            string file;
            lock(queue) { // synchronize on the queue
                if (queue.Count == 0) return;  // we are done
                file = queue.Dequeue(); // get file name to process
            } // release the lock to allow other consumers to access the queue
        //  do the job
            string result = fileChecker.Check(file);
            if (result != null && FileAdded != null)
                FileAdded(result);
        //  decrement the counter
            if(Interlocked.Decremet(counter) != 0)
                continue; // not the last
        //  all done - we were the last
            if(AllFilesProcessed != null)
                AllFilesProcessed()
            return;
        }
    }
}

Original

NOTE: Your example looks like idea, not a complete code. This solution was designed for freely running consumers - that are there sleeping and waiting for the job and reporting when they reached the empty queue.

Use some counter:

int counter;

Increment it before queing the files

Interlocked.Add(ref counter, files.length);
foreach (string item in files) {
    string filename = item;
    producerConsumer.Enqueue(filename); }

Decrement and test in consumers:

if (result != null) {
    if(OnFileAddEventHandler != null)
        OnFileAddEventHandler(result);
    if(Interlocked.Decrement(ref counter) == 0
    && OnAllFilesProcessed != null)
        OnAllFilesProcessed();

Link: System.Threading.Interlocked

firda
  • 3,268
  • 17
  • 30
  • Can you explain me please what for the Interlocked.Decrement ? (BTW this need to send only 1 params) – may danit Aug 30 '14 at 12:53
  • Updated. Be so kind and edit your question as well, that it contains short description of the problem in first paragraph (before any code). Take your time to word it properly. Have a nice day :) – firda Aug 30 '14 at 13:27
  • This code works perfect, thanks a lot ! BTW what would happen if i have several list of files and for each list i created new ProducerConsumer instance ? this will work simultaneously ? – may danit Aug 30 '14 at 13:54
  • Interface or instance? Instances will spawn their own consumers, all parallel. – firda Aug 30 '14 at 13:59
  • ProducerConsumer object – may danit Aug 30 '14 at 14:04
  • Every instance/object (`new ProducerConsumer`) will spawn its own consumers to finish the job. They are independent, work simultaneously (almost, they are queued for execution on ThreadPool). – firda Aug 30 '14 at 14:12