ObservableCollection is not updating inside of a Parallel.Foreach

Question

I have an ObservableCollection called AllVideos. I am using INotifyPropertyChanged on the video model as well as the AllVideos collection. For each file in a File Picker, I am running a method called ProcessMetaData() in a Parallel.ForEach. ProcessMetaData() adds new video file models to the AllVideos collection. The problem I am having is that the AllVideos collection is not being updated by the method. It works fine if the method is not executed in the Parallel.Foreach.

Below is the method that calls the Parallel.Foreach:

public void GetVideoFiles()
{                   
    OpenFileDialog openFileDialog = new OpenFileDialog();
    openFileDialog.Multiselect = true;
        
    if(openFileDialog.ShowDialog() == true)
    {              
        List<string> files = new List<string>();
        foreach(string file in openFileDialog.FileNames)
        {
            files.Add(file);                        
        }
        Parallel.ForEach(files, f => ProcessMetaData(f));
        bool extractTrigger = true;
        while (extractTrigger == true)
        {
            if (AllVideos.Count == files.Count)
            {
                Debug.WriteLine("VALUES ARE EQUAL !!!!!!!!!!");
                Parallel.ForEach(AllVideos, v => ExtractFrames(v.VideoPath, v.FrameDestinationPath));
                extractTrigger = false;
            }
        }                                
    }            
}

Below is the method being called by the Parallel.Foreach:

private void ProcessMetaData(string fileName)
{                   
    var metaData = Helpers.MetaHelper.getExifDataViaTool(fileName);            
    Guid guid = Guid.NewGuid();
    var destinationDirectory = System.IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData), @"FCS\Temp\" + guid.ToString() + @"\");
    if (!System.IO.Directory.Exists(destinationDirectory))
    {
            System.IO.Directory.CreateDirectory(destinationDirectory);
            System.IO.Directory.CreateDirectory(destinationDirectory + @"\Thumb\");
    }
    else
    {
        MessageBox.Show("Directy already exist");
    }
        Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.ApplicationIdle, new Action(() =>
    {
        AllVideos.Add(new Models.VideoFileModel()
        {
            VideoName = metaData["File Name"],
            VideoPath = metaData["Directory"].Replace("/", @"\") + @"\" + metaData["File Name"],
            VideoDuration = Helpers.MetaHelper.ParseExifDuration(metaData["Duration"]),
            VideoCreated = Helpers.MetaHelper.ParseExifDateModified(metaData["Date/Time Original"]),
            VideoThumbPath = destinationDirectory + "Thumb\\thumb.jpg",
            FrameDestinationPath = destinationDirectory
            //VideoFrameRate = Helpers.MetaHelper.GetFrameRate(fileName)
        });   
    }));    
    NotifyPropertyChanged(nameof(AllVideos));    
    ProcessThumb(fileName, destinationDirectory);   
}

Any ideas as to why AllVideos is not being updated? My code is stuck in the while loop because AllVideos.Count is always equal to 0 and not file.Count.

I would appreciate any advice! Been stuck on this one for about 8 hours.

You're hanging the UI thread with a closed while loop, and the dispatcher will never be idle, hence your code AllVideos.Add() will never run, and boom you're deadlocked. — Federico Berasategui, May 07 '22 at 20:09
I highly suggest you turn this into an async Task method and use await Task.Delay(1000) or something like that inside the while loop. Leave the dispatcher breathe so it can be idle to run your other code. — Federico Berasategui, May 07 '22 at 20:10
Thank you so much for your advice Federico. At the risk of sounding like an idiot, I have to ask how I would go about doing that. I have very little experience with async. I am self taught and some things I guess I never really grasped. — Jacob, May 07 '22 at 20:16
change `public void` to `public async Task` and add `await Task.Delay(1000);` inside the while loop — Federico Berasategui, May 07 '22 at 20:28
What's the intention behind using `Parallel.ForEach` instead of the `foreach`? Do you want to keep the UI responsive, or you want to make the whole procedure faster? — Theodor Zoulias, May 07 '22 at 20:32
Each loop is doing some heavy lifting. I was hoping to have all running at the same time to make the video processing a lot faster. — Jacob, May 07 '22 at 20:48
Before getting into the trouble of fixing the `ObservableCollection`-related problem, I would suggest to comment-out all UI related code and just run the parallel loop and measure the duration with a `Stopwatch`, to make sure that it's actually faster than the sequential loop. I wouldn't be surprised if the performance of the parallel version was the same or even worse. — Theodor Zoulias, May 07 '22 at 20:55
in parallel it is 5.45 seconds. In normal foreach it is 9.66 seconds. It doesnt sound like a big deal because while testing I am using some really small video files. This application , when finished, will have files that are rather large. — Jacob, May 07 '22 at 21:05

score 2 · Answer 1 · answered May 09 '22 at 14:36

I am not proud of what I am about to say; however, with all the work and advice that you have all given, I feel that I owe an explanation. BTW, thank you Theodor for your information about the Parallell.ForEach. I did not know much about it and now have a much better understanding.

The ProgressBar that I have for each video in AllVideos was not moving so I just assumed that the collection was not being updated. Actually, the problem was that the ProgressBar was in a StackPanel and the width was not defined. The StackPanel reduced the width of the ProgressBar to One pixel, making it appear to not update.

Three days of trying to solve this problem over something this stupid. I am just curious. I have never worked in the Software Development industry as I am self taught and just code projects mainly for learning purposes. Are stupid mistakes like this normal in the actual industry?

I would like to change careers at some point; however, I do not have an education in CS and don't think I would even be considered.

Yes, they are totally normal. As you mature the frequency would decrease. So, don't feel bad about this. If you come out from these situations by learning something then that's the best. :) — Peter Csala, May 11 '22 at 06:22

Theodor Zoulias · Answer 2 · 2022-05-08T10:16:07.023

The problem is that the Parallel.ForEach method is a synchronous method that blocks the current thread¹ during the parallel operation. And when the UI thread gets blocked, the UI becomes non-responsive. You could try offloading the parallel loop to the ThreadPool, by using the Task.Run method:

var options = new ParallelOptions() { MaxDegreeOfParallelism = 2 };
await Task.Run(() => Parallel.ForEach(files, options, f => ProcessMetaData(f)));

This call should be in an async method with Task return value, or in an async void event handler. You might have to disable some buttons/menus when the operation starts and until it ends, to prevent the user from interacting with the UI while the operation is running.

Btw I recommend that the MaxDegreeOfParallelism is explicitly specified when calling the Parallel.ForEach method, because the default is -1, which means unlimited parallelism. Which practically means that the limit is the ThreadPool availability, in other words the ThreadPool becomes saturated.

¹ _{To be more precise the current thread participates in the parallel processing as a worker thread.}

ObservableCollection is not updating inside of a Parallel.Foreach

2 Answers2