How to avoid collection modification during JSON serialization in looped multithreaded task?

Question

I have a problem during serialization to JSON file, when using Newtonsoft.Json.

In a loop I am fiering tasks in various threads:

List<Task> jockeysTasks = new List<Task>();

            for (int i = 1; i < 1100; i++)
            {
                int j = i;
                Task task = Task.Run(async () =>
                {
                    LoadedJockey jockey = new LoadedJockey();
                    jockey = await Task.Run(() => _scrapServices.ScrapSingleJockeyPL(j));

                    if (jockey.Name != null)
                    {
                        _allJockeys.Add(jockey);
                    }

                    UpdateStatusBar = j * 100 / 1100;

                    if (j % 100 == 0)
                    {
                        await Task.Run(() => _dataServices.SaveAllJockeys(_allJockeys)); //saves everything to JSON file
                    }
                });

                jockeysTasks.Add(task);
            }

            await Task.WhenAll(jockeysTasks);

And if (j % 100 == 0), it is rying to save the collection _allJockeys to the file (I will make some counter to make it more reliable, but that is not the point):

public void SaveAllJockeys(List<LoadedJockey> allJockeys)
        {
            if (allJockeys.Count != 0)
            {
                if (File.Exists(_jockeysFileName)) File.Delete(_jockeysFileName);

                try
                {
                    using (StreamWriter file = File.CreateText(_jockeysFileName))
                    {
                        JsonSerializer serializer = new JsonSerializer();
                        serializer.Serialize(file, allJockeys);
                    }
                }
                catch (Exception e)
                {
                    dialog.ShowDialog("Could not save the results, " + e.ToString(), "Error");
                }
            }
        }

During that time, as I belive, another tasks are adding new collection item to the collection, and it is throwing to me the exception:

Collection was modified; enumeration operation may not execute.

As I was reading in THE ARTICLE, you can change type of iteration to avoid an exception. As far as I know, I can not modify the way, how Newtonsoft.Json pack is doing it.

Thank you in advance for any tips how to avoid the exception and save the collection wihout unexpected changes.

score 2 · Accepted Answer · answered Jul 06 '19 at 14:31

2

I tied to use ToList() on the collection, what creates copy of the list, with positive effect.

answered Jul 06 '19 at 14:31

bakunet

197
1
12

score 1 · Answer 2 · answered May 09 '19 at 10:32

You should probably inherit from List and use a ReaderWriterLock (https://learn.microsoft.com/en-us/dotnet/api/system.threading.readerwriterlock?view=netframework-4.8)

i.e. (not tested pseudo C#)

public class MyJockeys: List<LoadedJockey>
{

    System.Threading.ReaderWriterLock _rw_lock = new System.Threading.ReaderWriterLock();

    public new Add(LoadedJockey j)
    {
       try
       {
           _rw_lock.AcquireWriterLock(5000); // or whatever you deem an acceptable timeout
          base.Add(j);
       }
       finally
       {
           _rw_lock.ReleaseWriterLock();
       }
    }

    public ToJSON()
    {
       try
       {
           _rw_lock.AcquireReaderLock(5000); // or whatever you deem an acceptable timeout
          string s = "";  // Serialize here using Newtonsoft
          return s;
       }
       finally
       {
           _rw_lock.ReleaseReaderLock();
       }
    }
    // And override Remove and anything else you need
}

Get the idea?

Hope this helps.

Regards,

Adam.

Thank you for your help, but I found a bit simpler solution. I tyed to use `ToList()` on the collection, what creates copy of the list, with positive effect. — bakunet, May 09 '19 at 11:31

How to avoid collection modification during JSON serialization in looped multithreaded task?

2 Answers2