Error when writing to log while threading on web requests

Question

I am building a program in visual studio 2010 using C# .Net4.0

I have a function that takes in a url and try to get a response. Inside the function, I write to a log file stating successes and errors.

public void getResponse(string url, System.IO.StreamWriter writer)
{
    try
    {
        WebRequest request = WebRequest.Create(url);
        WebResponse response = request.GetResponse();

        // Display the status.
        Console.WriteLine(response.Headers);
        writeToLog("Success: " + url, writer);
    }
    catch (WebException ex)
    {
        writeToLog("Error: " + ex.message, writer);
    }
}

I have a list of urls, and I use Parallel.ForEach to run multiple urls at once.

System.IO.StreamWriter writer = new System.IO.StreamWriter("program.log", true);
string[] urls = {url1, url2, url3, url4, etc}; //have around 30 urls
Parallel.ForEach(urls, url => getResponse(url, writer));

When I run the program, sometime I will get an error saying that program.log is being used by another program. This doesn't happen all the time, but once a while it will happen and the program will crash.

Question: What should I do to avoid this error? Or, is there an alternative way of doing it?

StreamWriter is not thread safe. See http://msdn.microsoft.com/en-us/library/system.io.streamwriter.aspx — user2509738, Jul 09 '13 at 20:36

score 2 · Accepted Answer · answered Jul 09 '13 at 20:40

2

Because StreamWriter isn't thread-safe use a lock statement to lock the writing procedure so the other threads can't enter that line and cause a conflict: http://msdn.microsoft.com/en-us/library/c5kehkcz(v=vs.100).aspx

answered Jul 09 '13 at 20:40

Pharap

3,826
5
37
51

so if I do it this way it will work? `Object mylock = new Object(); lock(mylock) { writeToLog("Error: " + ex.message, writer); }` – sora0419 Jul 09 '13 at 20:45
2

You should only create one lock object, not a new one each time you do a lock. You want using code to be trying to lock on the same thing. – hatchet - done with SOverflow Jul 09 '13 at 20:52
Like hatchet says, you should create a private field in the class that contains get response. If you do that, it should work. Give it a try and if the problem still happens I'll try and think of something else. – Pharap Jul 09 '13 at 20:54
Okay! I'll try that! Like I said, this problem doesn't happen on every run, so it's probably gonna take me a while before I'm confident to say that fix the problem. Thanks for the help :) – sora0419 Jul 09 '13 at 21:02
Glad to see this worked. Never had to use the lock statement myself, but it never hurts to know these things. – Pharap Jul 11 '13 at 02:02
Yeah, I haven't see this error again for a while so I think this actually fix it. Thanks for the help! – sora0419 Jul 11 '13 at 16:44

Error when writing to log while threading on web requests

1 Answers1