0

I have .NET 4 code (not ASP) running both on win 7 and server 2008 that creates multiple threads (each with a webclient, as I need separate cookiecontainers) for requests and responses to websites.

I use multiple proxies and for each proxy I will have a number of different user-agents strings, so each thread will be opening a connection based on proxy + user-agent. i.e.: 10 Proxy’s x 8 UA strings = 80 connections on 80 threads. [I do believe Async would be a better choice, although I do need separate web-clients]

In order not to overwhelm website with requests I want to rely on .NET internal pipe-lining to prevent more than one parallel request at once to same endpoint. i.e. I added this in my config file: 

<connectionManagement>
<add address="*" maxconnection="1" />
</connectionManagement>

Yet my logs show that it is being ignored and all 8 connections to same endpoint are being created more or less simultaneously. Is this happening because each connection is being made on a separate thread? And if yes, any other setting for it to work per process besides having my code limit multiple simultaneous connections to same endpoint?

EDIT

As per request, idea of Code that launches each request:

Public Class cWebClient
    Inherits WebClient

    Public Property URL As String

    ......
    Public Sub GetUrl()
        .......
        Dim html As String = Me.DownloadString(u)
        .......
End Class

Code that creates each different class

Sub LaunchThreads()
Dim wc as new cWebClient
.....
Dim ts As New Threading.Thread(AddressOf wc.GetURL)
Mint
  • 84
  • 6
  • 1
    Just a note: Your XML is not valid – leppie Jul 13 '15 at 14:41
  • 1
    I don't know the answer but using `SemaphoreSlim` is a quick fix that gives you more control. No need for global settings. Just turn them off. – usr Jul 13 '15 at 15:28
  • Please provide more information, show relevant code and tell us please what technology do you use to perform server-client operations. Is it WCF? – Fabjan Jul 14 '15 at 11:50

0 Answers0