Webrequest faster downloading (Googled for days)

Question

Im working on a project that downloads up to 5000 individual pieces of data from a server. It basically is a PHP page that takes POST variable, gets the data from the DB and sends it back to the .NET client.

It is slow. It takes about 1 second per request. I've googled a lot and tried all sorts of tweaks to the code, like the famous proxy-setting etc. But nothing speeds it up.

Any idea's? All solutions that make this super fast are welcome. Even C-written DLL's or anything you can think of. This just needs to be a lot faster.

Public Function askServer(oCode As String) As String

    oBytesToSend = Encoding.ASCII.GetBytes("cmd=" & System.Web.HttpUtility.UrlEncode(oCode))

    Try
        oRequest = WebRequest.Create(webServiceUrl)
        oRequest.Timeout = 60000
        oRequest.Proxy = WebRequest.DefaultWebProxy
        CType(oRequest, HttpWebRequest).UserAgent = "XXXXX"
        oRequest.Method = "POST"
        oRequest.ContentLength = oBytesToSend.Length
        oRequest.ContentType = "application/x-www-form-urlencoded"

        oStream = oRequest.GetRequestStream()
        oStream.Write(oBytesToSend, 0, oBytesToSend.Length)

        oResponse = oRequest.GetResponse()
        If CType(oResponse, HttpWebResponse).StatusCode = Net.HttpStatusCode.OK Then
            oStream = oResponse.GetResponseStream()
            oReader = New StreamReader(oStream)
            oResponseFromServer = oReader.ReadToEnd()
            oResponseFromServer = System.Web.HttpUtility.UrlDecode(oResponseFromServer)
            Return oResponseFromServer
        Else
            MsgBox("Server error", CType(vbOKOnly + vbCritical, MsgBoxStyle), "")
            Return ""
        End If

    Catch e As Exception

        MsgBox("Oops" & vbCrLf & e.Message, CType(vbOKOnly + vbCritical, MsgBoxStyle), "")
        Return ""

    End Try

End Function

Are you using PDO with `localhost` as the host? If so change it to `127.0.0.1` — Lawrence Cherone, Jul 24 '14 at 08:17
Hi Loz. Its a MySQL_Connect still. The host is a name since the hoster put all databases in a cluster. — JaredNinja, Jul 24 '14 at 08:24

score 0 · Answer 1 · answered Jul 24 '14 at 08:16

0

Some ideas :

Run the http requests in parallel. (Client)
If the data response size allows it get all data needed in one request (you need change your server implementation).
Caching data. (Server)

answered Jul 24 '14 at 08:16

José Ricardo Pla

1,043
10
16

First one sounds interesting. Separate threads? How many could you do simultaneously? All data at once was tried but not the right solution. Its only small data, but sometimes over phone-internet connections (3G). With all at once, they could crash at 500/5000 and have nothing. Ill google Caching. Maybe thats an option too. – JaredNinja Jul 24 '14 at 08:27

Webrequest faster downloading (Googled for days)

1 Answers1