Sending files over TCP/ .NET SSLStream is slow/not working

Question

Im writing an Server/Client Application which works with SSL(over SSLStream), which has to do many things(not only file receiving/sending). Currently, It works so: Theres only one connection. I always send the data from the client/server using SSLStream.WriteLine() and receive it using SSLStream.ReadLine(), because I can send all informations over one connection and I can send from all threads without destroying the data.

Now I wanted to implement the file sending and receiving. Like other things in my client/server apps, every message has a prefix (like cl_files or sth) and a base64 encoded content part(prefix and content are seperated by |). I implemented the file sharing like that: The uploader send to the receiver a message about the total file size and after that the uploader sends the base64 encoded parts of the file over the prefix r.

My problem is that the file sharing is really slow. I got around 20KB/s from localhost to localhost. I have also another problem. If I increase the size of the base64 encoded parts of the file(which makes file sharing faster), the prefix r doesnt go out to the receiver anymore(so the datas couldnt be identified).

How can I make it faster?

Any help will be greatly appreciated.

My(propably bad) code is for the client:

//its running inside a Thread
FileInfo x = new FileInfo(ThreadInfos.Path);
long size = x.Length; //gets total size
long cursize = 0;
FileStream fs = new FileStream(ThreadInfos.Path, FileMode.Open);
Int16 readblocks = default(Int16);
while (cursize < size) {
    byte[] buffer = new byte[4096];
    readblocks = fs.Read(buffer, 0, 4096);
    ServerConnector.send("r", getBase64FromBytes(buffer));//It sends the encoded Data with the prefix r over SSLStream.WriteLine
    cursize = cursize + Convert.ToInt64(readblocks);
    ThreadInfos.wait.setvalue((csize / size) * 100);//outputs value to the gui
}
fs.Close();

For the Server:

case "r"://switch case for prefixes
           if (isreceiving)
           {
              byte[] buffer = getBytesFromBase64(splited[1]);//splited ist the received Line over ReadLine splitted by the seperator "|"
              rsize = rsize + buffer.LongLength;
              writer.Write(buffer, 0, buffer.Length);//it writes the decoded data into the file
              if (rsize == rtotalsize)//checks if file is completed
              {
                 writer.Close();
              }
           }
break;

Can you read larger chunks out than 4096? How large are you files going to be on average? — mj_, Sep 10 '12 at 18:12
I tried it with 8000, some messages were destroyed, tried it also with 2048, it worked but it was very slow. On the average the files will be from 500kb-10mb — Tearsdontfalls, Sep 10 '12 at 18:24
Whenever I had to send messages, I would set the size to be very large (hundred megs) and let the underlying TCP stream handle the flow control for you. Can you add your getBytesFromBase64 function? Is that a custom one you wrote? Try removing it temporarily and send the plain bytes and see if that improves it. — mj_, Sep 10 '12 at 18:29
The getBytesFromBase64 is nothing more than `return System.Convert.FromBase64String(input);` Ill try to remove Base64 tomorrow, but how can I send a byte array over Stream.ReadLine()? — Tearsdontfalls, Sep 10 '12 at 18:59
When you convert something to base 64, how does ReadLine even work? Do the \n characters get base-64 encoded. I'm worried that ReadLine could possibly read too little of your data that comes in if probabilistically the base-64 encoding puts in a \n by chance. As an example if you send "hamburger\n" and it turns into "3df232\n42sdf" (I just made that up), is that a possibility? — mj_, Sep 10 '12 at 20:53
An alternative way of doing it is create a StreamWriter/StreamReader and put the SslStream in the constructor parameter for stream. You should get Write/ReadLine then. (see comment above too) — mj_, Sep 10 '12 at 21:00
Your base64 encoding cannot possibly work correctly if you don't provide it with the read count as well as the buffer. You can't assume the buffer was filled by every read. — user207421, Sep 11 '12 at 00:37
@mj_ For example hamburger\n returned in c# & php aGFtYnVyZ2VyXFxu At Wikipedia, theres a table, which shows which characters could be in a base64 string(the backslash isnt in the list, so that couldnt be the failure). — Tearsdontfalls, Sep 11 '12 at 14:11
@EJP How can I do this? Tried many things but none of these worked.(initializing buffer without array count throws exception, creating new buffer and make newbuffer=oldbuffer; also didnt work) — Tearsdontfalls, Sep 11 '12 at 14:12
How can anybody possiboy answer that when you don't show your base 64 code? — user207421, Sep 14 '12 at 23:44
Actually, can you add the code where you create SSLStream on client/server? — mj_, Sep 19 '12 at 02:09
Got another fact. The file transfering is failing cause Convert.FromBase64 throws a excpection with the Message:"invalid length for a base-64 char array". With 50 Bytes parts, the error doesnt happen. Can it be that there is maximal length for base64 char arrays? — Tearsdontfalls, Sep 19 '12 at 17:08

score 5 · Accepted Answer · answered Sep 19 '12 at 00:28

Your problem stems from the fact that you are performing what is essentially a binary operation through a text protocol and you are exacerbating that problem by doing it over an encrypted channel. I'm not going to re-invent this for you, but here are some options...

Consider converting to an HTTPS client/server model instead of reinventing the wheel. This will give you a well-defined model for PUT/GET operations on files.
If you can not (or will not) convert to HTTPS, consider other client/server libraries that provide a secure transport and well-defined protocol for binary data. For example, I often use protobuf-csharp-port and protobuf-csharp-rpc to provide a secure protocol and transport within our datacenter or local network.
If you are stuck with your transport being a raw SslStream, try using a well-defined and proven binary serialization framework like protobuf-csharp-port or protobuf-net to define your protocol.
Lastly, if you must continue with the framework you have, try some http-like tricks. Write a name/value pair as text that defines the raw-binary content that follows.

As it is trash software, I would like to make it with the existing framework I have. Any idea that i could send bytes directly instead of base64 without interrupting other communication? — Tearsdontfalls, Sep 19 '12 at 16:04

score 2 · Answer 2 · answered Sep 19 '12 at 07:16

First of all base64 over ssl will be slow anyway, ssl itself is slower then raw transport. File transfers are not done over base64 now days, http protocol is much more stable than anything else and most libraries on all platforms are very well stable. Base64 takes more size then actual data, plus the time to encode.

Also, your following line may be a problem.

ThreadInfos.wait.setvalue((csize / size) * 100);//outputs value to the gui

If your this line is blocking, then this will slow down for every 4kb. Updating for every 4kb is also not right, unless a progress value from previous value differs by significant amount, there is no need to update ui for it.

score 1 · Answer 3 · answered Sep 19 '12 at 14:49

I'd give a try of gzip compress before/after the network. From my experience, it helps. I'd say some code like this could help :

using(GZipStream stream = new GZipStream(sslStream, CompressionMode.Compress)) 
{
    stream.Write(...);
    stream.Flush();
    stream.Close();
}

Warning : It may interfer with SSL if the Flush is not done. and it will need some tests... and I didn't try to compile the code.

score 0 · Answer 4 · answered Sep 19 '12 at 14:49

I think Akash Kava is right.

while (cursize < size) {
    DateTime start = DateTime.Now;
    byte[] buffer = new byte[4096];
    readblocks = fs.Read(buffer, 0, 4096);
    ServerConnector.send("r", getBase64FromBytes(buffer));
    DateTime end = DateTime.Now;
    Console.Writline((end-start).TotalSeconds);
    cursize = cursize + Convert.ToInt64(readblocks);
    ThreadInfos.wait.setvalue((csize / size) * 100);
    end = DateTime.Now;
    Console.Writline((end-start).TotalSeconds);
}

By doing this you can find out where is the bottle neck.

Also the way you sending data packets to server is not robust.

Is it possible to paste your implementation of

ThreadInfos.wait.setvalue((csize / size) * 100);

Sending files over TCP/ .NET SSLStream is slow/not working

4 Answers4

Linked