2

I hope I'll explain myself correctly. I have a C# web service with a post method get_file(), from my client side, I make a POST request to that method in my server, with transfer-encoding chunked, and send the file in chunks.

The method starts once the request is done (0\r\n\r\n is entered), is there a way to handle every chunk once I get it to the server, instead of waiting to the end?

Maybe the alternative of sending multiple POST request would be better? (but then I get an error after ~100 requests).

Again, I hope it's explained properly. Thanks in advance

Uwe Keim
  • 39,551
  • 56
  • 175
  • 291
DanielY
  • 1,141
  • 30
  • 58
  • If you just want to be able to read the data as it comes in (but not see the separate chunks) then http://tech.edelste.in/post/116646600708/c-read-lines-from-a-transfer-encoding-chunked should help. – Richard Oct 21 '15 at 15:23
  • I would like to be able to process the chunk (do something with it) immedietly – DanielY Oct 22 '15 at 04:43

1 Answers1

1

As I understand you want a C# client to upload a big file in chunks to a server hosting your C# service.

In this case, I have used the following solution for some time now, and it has shown to be very robust:

Server Side function:

    [WebMethod]
    public void UploadFile(string FileName, byte[] buffer, long Offset, out bool UploadOK, out string msg)
    {
        Log(string.Format("Upload File {0}. Offset {1}, Bytes {2}...", FileName, Offset, buffer.Length));
        UploadOK = false;
        try
        {
            // setting the file location to be saved in the server. 
            // reading from the web.config file 
            string FilePath = Path.Combine( ConfigurationManager.AppSettings["upload_path"], FileName);

            if (Offset == 0) // new file, create an empty file
                File.Create(FilePath).Close();
            // open a file stream and write the buffer. 
            // Don't open with FileMode.Append because the transfer may wish to 
            // start a different point
            using (FileStream fs = new FileStream(FilePath, FileMode.Open,
                FileAccess.ReadWrite, FileShare.Read))
            {
                fs.Seek(Offset, SeekOrigin.Begin);
                fs.Write(buffer, 0, buffer.Length);
            }
            UploadOK = true;
            msg = "uploaded to " + FilePath;

            Log(string.Format("Sucessfully Uploaded to File {0}: {1}", FileName, msg));
        }
        catch (Exception ex)
        {
            //sending error:
            msg = "failed to upload: " + ex.Message;
            UploadOK = false;
            Log(string.Format("Failed Upload File {0}: {1}", EmlFileName, ex.Message));
        }
    }

Client Side upload function:

static void SendFile(YourWebService webservice, string filename)
    {
        Console.WriteLine("uploading file: " + filename);

        int Offset = 0; // starting offset.

        //define the chunk size
        int ChunkSize = 65536; // 64 * 1024 kb

        //define the buffer array according to the chunksize.
        byte[] Buffer = new byte[ChunkSize];
        //opening the file for read.
        FileStream fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
        try
        {
            long FileSize = new FileInfo(filename).Length; // File size of file being uploaded.
            // reading the file.
            fs.Position = Offset;
            int BytesRead = 0;
            string msg = "";
            while (Offset != FileSize) // continue uploading the file chunks until offset = file size.
            {
                BytesRead = fs.Read(Buffer, 0, ChunkSize); // read the next chunk 
                // (if it exists) into the buffer. 
                // the while loop will terminate if there is nothing left to read
                // check if this is the last chunk and resize the buffer as needed 
                // to avoid sending a mostly empty buffer 
                // (could be 10Mb of 000000000000s in a large chunk)
                if (BytesRead != Buffer.Length)
                {
                    ChunkSize = BytesRead;
                    byte[] TrimmedBuffer = new byte[BytesRead];
                    Array.Copy(Buffer, TrimmedBuffer, BytesRead);
                    Buffer = TrimmedBuffer; // the trimmed buffer should become the new 'buffer'
                }
                // send this chunk to the server. it is sent as a byte[] parameter, 
                // but the client and server have been configured to encode byte[] using MTOM. 
                bool ChunkAppened = webservice.UploadFile(Path.GetFileName(filename), Buffer, Offset, out msg);
                if (!ChunkAppened)
                {
                    Console.WriteLine("failed to upload. server return error: " + msg);
                    break;
                }
                // Offset is only updated AFTER a successful send of the bytes. 
                Offset += BytesRead; // save the offset position for resume
            }
            Console.WriteLine("successfully uploaded file: " + filename);
        }
        catch (Exception ex)
        {
            Console.WriteLine("failed to upload file: " + ex.Message);
        }
        finally
        {
            fs.Close();
        }
    }
  • That's what I have (kind of) at the moment - a loop of POST requests to a method in the server, the problem in my code is that the server rejects my requests after ~100 requests – DanielY Oct 22 '15 at 04:46