0

My application produces a compressed (gzip) xml messages to transfer over https to a webserver. It receives up to 10,000 data packets/sec from client apps through TCP/IP. Size of each packet could be from 500 to 2000 bytes. It should add a received packet to a corresponding xml message in the collection based on the packet metadata. When the length of the particular message reaches the specified limit (1024Kb, for example) it should be saved to file in the specified folder and then sent from from there to a web server.

What is the correct/efficient approach to do this?

I am considering two ways:

1) GZipStream > FileStream Write each packet on-the-fly directly to the corresponding file and watch on its size. But I am not sure that so frequent writes is good and i would like to keep the file system workload as low as possible.

2) GZipStream > MemoryStream > File Use a memory as a buffer for messages. In this case i should create a MemoryStream for each message and keep it open while adding packets. But in this case i don't know how to get memory stream's current length without closing it.

Example: I can get the ms.Length only after ms.close() call.

var ms = new MemoryStream();
var gstream = new GZipStream(ms, CompressionMode.Compress);
var writer = XmlWriter.Create(gstream);

writer.WriteStartElement("x", "root", "123");
writer.WriteElementString("packet_ID", "some data");
writer.WriteEndElement();                                   

writer.Close();
gstream.Close();
ms.Close();

Console.WriteLine("Memory size: " + ms.ToArray().Length);            
Console.ReadLine();
user3447070
  • 93
  • 7
  • I have edited your title. Please see, "[Should questions include “tags” in their titles?](http://meta.stackoverflow.com/questions/19190/)", where the consensus is "no, they should not". – John Saunders May 28 '14 at 19:43
  • 1
    Your `MemoryStream`, `GZipStream` and `XmlWriter` all need to be in `using` blocks to ensure they are properly disposed, especially in the case of exceptions. – John Saunders May 28 '14 at 19:44
  • @JohnSaunders thanks for comments. I read it. :) – user3447070 May 28 '14 at 20:07
  • 1
    What do you mean you can only get the length of the `MemoryStream` after closing it? `ms.Length` always returns the number of bytes contained within it. Also, with TCP/IP you don't talk about packets, it's a stream protocol. – C.Evenhuis May 28 '14 at 21:06
  • 1
    @C.Evenhuis You are correct about TCP. In my case "packets" are serialized objects sent over TCP with specified marker at the end of data. – user3447070 May 29 '14 at 08:34
  • And everything is ok with memstream now, i found that i need to close only the writer before i can get the number of bytes in it. – user3447070 May 29 '14 at 08:43

0 Answers0