4

I am faced with writing a WCF server running on an IIS to be consumed by other platforms on different computers. I need to return a large number of objects (serialized) to the clients. I need it to be streaming since getting all of the data from my DB and arranging it according to the cilent's requirements is a very long process, so I need the clients to be able to beging processing the results as they stream out.

After doing some reading on the subject it seems that if I use WCF with transferMode StreamedResponse I can return the objects with deferred execution and this may answer my needs.

I read the following article: http://weblogs.asp.net/cibrax/archive/2008/06/10/streaming-large-content-with-wcf-and-deferred-execution.aspx

and implemented a similar WCF to the one they describe. However I changed their code from:

for(long i = 0; i < 1000; i++) //All the customers should be read from the database
{
   yield return new Customer { FirstName = "Foo", LastName = "Bar", Address = "FooBar 123" };
}

to:

for(long i = 0; i < 1000000; i++) //All the customers should be read from the database
{
   yield return new Customer { FirstName = "Foo", LastName = "Bar", Address = "FooBar 123" };
}

So it would simulate a large amount of data. I noticed that when I do this the function on the client side stalls for a number of seconds before returning the objects.
Why does this happen? Shouldn't the data start flowing on the stream immediately? Isn't that the point of the streaming WCF with deferred execution? Do you have a better suggestion for my problem?

Community
  • 1
  • 1
Zohar Levin
  • 2,364
  • 3
  • 19
  • 24
  • After reading the article, my best guess would be that you are actually streaming the data right away, but the client still needs to have received all of it, before it can begin parsing/processing. The main purpose of this seems to be to keep the server from having to store the complete response in RAM, before sending it out all at once. Step 4 says (roughly) "Client reads one customer at a time from response message", but it looks like the response message was already fully received at that point, the client is just deserializing it one customer at a time. – Wutz Nov 21 '12 at 14:25
  • 2
    What is your client code? Are you waiting for all the objects to come accross before you do anything? If you are in a debugger, and considering the code you have, you might be forcing enumeration (getting all the data) while you are trying to step through something. Certain security settings might slow down a WCF connection as well... – iMortalitySX Nov 21 '12 at 14:54
  • so you think the client is waiting for all the customers to arrive? is there a way to prevent this? on my client side i used the function static IEnumerable GetAllCustomers(Message message) and called it from my Main to iterate over the customers – Zohar Levin Nov 22 '12 at 08:06

1 Answers1

1

I had a look at the code in the link.

It looks like the CreateMessage is run on the server, placing the entire result in the variable message, before the server starts returning any data.

   Message message = Message.CreateMessage(MessageVersion.Soap11, "GetAllCustomers", new CustomBodyWriter(GetAllCustomersImpl()));
   return message;
Shiraz Bhaiji
  • 64,065
  • 34
  • 143
  • 252
  • the interface is: [ServiceContract] public interface IService1 { [OperationContract] Message GetAllCustomers(); }. does that mean that the client waits for all the data? if so, what's the point of streaming? – Zohar Levin Nov 22 '12 at 08:02
  • 2
    It is not that the client waits, it is that the server does not start sending before it has generated the whole list. The wait is before the streaming starts. – Shiraz Bhaiji Nov 22 '12 at 09:06
  • I tested this again with a breakpoint on the 'return message;' line and a breakpoint on the client side and i noticed that the time is not in building the message on the server side, it's in the actual returning of the message. meaning it's in the wcf implementation of sending the data. – Zohar Levin Nov 22 '12 at 12:31