3

We're building a Delphi REST server that serves up rather large chunks of data (1.5MB per request, of which there are many) to a native Android application. All works fine, except the data sizes in this case will be problematic, causing long transfer times in our environment (limited mobile data rates). I've tried adding the ZLibCompression filter on the DSHTTPWebDispatcher, but the response only comes back again as uncompressed text/html.

Is there any way to force the server to use the filter added as an event before the dispatch?

The server is built using Delphi XE3.

mjn
  • 36,362
  • 28
  • 176
  • 378
bosvos
  • 549
  • 5
  • 14
  • 1
    what is the accept-encoding header value for your requests? – jachguate Jan 30 '13 at 02:58
  • how do you _send_ the data (method firm)? – jachguate Jan 30 '13 at 02:59
  • can you also show the client side Android code? – mjn Jan 30 '13 at 08:31
  • Thanks for the questions...it does not really matter who the client is, it should be consumable by applications in any OS. The issue here was to get the actual compression to work in the first place, and also to set the content-type properly. I have since figured it out and will post the result here shortly. – bosvos Feb 07 '13 at 11:22

1 Answers1

4

I've managed to figure out where to add the compression and relevant header changes in the DataSnap project.

The key here is the TWebModule class. If one uses the wizard to create a new project, a default implementation of the TWebModule class is constructed, with event properties for BeforeDispatch, AfterDispatch etc. The naming here refers to the action of dispatching the incoming request to where it will be handled. So, BeforeDispatch happens when the request arrives, some processing happens on the server and AfterDispatch triggers just before the response is sent back to the caller.

AfterDispatch is therefore the correct event to use if one wants to modify the constructed response after the fact. This can include changes to both the content and headers.

On the AfterDispatch event:

procedure TWebModule1.WebModuleAfterDispatch(
  Sender: TObject;
  Request: TWebRequest; 
  Response: TWebResponse;
  var Handled: Boolean);
var

srcbuf, destbuf : TBytes;
str : string;

begin
  str := Response.Content;

  //prepare byte array
  srcbuf := BytesOf(str);

  //compress to buff (System.ZLib)
  ZCompress(srcbuf, destbuf, zcMax);

  //prepare responsestream and set content encoding and type
  Response.Content := '';
  Response.ContentStream := TMemoryStream.Create;
  Response.ContentEncoding := 'deflate';
  Response.ContentType := 'application/json';

  //current browser implementations incorrectly handles the first 2 bytes 
  //of a ZLib compressed stream, remove them
  Response.ContentStream.Write(@(destbuf[2]),length(destbuf)-2);
  Response.ContentLength := (length(destbuf))-2;
end;

Not very fancy, one could enable/disable compression depending on the content sent back, but for our implementation we kept it simple.

This works 100% with Fiddler and browsers that can handle deflate.

bosvos
  • 549
  • 5
  • 14