3

I need to send compressed Base64 data over an Azure queue, which has a limitation of 64K.
My code compresses the data and then encodes it as a Base64 string.
I verify the compressed and encoded string does not exceed 64000 bytes (see encodedLen below), however, my code crashed when I tried to add a message of ~57,000 bytes . 

var byteString = Encoding.UTF8.GetBytes(articleDataToSend);  
var compressed = QuickLZ.compress(byteString, 1);  
var encoded = Convert.ToBase64String(compressed);  

var encodedLen = Encoding.UTF8.GetByteCount(encoded);  
if(encodedLen < 64000)
{
    QueueMessage(_nlpInputQueue, encoded);
}

I'm using Visual Studio 2012 and .Net 4.5.
What am I missing here?

Gilad Gat
  • 1,468
  • 2
  • 14
  • 19

2 Answers2

4

Maximum message size is 48k when using Base64 encoding http://msdn.microsoft.com/en-us/library/windowsazure/hh767287.aspx

Use Azure Service Bus Queues and you can get up to 256k

Igorek
  • 15,716
  • 3
  • 54
  • 92
1

I believe you're ending up with a double-encoded string. The storage client library has historically base64-encoded everything put on a queue, so it's base64-encoding your base64-encoded string. My guess is that if you did QueueMessage(_nlpInputQueue, compressed) instead, this would work.

user94559
  • 59,196
  • 6
  • 103
  • 103