Getting poor performance while saving to Redis cache (using ServiceStack.Redis)

Question

I am getting very poor performance while saving data to Redis cache.

Scenario :

1) Utilizing Redis cache service (provided by Microsoft Azure).
2) Running code in Virtual Machine created on Azure.
3) Both VM and Cache service are created on same Location

Code Snippet:

    public void MyCustomFunction()
    {
        Stopwatch totalTime = Stopwatch.StartNew();

        RedisEndpoint config = new RedisEndpoint();
        config.Ssl = true;
        config.Host = "redis.redis.cache.windows.net";
        config.Password = Form1.Password;
        config.Port = 6380;
        RedisClient client = new RedisClient(config);

        int j = 0;

        for (int i = 0; i < 500; i++)
        {
            var currentStopWatchTime = Stopwatch.StartNew();
            var msgClient = client.As<Message>();

            List<string> dataToUpload = ClientData.GetRandomData();
            string myCachedItem_1 = dataToUpload[1].ToString();

            Random ran = new Random();
            string newKey = string.Empty;
            newKey = Guid.NewGuid().ToString();

            Message newItem = new Message
            {
                Id = msgClient.GetNextSequence(), // Size : Long variable
                //Id = (long)ran.Next(),
                Key = j.ToString(),             // Size: Int32 variable
                Value = newKey,                 // Size : Guid string variable
                Description = myCachedItem_1    // Size : 5 KB
            };

            string listName = ran.Next(1, 6).ToString();
            msgClient.Lists[listName].Add(newItem);
            //msgClient.Store(newItem);

            Console.WriteLine("Loop Count : " + j++ + " , Total no. of items in List : " + listName + " are : " + msgClient.Lists[listName].Count);

            Console.WriteLine("Current Time: " + currentStopWatchTime.ElapsedMilliseconds + " Total time:" + totalTime.ElapsedMilliseconds);

            Console.WriteLine("Cache saved");
        }
    }

Performance (While Saving):

Note : (All times are in milliseconds)

Loop Count : 0 , Total no. of items in List : 2 are : 1 Current Time: 310 Total time:342 Cache saved Loop Count : 1 , Total no. of items in List : 3 are : 1 Current Time: 6 Total time:349 Cache saved Loop Count : 2 , Total no. of items in List : 5 are : 1 Current Time: 3 Total time:353 Cache saved Loop Count : 3 , Total no. of items in List : 5 are : 2 Current Time: 3 Total time:356 Cache saved Loop Count : 4 , Total no. of items in List : 5 are : 3 Current Time: 3 Total time:360 Cache saved

. . . . .

Loop Count : 330 , Total no. of items in List : 4 are : 69 Current Time: 2 Total time:7057 Cache saved
Loop Count : 331 , Total no. of items in List : 4 are : 70 Current Time: 3 Total time:7061 Cache saved
Loop Count : 332 , Total no. of items in List : 4 are : 71 Current Time: 2 Total time:7064 Cache saved

Performance (While Fetching)

List : 1 No. of items : 110 Time : 57

List : 2 No. of items : 90 Time : 45

List : 3 No. of items : 51 Time : 23

List : 4 No. of items : 75 Time : 32

List : 5 No. of items : 63 Time : 33

Answer 1

If you're dealing in batches you should look at reducing the number of synchronous network requests that you're making to reduce your latency which is going to be the major performance issue when communicating with network services.

For this example you're making a read when you call:

msgClient.GetNextSequence();

and a write when you make:

msgClient.Lists[listName].Add(newItem);

Which is a total of 1000 synchronous request/reply network requests in a single thread where each operation is dependent and has to complete before the next one can be sent which is why network latency is going to be a major source of performance issues which you should look at optimizing.

Batching Requests

If you're dealing with batched requests this can be optimized greatly by reducing the number of reads and writes by fetching all ids in a single request and storing them using the AddRange() batch operation, e.g:

var redisMessages = Redis.As<Message>();
const int batchSize = 500;

//fetch next 500 sequence of ids in a single request
var nextIds = redisMessages.GetNextSequence(batchSize); 

var msgBatch = batchSize.Times(i => 
    new Message {
        Id = nextIds - (batchSize - i) + 1,
        Key = i.ToString(), 
        Value = Guid.NewGuid().ToString(),
        Description = "Description"
    });

//Store all messages in a single multi operation request
redisMessages.Lists[listName].AddRange(msgBatch);

This will condense the 1000 redis operations down to 2 operations.

Than if you need to you can fetch all messages with:

var allMsgs = redisMessages.Lists[listName].GetAll();

or specific ranges using GetRange(startingFrom,endingAt) API's.