0

We are including Redis cache in our Saas application in Azure.

But we are having some problems, we see these types of errors from time to time

Timeout performing EVAL, inst: 2, queue: 23, qu: 0, qs: 23, qc: 0, wr: 0, wq: 0, in: 65536, ar: 0, 
IOCP: (Busy=1,Free=999,Min=1,Max=1000), WORKER: (Busy=2,Free=8189,Min=1,Max=8191)

Timeout performing EVAL, inst: 2, queue: 3, qu: 0, qs: 3, qc: 0, wr: 0, wq: 0, in: 65536, ar: 0, 
IOCP: (Busy=1,Free=999,Min=1,Max=1000), WORKER: (Busy=3,Free=8188,Min=1,Max=8191)

When trying to interpret these logs, I have a hard time knowing where to start, where to look and draw conclusions.

Please, can someone tell me something about these particular values? The Redis instance is in the same zone as the server, I use the c1 plan with 1GB of memory, however these errors come out by doing simple tests.

The settings are:

ConnectionTimeoutInMilliseconds = "2000" operationTimeoutInMilliseconds = "1500" retryTimeoutInMilliseconds = "4500"

In the redis dashboards in Azure I do not see any memory spikes, connections or anything else, should I look at any particular value?

David Makogon
  • 69,407
  • 21
  • 141
  • 189
Jsanchez
  • 83
  • 9

1 Answers1

1

The “in: 65536” value in the timeout is very high.  This value indicates how much data is sitting in the client’s socket kernel buffer.  This indicates that the data has arrived at the local machine but has not been read by the application layer.  This typically happens when 1) thread pool settings need to be adjusted or 2) when client CPU is running high.

IOCP: (Busy=1,Free=999,Min=1,Max=1000), WORKER: (Busy=2,Free=8189,Min=1,Max=8191) and IOCP: (Busy=1,Free=999,Min=1,Max=1000), WORKER: (Busy=3,Free=8188,Min=1,Max=8191) indicate that you are hitting threadpool growth throttling issues. You might want to look into the explanation and recommendations here: https://gist.github.com/JonCole/e65411214030f0d823cb#file-threadpool-md.

Carl Dacosta
  • 873
  • 4
  • 13
  • Thanks for your answer. I have configured the threadpool growth. But unfortunately it has not worked for me. One of the problems I have detected is that when using ReportViewer to render reports, it creates very complex objects – Jsanchez Jun 01 '17 at 09:56
  • When you say "it has not worked for me", what errors are you receiving now? – Carl Dacosta Jun 01 '17 at 13:34
  • The same error as detailed above. I have tried to develop a solution, deleting the Keys from ReportViewer every time I render a new one. It seems to work right now. I tried to configure the threadPool but I do not know how to do it when configured from a SessionStateProvider in a web.config Thank you for your reply. – Jsanchez Jun 08 '17 at 08:06
  • Can you not call the Threadpool.SetMinThreads(...) API from Global.asax inside your application_start? https://msdn.microsoft.com//en-us/library/system.threading.threadpool.setminthreads(v=vs.100).aspx. Alternatively, you could use the minioThreads or minWorkerThreads settings in config. https://msdn.microsoft.com/en-us/library/7w2sway1(v=vs.100).aspx. Important Note: The value specified in this configuration element IS a per-core setting. For example, if you have a 4 core machine and want your minIOThreads setting to be 200 at runtime, you would use . – Carl Dacosta Jun 08 '17 at 16:43