0

I'm getting this error from time to time performing operations with StackExchange.Redis 1.2.0.0 on Azure Redis from ASP.NET app running on Azure App Service:

Timeout performing EXISTS bookmark:afad5065-29b1-48c7-8bbc-38ddcc9ab4e4type:1, inst: 1, mgr: Inactive, err: never, queue: 3, qu: 0, qs: 3, qc: 0, wr: 0, wq: 0, in: 0, ar: 0, clientName: mywebapp, serverEndpoint: Unspecified/myapp.redis.cache.windows.net:6380, keyHashSlot: 8377, IOCP: (Busy=1,Free=999,Min=200,Max=1000), WORKER: (Busy=1,Free=32766,Min=200,Max=32767), Local-CPU: unavailable (Please take a look at this article for some common client-side issues that can cause timeouts: https://github.com/StackExchange/StackExchange.Redis/tree/master/Docs/Timeouts.md) System.TimeoutException

Connecting with the following configuration: allowAdmin=true,ssl=true,abortConnect=false,connectTimeout=5000,syncTimeout=2000.

Neither client machine nor Redis service does not experience high load in that time. Have read about timeouts already but still have no idea what is causing them in my case. Would appreciate any help.

savbace
  • 185
  • 1
  • 16
  • Have you looked at the Azure documentation specifically for Azure Redis? I would start here: https://azure.microsoft.com/en-gb/blog/investigating-timeout-exceptions-in-stackexchange-redis-for-azure-redis-cache/ – Andy Sinclair May 08 '17 at 14:15
  • Have you net traffic to redis? All ports open? Is your IP on Redis ACL? If configured. – BWA May 08 '17 at 14:15
  • @AndySinclair, yes I've already tried several recommendations from that article but no success. I'm feeling that I'm missing some subtle point in all this. – savbace May 08 '17 at 14:53

1 Answers1

2

Timeouts can be caused by client-side or server-side performance problems, so you have to look at both sides to determine what would be causing it for your application.

Some things to consider:

  • When looking at client or server side perf counters in the azure portal, be careful to look at a relatively small time window (e.g. 1 hour). This is important because the portal aggregates values when you zoom out to longer time windows and this can hide short-lived spikes in load
  • Many performance counters are captured using sampling instead of continuous monitoring, so it is possible that a spike in load is not long enough to show up in the portal. For instance, if CPU usage is logged every 15 seconds but the spike in CPU is only 5 seconds long, it can happen between samples and will not be seen. This is why StackExchange.Redis tries to capture the instantaneous CPU usage when it is about to throw a timeout. Unfortunately, some environments disallow access to system performance counters - thus the "local-cpu: unavailable" message in the timeout error. To counter this, you can try to capture a dump of your process when the timeout happens. The dump will typically contain the CPU info at the time the dump was taken.

I have compiled a list of best practices that help avoid common pitfalls in Redis.

If you still don't see any clear indication of the problem, open a support case with Azure and they will help you investigate.

JonCole
  • 2,902
  • 1
  • 17
  • 19