4

I am trying to improve the performance of a Windows Service, developed in C# and .NET 2.0, that processes a great amount of files. I want to process more files per second.

In its process, for each file, the service does a database query to retrieve some parameters of the system.

Those parameters change annually, and I am thinking that I would gain some performance, if a loaded those parameters as a singleton and refreshed this singleton periodically. Instead of make a database query for each file being processed, I would get the parameters from memory.

To complete the scenario : I am using Windows Server 2008 R2 64 Bits, SQL Server 2008 is the database, C# and .NET 2.0 as already mentioned.

I am right in my approach? What would you do?

Thanks!

Enrico Campidoglio
  • 56,676
  • 12
  • 126
  • 154
Guilherme de Jesus Santos
  • 4,955
  • 5
  • 43
  • 68
  • Sounds about right... Don't know what else to answer though? :P – FarligOpptreden Feb 14 '12 at 12:47
  • Sounds like a good idea to me. A caching system would be efficient. – ken2k Feb 14 '12 at 12:47
  • I guess it depends on how much stuff you need to store. If it is a small amount of data, then go ahead and store it in memory. Memory will always be faster then hard disk reads. While sql server will do a good job at keeping things buffered, if you store everything yourself it will always be in memory. Not sure how much of a trade off you will get though. Use a .net EQATEC profiler (free edition) and compare a query to a memory read. – JeremyK Feb 14 '12 at 12:48
  • Thanks @JeremyK I'll look for EQATEC profiler. – Guilherme de Jesus Santos Feb 14 '12 at 12:55
  • It is a great application. It will show you how long each function took to complete, how many times it was called, etc. And its free. Can't complain with that :) Good luck. – JeremyK Feb 14 '12 at 14:00

3 Answers3

6

Those parameters change anually

Yes, do cache them in memory. Especially if they are large or complex.

You should take care to invalidate them at the right time once a year, depending how accurate that has to be.

Simply caching them for an hour or even for a few minutes might be a good compromise.

H H
  • 263,252
  • 30
  • 330
  • 514
3

RAM memory data access is definitely faster that any other data access, except than cpu memories like registries and CPU cache

Chaching would be faster even if you change it every minute, so yes, caching that query is very faster

Luca C.
  • 11,714
  • 1
  • 86
  • 77
2

Crossing a network or going to disk is always orders of magnitude slower than in memory access.

Databases can cache data in memory so if you can achieve that and you're not crossing a network, the database might be faster since their data access patterns/indexes etc... may be faster than you're code. But, that's best case - if you need it faster, in memory caches help.

But, be aware that in memory caches can add complexity and bugs. You have to determine the lifetime of the cached data, how to refresh and the more complex it is, the more wierd edge case state bugs you will have. Even though they change annually, you have to handle that cusp.

bryanmac
  • 38,941
  • 11
  • 91
  • 99