18

Is there a way to tell Windows that it shouldn't swap out a particular processes' memory to disk?

Its a .Net windows service with fairly large memory usage. I got lot of physical RAM but the OS seems to move part of the process memory to the pagefile anyway.

Luca Martinetti
  • 3,396
  • 6
  • 34
  • 49
  • 2
    Windows moves pages from process working sets on a least recently accessed basis. Generally this gives good results, but specific use cases can require control. Before you do this read Windows Internals (to start with) to better understand what you are trying to do. – Richard Jul 02 '09 at 16:23
  • 1
    I second the comment. In most cases trying to out-smart the Windows memory manager will lead to worse performance. – Martin C. Jul 04 '09 at 18:10
  • 1
    Key question: why is this ("moving part of the process memory to the pagefile") a problem anyway? – Stobor Jul 06 '09 at 05:59
  • 4
    I think it's bad form to question motivation's like that... the constraint's that developers have to uphold are often not of their choosing and firm requirements. – RandomNickName42 Jul 06 '09 at 08:48
  • 2
    @RandomNickName42: trust me, I understand the pressures on the programmer, I wear that hat too. However, as a programmer, it's important for me to know when I'm doing something which is considered unusual, and to know what extenuating circumstances mandate it... I was actually asking to figure out whether executable files needed to maintained in RAM as well and whether file IO buffers would be affected... – Stobor Jul 06 '09 at 14:09
  • @Stobor In a server system, I want the server app to be ready to roll with <100ms response times when a user makes a request after no action of a few hours - even if the virus scanner was running in the meantime. – John Jul 13 '17 at 17:16
  • 1
    @Stobor One common use reason to require this is if you have a memory block storing sensitive information like unencrypted user passwords or medical data. – Randy the Dev Mar 01 '18 at 02:05
  • @AndrewDunn Good example. Although, unless a fairly large chunk of that "fairly large memory usage" is storing the sensitive data, it might make more sense to lock only a section of the process memory into RAM, rather than the whole process. – Stobor Mar 04 '18 at 13:42

3 Answers3

18

You can use VirtualLock to prevent memory from being paged to disk, but I really think you're better off letting the OS manage the system's memory. It's pretty good at it, and I wouldn't second guess why the OS was swapping things to disk unless I really knew what I was doing.

Eric Petroelje
  • 59,820
  • 9
  • 127
  • 177
  • 4
    If it was good at it, would we see so many apps taking seconds to open a context menu just because some copy operation is on and blocking paging io? – John Jul 13 '17 at 17:13
  • Disabled paging requirement is legitimate for presentation applications, specially, for streaming ones, for live audio applications it is inevitable requirement, since ASIO and other modern audio drivers do not actually take any intelligent care of buffer state (to reduce latency), thus any unexpected delay may break audiobility and introduce extremely unwanted effects. – azis.mrazish Jun 05 '21 at 11:59
14

VirtualLock by default will run out of quota in a hurry, even if you permission the user (by default the security policy setting of "Lock pages in memory" has no entries (not even admin)).

If you do configure this, you will need to set "Adjust memory quotas for a process" also.

This question is a bit ambiguous, what part of the VM space do you want to prevent from being swapped out? Heap/Stack/Module's?...?

Overall, one solution that I had used in the past, is to create a DLL with a large section which was READ + WRITE, unmoveable, you can also mark it EXECUTE and SHARED if need be, but then I would HeapCreate into that module's section, allowing me to use it as a heap.

You can look up the exact bit's to set here, IMAGE_SCN_MEM_NOT_PAGED looks like it would do the trick.

If you go through the process of marking all of your module's and sections with this bit, the Windows module loader will not charge a articular user quota or require policy settings be modified on each system you deploy your code. You will need to code in a special shim to provide access to the HEAP's you create into these NON_PAGEABLE dll's also.

It is a bit of work but it worked well for me in the past, I had the added requirement to share memory to multiple processes, which was based at the same address.

I would think the easiest thing to try would be to disable your page file entirely. But before this, if your using Vista/2008+ OS's, the swapping you see may be due to superfetch, which could be proactively "tuning" your system with the assumption that in the near future you need to use one application or another. Some other easy tasks would be to stop unused services like search, which could be indexing huge amounts of files, also you should go into your "task scheduler", which configures system tasks, by default there are a few dozen on most systems with default action's that will transmit all Dr. Watson dump's to MS, defrag your drive and a number of other possibly exceedingly memory intensive type operations.

You could reply with a bit more detail to get some better answers... but one other suggestion would be to simply purchase a large solid state drive and use that exclusively for swap, beware that these degrade over time in overall performance an size due to bad block mappings that are common with all existing SSD technologies.

I recently came across a codeplex project called "non-paged clr host", from their page:

Implementation From an implementation perspective, the non-paged CLR host uses the SetProcessWorkingSetSize, SetProcessWorkingSetSizeEx (on Windows Server 2003 and above) and VirtualLock APIs to ensure that memory allocated by it is locked into physical memory. Note that using the above APIs does not guarantee with absolute certainty that no paging will occur; instead, it minimizes the odds of it occurring to very exceptional scenarios. In some load tests we have conducted, even when the system as a whole was hogged by lack of physical memory, no page faults were observed in the process using the non-paged CLR host.

RandomNickName42
  • 5,923
  • 1
  • 36
  • 35
0

Are you sure that the pages in question are being evicted from memory? It's possible that the VM subsystem is proactively writing dirty pages to disk, without eviction, so that potential future allocations have lower latency.

Also, if the system is doing a large amount of IO, it may be better for performance to use the memory in question for buffers rather than application data, as described by Martin Pool

Phil Miller
  • 36,389
  • 13
  • 67
  • 90
  • MS released a tool DynCache recently, which allows you to pre-configure or otherwise setup some preferences in these scenerio's, http://blogs.msdn.com/ntdebugging/archive/2009/02/06/microsoft-windows-dynamic-cache-service.aspx & http://sqlblogcasts.com/blogs/grumpyolddba/archive/2009/03/18/x64-memory-problems.aspx – RandomNickName42 Jul 06 '09 at 06:56
  • Gheeze I hate to rip on you but I really need the points ;). The reference you cited is for linux, the subtile varieance of vernacular can be quite confusing if not clearly stated... If you refer to ( http://msdn.microsoft.com/en-us/library/cc644950(VS.85).aspx ), "When FILE_FLAG_NO_BUFFERING is combined with FILE_FLAG_OVERLAPPED, the flags give maximum asynchronous performance, because the I/O does not rely on the synchronous operations of the memory manager. However, some I/O operations take more time, because data is not being held in the cache." - So Windows = NO_BUFFER > * :) – RandomNickName42 Jul 06 '09 at 09:15