Is it possible to keep DLL in memory after calling process exits?

Question

I have a DLL that takes 5 to 10 seconds to load, which means that I have to wait that long every time I compile and run the executable that uses it. Is there a way to keep the DLL loaded in memory so that it can be immediately accessed every time I compile the corresponding executable? I'm compiling the program on QT MinGW, if that's relevant.

EDIT: No luck so far. Loading the DLL on another program seems to have no effect (the original program still loads the DLL, and takes just as long to do so). I would guess I need to load the DLL and its functions differently if they've been loaded into another program, but I don't know how to do this. Right now I'm using LoadLibrary and GetProcAddress.

It's likely not the loading-into-memory, but initialization logic inside the DLL, that's taking so long. Keeping the DLL in memory wouldn't avoid the initialization delay. — Ben Voigt, Jun 04 '12 at 16:06
That would be unfortunate if true. I assume there's nothing I can do about that without access to the source code? — SharpHawk, Jun 04 '12 at 16:17
You can wrap it into an out-of-process RPC server, but that trades fast re-connection for inefficiency using it. — Ben Voigt, Jun 04 '12 at 19:00
Ben Voigt's answer is most probably correct. Hard disk's reading speed is about 40-80 MB/s. So unless your dll is really huge (200-400 MB), the time is not spent loading the dll from disk. In your [other question](http://stackoverflow.com/questions/10835501/how-to-reduce-qt-program-start-up-time-when-using-a-dll) you mention a dll of size 5 MB. That dll seems to deal with some external hardware, so the time is most probably spent detecting or initializing the hardware. You could try to contant the dll vendor and ask if they could do something about the starting time. — , Jun 04 '12 at 21:28
And there's no way to keep the DLL initialized and usable by whatever program I choose to run? In other words run program A which loads and initializes the DLL, then run program B from that point on which actually uses the functions on the DLL (I'm asking whether this can be done generally, not for this specific library). I realize that DLL's function by being linked to the program that calls it, so the above is unlikely, but I figured I'd ask just in case. — SharpHawk, Jun 04 '12 at 21:53
You said in the comment to user877329's answer that you had multiple instances running, but each of them had this startup delay. So my guess is that the time is spent in the [dll's entry point](http://msdn.microsoft.com/en-us/library/windows/desktop/ms682583%28v=vs.85%29.aspx), which is called for every process that loads the dll. As somebody already suggested, you could try to load the dll dynamically in another thead, but still, of course, you can't use the dll's functions before it is initialized. But at least the user interface would be responsive. — , Jun 05 '12 at 05:13
True, and it will help the end-user, but I as the developer will still have to wait for the DLL to be up and running. Right now I'm using delay loading so that I only have to wait if the DLL is actually used. Alright, thanks for the help. — SharpHawk, Jun 05 '12 at 14:16

score 2 · Answer 1 · edited May 23 '17 at 12:06

I am not MinGW developer, but your question is very common and not really depend on how you create the DLL. Such problems will by typically solved with the usage of three techniques:

choosing of unique base addresses of the DLLs
binding the DLLs and exe (during installation of the application)
usage of DLL delay loading technique
a little improve the loading time the call of DisableThreadLibraryCalls inside of DLL_PROCESS_ATTACH part of DllMain

The exact switches of linker or other tools which you can use depend on your development environment.

To understand the problem you should know how executable or DLL will be loaded. First of all the EXE or DLL will be mapped in the memory. Memory mapped file (section) will be created which points to the EXE/DLL. So you will have some addresses in the process which access will corresponds to the EXE/DLL file. If you link the DLL you can choose the base address. If the address is unused in the process address space then nothing will be done. If the first line of code will be used (you call some function from the DLL) then the page of memory 8K near the used address will be loaded in memory from the file. If two processes use the same DLL then the physical memory for the code will be shared between processes. Even if you hold initialized variables the page with the variables will be shared till the first changes of the variable. On modification will be made the copy of the page of memory for the process which did the modification.

After the DLL is loaded in the process some small parts of the caller (the EXE for example) have to be modified to include the actual addresses of the functions used from the DLL. The same will be done with the DLLs which use functions from another DLL.

Everything sound perfect, but if you don't set any linker options during DLL compilation (if you don't use --image-base or --enable-auto-image-base linker option) you will have all your DLLs with the same base address (default value for the linker). So the first DLL could be loaded at the address. During loading of the second DLL which are linked with the same (or some overlapped address) the relocation of the DLL will be done. During the relocation the code of DLL sill be modified and so 1) the loading of the DLL will be slowly 2) the modified copy of the code will be made in the process (which include the memory used by DLL) 3) the modified copy will be not shared between multiple instances of the DLL (even if all instances will be modified in the same way).

I recommend you first of all to use Process Explorer for example to verify which DLLs will be relocated in your application. You should choose the option "DLLs" in the "View"/"Lower Pain View" menu and select "Relocation DLLs" checkbox in "Configure Highlighting" of the "Options" menu. You can additionally customize which information about every DLL will be displayed. The more information like below you will see the more slow the program will be loaded and the more address space will be not shared between instances of your application or between different applications which use the same DLL:

enter image description here

In the above example you see that tree Lenovo DLLs TPOSDSVC.dll, HKVOLKEY.dll and TPLHMM.dll are linked with the same base address 0x10000000 and only one DLL (TPOSDSVC.dll here) will be loaded at the address. Two other DLLs have to re relocated.

I can't write a book here about the subject. I recommend you to examine your application on the relocation problem. You can either use linker options (--image-base or --enable-auto-image-base seems be what you need). You can use dumpbin.exe tool (from Visual Studio also in the free edition) to examine the PE image.

After all your DLLs will have unique base address you can use another tool bind.exe with option -u to bind the EXE and your DLLs to its dependent DLLs. It will additionally reduce the memory size and improved the starting time of the application. It will update IMAGE_DIRECTORY_ENTRY_IMPORT and IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT parts of your DLL and EXE (see the answer). Bind.exe uses BindImageEx API internally. Many Windows Installer setups use BindImage action and BindImage table to make the binding at the end of installation of EXE and DLLs.

You can consider other techniques (see here) to reduce the size of DLL and EXE.

I don't know exactly how you can use delay-load technique in MinGW, but it should be definitively possible. The Visual Studio you need to do two steps: include Delayimp.lib as additional library and use /DELAYLOAD option (see here) to specify which from the DLLs should be loaded at the first usage instead of directly. Using very helpful Tool Dependency Walker you can see that most standard Microsoft DLLs uses the technique. You can improve the start time of your application and reduce the used memory if you would use the technique too.

totowtwo · Answer 2 · 2012-06-04T16:41:23.993

1

Create an explicitly installed system service that keeps your DLL loaded. That way initialization happens at boot time and never again. I would recommend against the methods outlined in most of these other answers. While they look like they work, they feel to me like bad behavior on the part of your software. From both a user's and maintainer's viewpoint, I would prefer an explicitly installed system service over hooking something into winlogin.exe. The more honestly you use the Windows APIs and environment, the less breaking changes you will have across versions and at version upgrades.

edited Jun 04 '12 at 16:41

answered Jun 04 '12 at 16:12

totowtwo

2,101
1
14
21

It certainly appears that way, so I edited away my "No". I feel uneasy about those solutions though. – totowtwo Jun 04 '12 at 16:42
Initialization occurs for every process and thread that attaches to the DLL. Keeping the DLL loaded into a service only saves the file I/O. – Ben Voigt Jun 04 '12 at 18:53
Can you go into more detail on how to do this? As it stands your answer is mostly a critique of the other suggestions provided. – SharpHawk Jun 04 '12 at 18:55
@SharpHawk: If the concern is compile iteration time, there's no real need to make a service. Just write a program and leave it open, as suggested by user877329. A system service is more commonly used for keeping DLLs in memory in the deployed customer environment. For example, the SeaMonkey browser does this, as do several office suites. It's all a lie, however, not actually making things faster, but just obscuring the reason the computer is slow, so the customer doesn't know which software to blame. – Ben Voigt Jun 04 '12 at 18:57
In light of the above and the fact that this answer suggests doing the loading at boot time, instead of the first time it's actually needed, I have to -1. – Ben Voigt Jun 04 '12 at 18:58

score 1 · Answer 3 · answered Jun 04 '12 at 16:14

1

If I am not wrong, Windows keeps one instance of the DLL in memory so keeping this alive should work:

#include <conio.h>
#include <windows.h>

int main()
    {
    HMODULE handle=LoadLibrary("yourdll.dll");
//  Make shure Windows resolves the DLL
    FARPROC dummy=GetProcAddress(handle,"functionInDll");
//  The process will now just wait for keyboard input.
    getch();
    CloseHandle(handle);
    return 0;
    }

answered Jun 04 '12 at 16:14

user877329

6,717
8
46
88

Why you do FARPROC dummy=GetProcAddress(handle,"functionInDll"); if you want just keep DLL loaded? – rkosegi Jun 04 '12 at 16:15
Mhm, I tried leaving an instance of my program open (with the DLL loaded on it), but newer instances of the program still took the same amount of time to load the DLL. Using the functions themselves is always instantaneous. I wonder what the problem is... – SharpHawk Jun 04 '12 at 19:22

score 1 · Accepted Answer · answered Jun 05 '12 at 07:58

1

The easiest solution is (assuming MSVC++) to make the DLL delay-loaded. The trade-off of course is that the initializaton still has to happen, but this will no longer delay other parts of your program. E.g. you can do it on a background thread.

answered Jun 05 '12 at 07:58

MSalters

173,980
10
155
350

Given the fact that I couldn't get the DLL to stay in memory, and that totowtwo's solution would have meant slowing down cheap laptops even further, loading the DLL in a parallel thread seems like the way to go. The same solution was presented in a related question I asked here on StackOverflow, also. – SharpHawk Aug 27 '12 at 21:07

Is it possible to keep DLL in memory after calling process exits?

4 Answers4