3

I was writing a program which does some basic object detection with cuda. I ran into a problem where I allocate unified memory with cudaMallocManaged, do some processing with it and then free it with cudaFree. Event though, cudaFree never returned an error, the memory never seems to actually get released, as task manager shows that both system memory usage and gpu shared memory usage are continously increasing. Is there something fundamentally wrong about my understanding of unified memory or is this a bug?

Minimal example:

#include "cuda_runtime.h"
#include "device_launch_parameters.h"
#include <cassert>

int main()
{
    while (1)
    {
        void* ptr;
        cudaMallocManaged(&ptr, 1 << 20);
        assert(cudaFree(ptr) == cudaSuccess);
    }
}

Im using Windows 10, cuda version is 10.2, driver version is 26.21.14.4122.

thebear8
  • 194
  • 2
  • 11
  • It's normal for memory not to return to the operating system for normal (non-CUDA) memory allocations even after free-ing it - it will depend on operating system, configuration, etc. I would expect the same is the case for CUDA unified memory. You're allocating 1mb repeatedly which is likely far under the memory capacity of whatever device you're using. It may be that you're just not allocating enough memory each time to hit a threshold where that memory is released back to the OS on free. – Ryan Pepper Apr 21 '20 at 15:05
  • @RyanPepper actually, i don't think that's whats happening. I just tested it with ~1gb allocations and it doesn't get released properly. – thebear8 Apr 21 '20 at 15:37
  • 2
    When I run your code with I don't see any indication that memory usage is continually increasing in task manager. The GPU shared memory usage is zero and remains at zero. For a 1GB allocation size (2^^30), the memory usage reported in dedicated memory usage fluctuates from a high of 1GB down to almost zero, in approximately a sinusoidal pattern. Win 10, RTX 2070 GPU, CUDA 10.1.243, driver 432.00. I don't have a machine convenient that has your exact setup, and you haven't indicated your GPU anyway. [here](https://imgur.com/a/E7ImCg1) task manager with your app running with 1GB allocation – Robert Crovella Apr 21 '20 at 16:51
  • @RobertCrovella I'm pretty sure my driver installation was corrupt or the driver had a bug because after reinstalling cuda and installing the new driver (Version 445.87), it works fine. Im using an RTX2060 by the way. – thebear8 Apr 21 '20 at 17:48

1 Answers1

1

Either my driver installation was corrupt or it's a driver bug. The way I fixed was by reinstalling cuda and then reinstalling the latest gpu driver (the game-ready driver from the nvidia website). Im not sure why it was corrupt in the first place though.

EDIT: new driver version is 445.87

thebear8
  • 194
  • 2
  • 11