0

I need to handle the large array of mpz_t and release the memory afterwords. However, when profiling the RSS at each stage with htop command, mpz_clear does not seem to free the memory completely.

Here's the sample C code:

#include <stdio.h>
#include <stdlib.h>
#include <gmp.h>

int main(void) {
    mpz_t *A;
    int size = 1<<20;
    A = (mpz_t *) malloc(size*sizeof(mpz_t));
    getchar(); // RSS=876KB

    for (int i=0; i<size; i++) {
        mpz_init_set_ui(A[i],0);
    }
    getchar(); // RSS=50MB

    for (int i=0; i<size; i++) {
        mpz_clear(A[i]);
    }
    free(A);
    getchar(); // RSS=34MB!!

    return 0;
}

Looking at the answer here, the memory deallocation process above (i.e. call mpz_clear on mpz_t elements & free array) is properly done.

I also tried with the vector of mpz_class C++ wrapper as follows, which resulted in the same situation.

#include <gmpxx.h>
#include <iostream>
#include <vector>

int main(void) {
    std::vector<mpz_class> A;
    int size = 1 << 20;
    A.reserve(size);
    std::cin.ignore();

    for (int i=0; i<(size); i++) {
        A.emplace_back(0);
    }
    std::cin.ignore();

    std::vector<mpz_class>().swap(A);
    std::cin.ignore();

    return 0;
}

Is there any way to release the whole memory allocated for an mpz_t array?

Update

Thanks to the comment section I've learnt that free does not necessarily release the memory back to the OS. But in my case, using glibc on Ubuntu 16.04, free() does release the amount of memory reserved by malloc (i.e 16*2^20 = 16MB, assuming that sizeof(mpz_t) is 16-byte) because it exceeds the default M_MMAP_THRESHOLD value (=128KB).

So I suspect that there is still some orphaned memory implicitly allocated by GMP (≈32MB in the above example) and free() needs to be called on it separately.

  • 1
    Why don't you use the C++ API for GMP? – Kerrek SB Jan 02 '18 at 17:22
  • 1
    Possible duplicate of https://stackoverflow.com/questions/15139436/why-free-doesnt-really-frees-memory ? – EdmCoff Jan 02 '18 at 17:23
  • Possible duplicate of [Why free() doesn't really frees memory?](https://stackoverflow.com/questions/15139436/why-free-doesnt-really-frees-memory) – melpomene Jan 02 '18 at 17:26
  • @KerrekSB I did. Please look at the second sample code. – Akira Takahashi Jan 02 '18 at 17:33
  • Ah, yes. Your system allocator probably just doesn't return memory to the OS. Which allocator are you using? – Kerrek SB Jan 02 '18 at 18:00
  • 1
    @AkiraTakahashi -- What would the results be if you called `malloc`, and then immediately called `free()`? Would your results be the same? If so, then the issue has nothing to do with `gmp`, but how the heap manager handles the memory. – PaulMcKenzie Jan 02 '18 at 18:02
  • @KerrekSB I'm using glibc's malloc on Ubuntu 16.04 – Akira Takahashi Jan 03 '18 at 03:54
  • @PaulMcKenzie It wasn't the same (either with `calloc`). `free` immediately returned the memory to the OS. Additionally, I experimented with an array/vector of `uint64_t` but the allocated memory was completely released back in this case. – Akira Takahashi Jan 03 '18 at 04:03
  • Big chunks are usually allocated by a different method, and *are* released back to the OS. You probably shouldn't care about any of this unless you are writing your own allocator. – n. m. could be an AI Jan 03 '18 at 09:39
  • malloc followed immediately by free won't necessarily be reflected in the RSS. Watching RSS is not a good way to understand your heap performance. – n. m. could be an AI Jan 03 '18 at 09:45

0 Answers0