Why does this memoized code segfault?

Question

I have the following code that computes the Stirling number of the second kind for a given n and k,

#include <cstdint>
#include <map>

#include <boost/multiprecision/cpp_int.hpp>

namespace mp = boost::multiprecision;


mp::cpp_int stirlingS2(unsigned n, unsigned k)
{
    if (n == 0 && k == 0) {
        return 1;
    }

    if (n == 0 || k == 0) {
        return 0;
    }

    static auto memo = std::map<std::pair<unsigned, unsigned>, mp::cpp_int>();
    auto nKPair = std::pair<unsigned, unsigned>(n, k);

    if (memo.count(nKPair) > 0) {
        return memo[nKPair];
    }

    auto val = k * stirlingS2(n - 1, k) + stirlingS2(n - 1, k - 1);

    memo[nKPair] = val;

    return val;
}

Unfortunately, when this code is running, it segfaults. It seems to run fine for the first 87795 values inserted into memo, but then crashes shortly thereafter. Specifically, the segfault happens at map::count, in the line if (memo.count(nKPair) > 0) {. I thought maybe this was a matter of memo running out of size, so I added the following caveat to the assignment to memo,

if (memo.size() < memo.max_size()) {
    memo[nKPair] = val;
}

But that didn't help. I've also noticed that the 87795 value is not indicative of when this crashes. With some minor modifications, changing the first if statement to,

if (n <= k) {
    return 1;
}

changes that value to 66453.

Does anyone know what's going on here?

are you positive the segfault is on the `memo.count` line and not to do with blowing the stack through deep recursion as a result of not properly memoizing? — obataku, Aug 08 '16 at 22:37
@DavidThomas, doesn't `operator[]` insert a new element for a key if the key doesn't already exist? — Jordy Dickinson, Aug 08 '16 at 22:38
@JordyDickinson yes it does... But everything in the map is a default constructed cpp_int. Is that intended? — David Thomas, Aug 08 '16 at 22:41
@DavidThomas, I thought it was initialized as empty? I didn't give the constructor any parameters. — Jordy Dickinson, Aug 08 '16 at 22:44
Agreed your map is empty on construction. Your only insert is protected by an if that will never be true. (Unless there is code that is missing from your example) — David Thomas, Aug 08 '16 at 22:47
Why do you do `if (memo.count(nKPair) > 0) { return memo[nKPair]; }` instead of `auto it = memo.find(nKPair); if (it != memo.end()) return it.second;`? And where do you insert into the map? — kfsone, Aug 08 '16 at 22:48
Oops. I left out a pretty critical part of the code. Sorry, guys. — Jordy Dickinson, Aug 08 '16 at 22:50
@kfsone, I could do it that way. It doesn't change the segfault though. — Jordy Dickinson, Aug 08 '16 at 22:54
Comment out your six lines for memoization and see if all is well. — David Thomas, Aug 08 '16 at 23:01
Could it be that I'm blowing the stack by not making `memo` a pointer? — Jordy Dickinson, Aug 08 '16 at 23:01
@DavidThomas, before I added the memoization it worked fine. It just took forever to compute the values I was interested in. — Jordy Dickinson, Aug 08 '16 at 23:02
@M.M I tried making an MCVE, and magically it works. It must have something to do with the calling code then, but then why a segfault here? Once it gets to this function, shouldn't everything be pretty much encapsulated? I'm not referencing any globals... — Jordy Dickinson, Aug 08 '16 at 23:28
@M.M So basically the error could be literally anywhere in the codebase... — Jordy Dickinson, Aug 08 '16 at 23:52
Basically yeah..:) Try to narrow it down with *divide and conquer* - comment out half your code, see whether the problem goes away or not, then you have a good idea which half the problem was in.. repeat recursively — M.M, Aug 08 '16 at 23:54
In times of segfault, always start by searching for `[index]` expressions with a wrong value of `index`. — barak manos, Aug 09 '16 at 05:27

score 1 · Answer 1 · answered Aug 09 '16 at 04:51

Ok, so after hours of confusion, I narrowed this down to an expression templates issue. I don't really fully understand why, but it all had to do with that little auto in the line

auto val = k * stirlingS2(n - 1, k) + stirlingS2(n - 1, k - 1)

Basically, change that auto to mp::cpp_int and suddenly, no segfault.

Why does this memoized code segfault?

1 Answers1