SIGSEGV within std::sort, how to narrow it down

Question

This is a related post to this one as it deals with the same program, but i now implemented it iterative and not recursive anymore, but I still get SIGSEGV (but later) while running the program. I did some other changes to my program to narrow it down, and I figured that changing a vector of objects, to a vector of ptr to the objects on the heap does give me some extra rounds (about 200), but still crashes. I suggest that somehow my memory to save variables within the program gets exhausted, but as I dump the stacksize of the program:

rlimit rlim;
getrlimit(RLIMIT_STACK,&rlim);

std::cout << "rlim_cur ist:" << rlim.rlim_cur << std::endl;
std::cout << "rlim_max ist:" << rlim.rlim_max << std::endl;

the output is:

rlim_cur ist:8388608
rlim_max ist:18446744073709551615

and this seems to be pretty big and not used up, are there any other relevant limits to dump to narrow my problem down further and hopefully solve it?

Here the dump of my debugger:

Program received signal SIGSEGV, Segmentation fault.
0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98
98      return left_over_capacity;
(gdb) backtrace
#0  0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98
#1  0x000000000040b9ab in Town::compare_by_capacity (eins=0x0, zwei=0x0) at ./solver/Darstellung.cpp:135
#2  0x00000000004124c7 in std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__a=..., __b=..., __c=..., 
    __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:108
#3  0x0000000000411250 in std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., 
    __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2260
#4  0x000000000040f111 in std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, long, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __depth_limit=21, 
    __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2302
#5  0x000000000040de63 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>)
    at /usr/include/c++/4.5/bits/stl_algo.h:5250
#6  0x000000000040ce5a in Solution_Stack::get_towns_by_capacity (this=0x7fffffffe010) at ./solver/Darstellung.cpp:331
#7  0x000000000040a6cf in solver::treat_towns_with_zero_capacity (ptr=0x7fffffffe010) at ./solver/Solver.cpp:184
#8  0x0000000000409ff2 in solver::solve_problem (ptr=0x7fffffffe010) at ./solver/Solver.cpp:94
#9  0x000000000041475f in main (argc=3, argv=0x7fffffffe208) at ./main/Main.cpp:50

The new dump after adding a check-function to trace down zero values:

    #0  0x000000000040b2a0 in Town::get_cur_capacity (this=0x40) at ./solver/Darstellung.cpp:98
    #1  0x000000000040b9e9 in Town::compare_by_index (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:139
    #2  0x000000000040bad1 in Town::compare_by_index_inv (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:153
    #3  0x00000000004127ea in std::__unguarded_partition<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, Town*, bool (*)(Town const*, Town const*)> (
        __first=..., __last=..., __pivot=@0x631ef0, __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>)
        at /usr/include/c++/4.5/bits/stl_algo.h:2229
    #4  0x0000000000411444 in std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (
        __first=..., __last=..., __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:2261
    #5  0x000000000040f2c5 in std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, long, bool (*)(Town const*, Town const*)> (
        __first=..., __last=..., __depth_limit=7, __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>)
        at /usr/include/c++/4.5/bits/stl_algo.h:2302
    #6  0x000000000040e017 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., 
        __comp=0x40baae <Town::compare_by_index_inv(Town const*, Town const*)>) at /usr/include/c++/4.5/bits/stl_algo.h:5250
    #7  0x000000000040d1e6 in Solution_Stack::get_partners_of_by_index_inv (this=0x7fffffffe010, id=523) at ./solver/Darstellung.cpp:371
    #8  0x000000000040a4d7 in solver::treat_towns_considering_their_index (ptr=0x7fffffffe010) at ./solver/Solver.cpp:165
    #9  0x000000000040a016 in solver::solve_problem (ptr=0x7fffffffe010) at ./solver/Solver.cpp:100
    #10 0x0000000000414913 in main (argc=3, argv=0x7fffffffe208) at ./main/Main.cpp:50

Valgrind gave me this as a dump, just at the place where the problem used to occure anyways. I is quite confusing to me, due to its size:

==16150== Invalid read of size 4
==16150==    at 0x40B2A0: Town::get_cur_capacity() const (Darstellung.cpp:98)
==16150==    by 0x40B9AA: Town::compare_by_capacity(Town const*, Town const*) (Darstellung.cpp:135)
==16150==    by 0x4124C6: void std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:108)
==16150==    by 0x41124F: __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > > std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:2260)
==16150==    by 0x40F110: void std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)) (stl_algo.h:2302)
==16150==    by 0x40DE62: void std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:5250)
==16150==    by 0x40CE59: Solution_Stack::get_towns_by_capacity() (Darstellung.cpp:331)
==16150==    by 0x40A6CE: solver::treat_towns_with_zero_capacity(Solution_Stack*) (Solver.cpp:184)
==16150==    by 0x409FF1: solver::solve_problem(Solution_Stack*) (Solver.cpp:94)
==16150==    by 0x41475E: main (Main.cpp:50)
==16150==  Address 0x8 is not stack'd, malloc'd or (recently) free'd
==16150== 
==16150== 
==16150== Process terminating with default action of signal 11 (SIGSEGV)
==16150==  Access not within mapped region at address 0x8
==16150==    at 0x40B2A0: Town::get_cur_capacity() const (Darstellung.cpp:98)
==16150==    by 0x40B9AA: Town::compare_by_capacity(Town const*, Town const*) (Darstellung.cpp:135)
==16150==    by 0x4124C6: void std::__move_median_first<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:108)
==16150==    by 0x41124F: __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > > std::__unguarded_partition_pivot<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:2260)
==16150==    by 0x40F110: void std::__introsort_loop<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, long, bool (*)(Town const*, Town const*)) (stl_algo.h:2302)
==16150==    by 0x40DE62: void std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)>(__gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, __gnu_cxx::__normal_iterator<Town**, std::vector<Town*, std::allocator<Town*> > >, bool (*)(Town const*, Town const*)) (stl_algo.h:5250)
==16150==    by 0x40CE59: Solution_Stack::get_towns_by_capacity() (Darstellung.cpp:331)
==16150==    by 0x40A6CE: solver::treat_towns_with_zero_capacity(Solution_Stack*) (Solver.cpp:184)
==16150==    by 0x409FF1: solver::solve_problem(Solution_Stack*) (Solver.cpp:94)
==16150==    by 0x41475E: main (Main.cpp:50)
==16150==  If you believe this happened as a result of a stack
==16150==  overflow in your program's main thread (unlikely but
==16150==  possible), you can try to increase the size of the
==16150==  main thread stack using the --main-stacksize= flag.
==16150==  The main thread stack size used in this run was 8388608.
==16150== 
==16150== HEAP SUMMARY:
==16150==     in use at exit: 771,174 bytes in 19,239 blocks
==16150==   total heap usage: 9,821,251 allocs, 9,802,012 frees, 384,861,557 bytes allocated
==16150== 
==16150== 50,678 bytes in 1,491 blocks are possibly lost in loss record 28 of 35
==16150==    at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150==    by 0x4ECBE6C: std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150==    by 0x4ECC08D: std::string::_M_mutate(unsigned long, unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150==    by 0x4ECC730: std::string::erase(unsigned long, unsigned long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.14)
==16150==    by 0x407FB6: utility::split_helper(std::string, std::string) (Tools.cpp:28)
==16150==    by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150==    by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150==    by 0x40539C: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:27)
==16150==    by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150==    by 0x414699: main (Main.cpp:34)
==16150== 
==16150== 62,606 (11,928 direct, 50,678 indirect) bytes in 1,491 blocks are definitely lost in loss record 30 of 35
==16150==    at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150==    by 0x407F68: utility::split_helper(std::string, std::string) (Tools.cpp:24)
==16150==    by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150==    by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150==    by 0x40539C: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:27)
==16150==    by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150==    by 0x414699: main (Main.cpp:34)
==16150== 
==16150== 94,406 (18,440 direct, 75,966 indirect) bytes in 2,305 blocks are definitely lost in loss record 32 of 35
==16150==    at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150==    by 0x407F68: utility::split_helper(std::string, std::string) (Tools.cpp:24)
==16150==    by 0x4080B5: utility::split_helper(std::string, std::string) (Tools.cpp:49)
==16150==    by 0x4081C3: utility::split(std::string, std::string) (Tools.cpp:66)
==16150==    by 0x40573F: parser::get_finished_cities(std::vector<std::string, std::allocator<std::string> >, std::vector<City*, std::allocator<City*> >) (Parser.cpp:42)
==16150==    by 0x40511A: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150==    by 0x414699: main (Main.cpp:34)
==16150== 
==16150== 178,720 (131,208 direct, 47,512 indirect) bytes in 1,491 blocks are definitely lost in loss record 35 of 35
==16150==    at 0x4C28B42: operator new(unsigned long) (vg_replace_malloc.c:261)
==16150==    by 0x40541B: parser::get_city_prototypes(std::vector<std::string, std::allocator<std::string> >) (Parser.cpp:28)
==16150==    by 0x4050EB: parser::get_problem_configuration(std::string, std::string) (Parser.cpp:17)
==16150==    by 0x414699: main (Main.cpp:34)
==16150== 
==16150== LEAK SUMMARY:
==16150==    definitely lost: 161,576 bytes in 5,287 blocks
==16150==    indirectly lost: 174,156 bytes in 5,287 blocks
==16150==      possibly lost: 50,678 bytes in 1,491 blocks
==16150==    still reachable: 384,764 bytes in 7,174 blocks
==16150==         suppressed: 0 bytes in 0 blocks
==16150== Reachable blocks (those to which a pointer was found) are not shown.
==16150== To see them, rerun with: --leak-check=full --show-reachable=yes
==16150== 
==16150== For counts of detected and suppressed errors, rerun with: -v
==16150== Use --track-origins=yes to see where uninitialised values come from
==16150== ERROR SUMMARY: 7 errors from 7 contexts (suppressed: 4 from 4)

What does the compare_by_capacity look like? The most common problem with std::sort is that the comparison does not implement a strict weak ordering. This can make the sort run off the end of the vector. — Bo Persson, Oct 06 '11 at 22:03
this is the method: `bool Town::compare_by_capacity(const Town* eins, const Town* zwei) { return (eins->get_cur_capacity() > zwei->get_cur_capacity() ); }` — Sim, Oct 06 '11 at 22:06
@Bo well you were actually right, i miscoded the strict weak ordering — Sim, Oct 25 '11 at 15:17

bdonlan · Accepted Answer · 2011-10-07T00:01:09.187

5

#0  0x000000000040b2a0 in Town::get_cur_capacity (this=0x0) at ./solver/Darstellung.cpp:98

It looks like you're invoking get_cur_capacity on a NULL pointer. You're getting here from a comparator:

#1  0x000000000040b9ab in Town::compare_by_capacity (eins=0x0, zwei=0x0) at ./solver/Darstellung.cpp:135

which is comparing NULL to NULL. You further get here from a sort:

#5  0x000000000040de63 in std::sort<__gnu_cxx::__normal_iterator<Town**, std::vector<Town*> >, bool (*)(Town const*, Town const*)> (__first=..., __last=..., __comp=0x40b98e <Town::compare_by_capacity(Town const*, Town const*)>)
    at /usr/include/c++/4.5/bits/stl_algo.h:5250

This is sorting std::vector<Town*>, and is called from:

#6  0x000000000040ce5a in Solution_Stack::get_towns_by_capacity (this=0x7fffffffe010) at ./solver/Darstellung.cpp:331

Most likely the vector you're sorting contains NULL pointers, and your compare_by_capacity function is not prepared to handle this eventuality. Either make sure the vector has no NULLs, or have compare_by_capacity explicitly check for NULL and do something sensible (eg, sort it before anything other than another NULL).

#1  0x000000000040b9e9 in Town::compare_by_index (eins=0x40, zwei=0x73b4d0) at ./solver/Darstellung.cpp:139

This 0x40 looks like you have either uninitialized memory or corrupted memory. How are you resizing this vector?

It's really quite difficult to try to diagnose this without actually seeing any of your code.

edited Oct 07 '11 at 00:01

answered Oct 06 '11 at 20:41

bdonlan

224,562
31
268
324

well, the method to copy is used nearly 2k times before the crash, and this makes it kind of hard to check on it as there are no if-clause or anything which could cause any of this. This is the reason why i suggest exhausted memory in the first place. – Sim Oct 06 '11 at 20:51
@Sim, nope, exhausted memory would have caused a `std::bad_alloc` exception at the point of allocation, assuming you're using `new` and not `malloc`. You'll need to figure out when exactly you started putting NULLs into this vector. One useful technique would be to just write a function that checks if there are any nulls in the vector, and if so crashes right then and there (using `assert` or `abort()`) - and then run this _all over the place_. Try to make it crash closer to the problem's source this way. – bdonlan Oct 06 '11 at 21:02
@Sim: That doesn't change the fact that somehow you're getting NULLs in your vector. Double check places where the vector increases in size. – Mooing Duck Oct 06 '11 at 21:06
@bdonlan: by exhausted memory he meant stack space. – Mooing Duck Oct 06 '11 at 21:07
a checkfunction just caused it to crash about 300 rounds earlier with the edited dump in the question. – Sim Oct 06 '11 at 22:04
@MooingDuck, this would cause SIGSEGV or SIGBUS, not NULLs. – bdonlan Oct 07 '11 at 00:00
I accept this answer as this is basically right, my fault was the incorrect use of std::sort which then got inconsistent. – Sim Nov 28 '11 at 17:27

score 2 · Answer 2 · answered Oct 06 '11 at 20:44

2

If your code is compiled with debugging symbols and without optimization, then we can probably believe the Town::get_cur_capacity (this=0x0) line which means you got a null pointer in your vector somewhere. The code that put it in may have run at a totally different point in time.

If nulls are actually intended to be allowed in your vector then your sort predicate compare_by_capacity needs to be prepared to handle that eventuality.

In this case you may need to review the code that populates your vector, and valgrind may help you track down if there are memory problems in your code.

answered Oct 06 '11 at 20:44

Mark B

95,107
10
109
188

there are no nullpointers expected at all, as the objects which are put into, are always the same, just copied and slightly adjusted. And this happens nearly 2k times before the crash. – Sim Oct 06 '11 at 20:54
Well, check your expectations, e.g. iterate through your data and probe for NULL pointers. As mentioned earlier buffer overflows can cause the same symptoms so go over everywhere you might trash some memory (memset calls, strcpy, reading data etc.) – nos Oct 06 '11 at 21:16
well the funny thing is, that i avoided this on all cost, there are just two function allowed to alter the vector and just a few others are allowed to sort it. So the amount of code, which can have an effect on this is quite small, and i double checked several times. And as the std::sort function seems to appear in all my dumps i get somehow suspicious, is there any slight possibility that i pass bad arguments to it? – Sim Oct 06 '11 at 23:21
There is a very high probability that you are passing incorrect arguments to `std::sort` - a much higher probability than that there are bugs in `std::sort`. – Jonathan Leffler Oct 07 '11 at 20:51

SIGSEGV within std::sort, how to narrow it down

2 Answers2

Linked