0

I am working with a program somebody else created. I have quite some experience with coding, but not all that much with coding in C++, so I'm doing a lot "learning by doing" here. So the program seemed to be stable, I started my work on it, which mostly included doing small modifications in small parts of the program. Recently I did some performance optimization, which also seemed to be stable, but 2 days ago I changed something and kept getting crashes. So I reverted my changes and still got the crashes. I started using Application verifier and Global flags with activated Page heap and everything heap-related checked to find out what was causing those problems. So since then the debugger always crashed with a "std::bad_alloc" error. Since I was using SVN I also checked the very first version of the code that I ever got - and I also get the bad_alloc-crash there.

Now my question: Can I be absolutely sure, that this bad_alloc crash with Application Verifier enabled is an indicator for a bug inside the program? When using Application verifier, the program itself uses a lot of memory, around 1-1.1gb, but never any more. Total system memory is used up to 80-90% at most, so I don't think there is an actual allocation problem caused by too little free space. What do you think?

MrWayne
  • 131
  • 1
  • 7
  • 1
    `std::bad_alloc` can also be caused by passing `NULL` to an `std::string` (and several other potential causes). It is not solely raised when exhausting memory. – Borealid Feb 22 '13 at 17:52
  • Can you show us any code for this? – slugonamission Feb 22 '13 at 17:52
  • 1
    @Borealid: `std::bad_alloc can also be caused by passing NULL to an std::string` : According to http://www.cplusplus.com, if the pointer is NULL, then the effect it causes undefined behavior. Reading the standards, I see for the string constructor: "Requires: s shall not be a null pointer", which smells of "undefined behaviour", to me... – paercebal Feb 22 '13 at 18:15
  • 1
    Application Verifier modifies the heap so that when pages are deallocated, they are specially marked and are not able to be reused. For applications with large memory requirements, or lots of allocation/deallocation, Application Verifier will usually cause the application to exhaust memory in a manner that is not necessarily indicative of a problem in the application itself. – Chad Feb 22 '13 at 18:36
  • There are almost no strings involved at all, so it's VERY unlikely that this causes the issue. And it's not really possible to show you any code, because it's just way too much - and since the crash is caused at some apparently random position, there isn't any code that would help you. @Chad: Thank you - I thought there might be some issue like that. Indeed the program has to do many allocations/deallocations. That is in fact what I was working on to speed it up, memory pooling. – MrWayne Feb 23 '13 at 00:45

3 Answers3

2

No, you cannot be absolutely sure about anything in an arbitrary C++ program, because your program could contain undefined behaviour (in fact, it almost certainly does, although it may not be relevant to the issue at hand). That said, the usual cause of std::bad_alloc is an inability to allocate memory.

Use std::set_new_handler to set a custom new_handler, and put a breakpoint in it. If the breakpoint is triggered, then your issue is almost certainly an inability to allocate memory (and the state of the program at this point could be useful for debugging your issue).

Mankarse
  • 39,818
  • 11
  • 97
  • 141
1

By default, 32-bit processes on Windows for x86 are limited to 2GB of address space (the lower half of the full address space).

If your program performs a lot of allocation and deallocation, or if your program requires large contiguous allocations, then yes, it's quite possible that when your program is only using 1.1GB of working set there will be insufficient contiguous address space to service allocations.

(I worked on a large project a few years ago that was tightly constrained by address space. It would be common for us to "run out of memory" with a 1.2 to 1.4 GB working set.)

The Page Heap definitely makes this problem more acute since most allocations are much, much larger than they ordinarily would be.

James McNellis
  • 348,265
  • 75
  • 913
  • 977
  • Ok, generally the program needs about 200MB of memory - is it correct, that IF all allocations/deallocations are done correctly AND page heap is deactivated there shouldn't be a problem if there is no need for single large allocations? (If I get it right, frequent allocations/deallocations cause allocations "all over the place", which means that there might not be enough space in between to allocated chunks to allocate a single big chunk if needed - correct?) – MrWayne Feb 23 '13 at 01:01
  • @MrWayne I'd strongly suggest profiling your software with vmmap (http://technet.microsoft.com/en-us/sysinternals/dd535533.aspx). It allows you to see very precisely what is going on w/rt how much free memory you *really* have. It has been a godsend on certain large projects that were tightly constrained by address space. – Nathan Monteleone Mar 13 '13 at 13:38
0

You should study the stack received when the binary crashed with std::bad_alloc.

Usually, this should be because new was unable to allocate memory. With the stack, you should be able to know how much memory was asked, and if it means strange to you (e.g. "allocate 3Go, please!"), then you know where your bug is.

It is not clear when reading your question, but if what you're meaning is that the process uses up to 80%-90% of the available virtual memory, then perhaps your memory is fragmented, and you're trying to allocate an object too big to fit in the remaining free little memory chunks... Thus the bad alloc, despite your process still having memory to play with.

Doing a search in your code to see if some part of your code manually throws bad_alloc would be a good idea, too.

paercebal
  • 81,378
  • 38
  • 130
  • 159
  • Regarding the used memory: The program normally uses about 200MB of RAM (I don't really know a lot about virtual and non-virtual memory, 200MB is the amound shown in the task manager), with App Verifier enabled it's about 1,1GB and with all other programs together the task manager says that 80-90% physical memory are being used. How can I check the call stack at the moment of the crash? I'm using Eclipse, and all it does is telling me that there was a bad alloc exception causing a crash. It's not a breakpoint! – MrWayne Feb 23 '13 at 00:59
  • You can usually produce a core dump on any platform. Google for it for your platform, get the core dump (on Windows, it's usually some kind of ".dmp" file, on Linux, it's more like a "core" file), and "open it" with your debugger. – paercebal Feb 23 '13 at 07:39