15

I'm a recent convert to C++ for game programming - I have a lot of experience dealing with memory management and garbage collection woes in C#, but not as much with C++.

I've heard some vague advice in the past to avoid allocations and deallocations during gameplay (i.e. new and delete) and to pre-allocate everything you might need up front. But that's a lot more tedious and architecturally complex than just allocating and freeing game objects as needed while the game's running (enemies, particles, etc.).

I think the advice I read was referring to resource-constrained platforms - I'm aiming to develop mostly for PC, and I imagine the game state data that would be changing frequently would be on the order of a few megabytes at most. The rest are textures, sound assets, etc. that I'll be preloading.

So my question is: in a world of PCs with gigabytes of memory, is it worth the headache to set up elaborate memory pooling, pre-allocation, and so forth for my game state data? Or is this just some unquestioned "best practice" tradition that evolved when maxing out limited platforms, that is now repeated as gospel?

If my 2 MB of game data gets fragmented and is now spread over 4MB, I can't imagine that mattering in the slightest on a PC made after 1990 - but curious to know if I'm missing something :).

QuadrupleA
  • 876
  • 6
  • 23
  • 6
    It's not fragmentation that's the issue, it's just the cost of reaching out to the OS to get memory. It's just expensive. This is not mitigated by the amount of memory you have. – GManNickG Apr 09 '13 at 05:45
  • `I'm a recent convert to C++ for game programming` I pity your being ;) Just kidding of course. – Mike de Klerk Apr 09 '13 at 05:48
  • 1
    Memory fragmentation might occur if you allocate and free up memory fast and frequently. Especially when using different sizes of memory to allocate. But on a OS like Windows or Linux on a CPU that can handle some load, and with quiet some memory onboard I would worry about it unless you notice performance issues. I think its more of a challenge not to cause memory leaks! – Mike de Klerk Apr 09 '13 at 05:51
  • 2
    @MikedeKlerk: Been having an easier time with C++ than the "easy" C#, surprisingly :) – QuadrupleA Apr 09 '13 at 06:05
  • 1
    @QuadrupleA -"Been having an easier time with C++..." - That is because you have more freedom and power with C++ :) – SChepurin Apr 09 '13 at 06:13

2 Answers2

23

The main reasons to avoid calling new unless necessary in a game environment are that

  1. Dynamic allocation of memory really is surprisingly expensive.
  2. Cache misses are detrimental to performance.

Dynamic Allocation

At my work, we develop a game-like product (virtual surgery) and most of our memory is pre-allocated and handled via factories or memory pools. This is done because, dynamically allocating memory takes a very long time. The system has to deal with memory requests of many different sizes at any and all times. This means there's a lot of work going into processes such as minimizing fragmentation. If you ask the system for memory, you'll have to wait for it to do these things. If you pre-allocate memory, you can use factories or other block-size-specific memory managers to alleviate these concerns.

I can tell you from experience that a simple mistake like allocating a reasonably large std::vector from scratch every frame, instead of reusing pre-allocated memory, can drag the frame rate down into the gutter.

Cache Misses

Another related issue is cache coherence. Cache misses, which force the OS to bring a new page into the cache, are also very expensive. If this happens often, you'll have an unplayable game. If, however, you pre-allocate large chunks of memory, this can go a long way towards improving cache locality, which makes cache misses few and far between.

Moral of the story

So, in short: If you don't manage your own pre-allocated memory, you can expect a lot of your computational time to be lost to waiting for the system to allocate memory or handle cache misses.

Agentlien
  • 4,996
  • 1
  • 16
  • 27
  • Thanks for the explanation and details. In my situation the allocations/deletions would be gradual over time as the game progresses, rather than anything per-frame and super-frequent. I'll probably take the "don't optimize up-front" tack, and worry about pre-allocations if memory operations become a bottleneck. – QuadrupleA Apr 09 '13 at 06:10
  • @QuadrupleA That sounds really reasonable. It also depends very much on the scope of the game. If you have intricate collision detection, dynamics or constraint handling, that's a prime suspect for giving you these sort of headaches. – Agentlien Apr 09 '13 at 06:13
  • @QuadrupleA: While all of this is true, a good dynamic memory allocator can help a lot as well. I would recommend looking into the 'hoard' memory allocator. Especially if your program is multi-threaded. – Omnifarious Apr 09 '13 at 06:18
  • this answer is great. id like to also add i do the same thing but also just overload the new and del operators so that it uses the memory pool so it can work in any project without alot of hassles. comes in handy alot when picking up a dead project. great informative answer. – Zanven Sep 12 '13 at 07:19
  • 1
    @Zanven That is also how I usually do it. I find it a great solution, and I got the idea from Bjarne's "Design and Evolution of C++". – Agentlien Sep 16 '13 at 10:02
3

in a world of PCs with gigabytes of memory

And only a few megabytes for your process if a sane OS is running on that PC. (But that's not the primary issue here anyway.)

is it worth the headache to set up elaborate memory pooling, pre-allocation, and so forth for my game state data

I don't dare saying that "always preallocate everything", because sometimes it's just not possible, and for less frequently used objects, it might not worth the effort, but it is certainly true that dynamic memory management in C and C++ is resource-intensive and slow. So if you render, for example 30 frames a second to the screen, then possibly avoid allocating and de-allocating and then re-allocating the buffer for the objects to be rendered during each frame/iteration.

  • 1
    Interesting - do you know the ballpark in milliseconds that an OS would take to allocate an additional, say, 1K? What I'm envisioning would be for things like items & enemies that would accumulate or free up gradually as the game progresses, rather than anything per-frame (I use pooled constructs for that kind of stuff). – QuadrupleA Apr 09 '13 at 05:54
  • 1
    @QuadrupleA Don't expect one single piece of exact numerical data. "The OS" - what OS? What conditions? How many concurrent threads/processes running? Exact specs of the hardware? And so forth... If you are allocating memory for new enemies on the fly, that should be fine, provided that there's no zerg rush when in each second, 100 new character appears on stage. –  Apr 09 '13 at 05:56
  • Right - hard to quantify exactly - I guess I'll take the simple & direct approach first, and worry about pooling and pre-allocation later if it becomes a bottleneck. – QuadrupleA Apr 09 '13 at 06:01