Cleaning object pool based on execution time

Question

Problem

My current project implements a pool of objects to avoid constant memory allocation and de-allocation for speed purposes, the object pool is a std::vector<object> and I would like to implement a form of garbage collection to reduce memory usage and increase performance. Every loop the program iterates over the entire vector and if the object is active, executes the update function, this means that if my vector is full of inactive objects, I will be wasting a lot of time iterating over them and also memory storing them. I cannot clean the vector every frame as this would crush performance.

Current attempt

My current attempt at implementing this has been to measure the update time, and use a pre-defined function to determine whether or not we are spending too much time on the update, for the amount of objects currently active, if we are, I clean the vector once to allow the speed to return to normal.

#include <chrono>

void updateObjects()
{
  auto begin = std::chrono::high_resolution_clock::now();
  //update all objects
  for(auto o : objectVec)
  {
    //only update active objects
    if(o.m_alive)
    {
      o.update();
    }
  }
  //end time of update
  auto end = std::chrono::high_resolution_clock::now();
  //calculate time taken vs estimated time
  auto elapsed = (end-begin).count();
  //Estimate is based on performance testing
  long estimate = 25*m_particleCount+650000;

  //If we have no active objects, 
  //but are wasting memory on storing them, we clean up
  //If the update takes longer than it should, we clean up
  if(((objectCount <= 0) && (objectVec.size() > 0)) || (elapsed > estimate))
  {
    cleanVec(); //remove inactive objects
  }
}

This solution works well on my pc, however I am having issues with it on other computers as the time taken for the update to complete varies due to different CPU speeds, and my pre-defined function then doesn't work as it is based off of incorrect data. I am wondering if there is another measurement I can use for this calculation? Is there a way I can measure the pure amount of instructions executed, as this would be the same across computers, some would simply execute them faster? Any other suggestions are welcome, thank you!

Edit

The object pool can be as large as 100,000 objects, typical usage will range from 3000 to the maximum. My function for cleaning is:

objectVec.erase(std::remove_if(objectVec.begin(),
                               objectVec.end(),
                               [](const object& o) {return !(o->m_alive);}),
                               objectVec.end());

Answer 1

Typically, an object pool uses aging to determine when to expel an individual pool object.

One way to do that is to keep an iteration counter within your update loop. This counter would be incremented every loop. Each object could keep a "last active" time (the loop count the last time the object was active). Inactive objects would be moved to the end of the array, and when old enough would be expelled (destroyed). A separate index of the last active object would be kept, so looping for updates could stop when this was encountered.

If it isn't possible to store an active time in an object, you can still move the inactive objects to the end of the active list, and if there are too many inactive objects pare the list down some.

Answer 2

You should specify how expensive your cleanup is, and how large your pool is, they affect how cleaning up should be implemented.

To make it clear, the pool is effectively an allocator of one type of object and performance gains are completely independent of how individual objects are cleaned up.

I cannot clean the vector every frame as this would crush performance.

If the performance of inactive objects' cleanup dominates, there is nothing you can do in your pool algorithm.

Therefore I assume this is due to std::vector semantics where removing inactive objects involves relocation of other objects. It is then logical to ask do you really need std::vector's properties?

• Do you need your objects to be contiguous in memory?
• Do you need O(1) random access?
• Do you loop over the pool constantly?
• Otherwise, is the pool small? As long as it is small std::vector is fine.

If the answer is no, then just use stuff like std::deque and std::list, where removal is O(1) and you can cleanup on every frame.

Otherwise garbage collection of your pool can be done as

• keeping a counter of frames since last updated for each object, remove if counter is over a threshold
• keeping a counter of total inactive objects, clean pool if percentage of active objects is over a threshold