2

I know that the CrtAllocator is the C runtime allocator and uses malloc/realloc/free, and I know that the MemoryPoolAllocator is the default allocator and allocates memory sequentially.

I don't understand why the MemoryPoolAllocator is considered more efficient than the CrtAllocator. Does the MemoryPoolAllocator just default allocate a large block of memory and just add entries into it each time Parse() is called? If so, does that mean the CrtAllocator calls malloc for every new Parse()?

The docs indicate this for the CrtAllocator: When there is a lot of add and remove operations, this allocator may be preferred. But this allocator is far less efficient than MemoryPoolAllocator. but it does not explain why it is better for a lot of adds/remove vs the MemoryPoolAllocator or why/how it is far less efficient than MemoryPoolAllocator.

The primary use case I care about is quickly parsing JSON responses from the Web. My plan is to just allocate a large buffer and reuse it for every response. I think I can just use ParseInsitu() and clear the buffer every time.

Does it matter which Allocator I use for this use-case?

ajoseps
  • 1,871
  • 1
  • 16
  • 29

1 Answers1

1

If you're simply grabbing a huge block of memory once and implementing your own memory pooling, just use the crtAllocator. Otherwise use the MemoryPoolAllocator which effectively borrows memory from the system and subdivides it as and when required.

Using a memory pool means that when a memory grab request is made, you keep the code running in user space, so there is no context-switching to kernel/protected mode and there is no need to possibly wait for mutexes for the kernel's memory allocator. (Unless it needs to borrow more memory to satisfy a memory grab request, but this won't be often).

The caveat is that the program would appear to be "using" more memory than it would otherwise need to; you're effectively bolting on a massive cache. Also you may need to pay attention to how the blocks are managed by the pooler if fragmentation becomes an issue. Managers such as Delphi's FastMM provides different areas for small, medium and large blocks to mitigate this.

In most cases any downsides of a Memory Pool shouldn't be a problem though; use everything in moderation ;)

Check out the Wikipedia article: https://en.wikipedia.org/wiki/Memory_pool

In your case you may wish to just use CrtAllocator to make an initial memory grab, and then free it and grab a bigger block if it needs to grow. You might then consider freeing that at a later stage if the system memory becomes constrained. You may even want to consider keeping a small block and a large block so that the small block is always resident but you could let the large block come and go.

This keeps the memory footprint low, but allows you to also handle large requests, but also deny large requests if memory becomes constrained, whilst keeping the small block such that you don't end up with a situation of having no memory to grab at all.

Den-Jason
  • 2,395
  • 1
  • 22
  • 17
  • Isn’t reusing a large buffer the same as memory pooling in this sense? – ajoseps Oct 15 '20 at 23:22
  • yes - hence the line "If you're simply grabbing a huge block of memory once and implementing your own memory pooling.....". You're just not subdividing it if you keep reusing it. You might need to grow it if and when necessary. – Den-Jason Oct 15 '20 at 23:22
  • Yes that makes sense. So technically, I could just use the memory pool allocator and reserve a large amount, and clear it when needed, and that would be functionally equivalent to me allocating a large buffer and passing it in, and also clearing it when needed. Additionally, why would the CrtAllocator be more efficient for frequent add/removes? – ajoseps Oct 15 '20 at 23:29
  • The CrtAllocator would not be efficient for frequent add/removes; it's only efficient if you hold on to the memory and reuse it. Using the CrtAllocator would simply allow you to keep a small memory footprint, and also handle constrained memory scenarios better. See my edit. – Den-Jason Oct 15 '20 at 23:30
  • I’m quoting from the rapidjson documentation for the frequent add/remove efficiency comment: http://rapidjson.org/md_doc_dom.html – ajoseps Oct 15 '20 at 23:32
  • 1
    Yep, that's the same as what I'm saying; the pooled allocator is more efficient, but it holds on to memory. If the memory becomes constrained (due to there being a lot of add/removes) then you have to use memory more sparingly - hence the CrtAllocator. – Den-Jason Oct 15 '20 at 23:35
  • 1
    Aaah got it. I was thinking efficient in terms of performance and not in terms of memory usage – ajoseps Oct 15 '20 at 23:36
  • @ajoseps This is why people write books, blogs, perform presentations, run courses etc. Because in most situations, manuals suck ;) – Den-Jason Oct 15 '20 at 23:56