当前问题
目前有一个日志分析程序，随着数据的膨胀，工具的运行性能越来越差，甚至运行过程中出现申请不到内存的情况。在任务管理器里可以看到，还有足够的内存，可是却申请不到。由于程序在运行过程中，有许多次new/delete的操作，据此推测是有太多的内存碎片，导致无法申请到足够大的连续内存。
基础知识
new/delete做了什么？有哪些开销
slab
锁-查找空闲-找到合适大小连续的-分配-解锁。delete相反
内存碎片
Imagine that you have a “large” (32 bytes) expanse of free memory:
|_|
Now, allocate some of it (5 allocations):
|aaaabbccccccddeeee______________|
Now, free the first four allocations but not the fifth:
|__eeee_______________|
Now, try to allocate 16 bytes. Oops, I can’t, even though there’s nearly double that much free.
On systems with virtual memory, fragmentation is less of a problem than you might think, because large allocations only need to be contiguous in virtual address space, not in physical address space. So in my example, if I had virtual memory with a page size of 2 bytes then I could make my 16 byte allocation with no problem. Physical memory would look like this:
|ffffffffffffffeeeeff____________|
whereas virtual memory (being much bigger) could look like this:
____…
|_eeeeffffffffffffffff
____…
The classic symptom of memory fragmentation is that you try to allocate a large block and you can’t, even though you appear to have enough memory free. Another possible consequence is the inability of the process to release memory back to the OS (because there’s some object still in use in all the blocks it has allocated from the OS, even though those blocks are now mostly unused).
Tactics to prevent memory fragmentation in C++ work by allocating objects from different areas according to their size and/or their expected lifetime. So if you’re going to create a lot of objects and destroy them all together later, allocate them from a memory pool. Any other allocations you do in between them won’t be from the pool, hence won’t be located in between them in memory, so memory will not be fragmented as a result.
Generally you don’t need to worry about it much, unless your program is long-running and does a lot of allocation and freeing. It’s when you have mixtures of short-lived and long-lived objects that you’re most at risk, but even then malloc will do its best to help. Basically, ignore it until your program has allocation failures or unexpectedly causes the system to run low on memory (catch this in testing, for preference!).
The standard libraries are no worse than anything else that allocates memory, and standard containers all have an Alloc template parameter which you could use to fine-tune their allocation strategy if absolutely necessary.
本博客欢迎转发,但请保留原作者信息
github:codejuan
博客地址:http://blog.decbug.com/