I'm using bcc-memleak to locate memory leak in my project. But the top command shows that the VIRT increase from 100GB to 200GB and RES from 60GB to 100GB in 10-hours, while memleak shows that only 150MB memory was not released.
I compiled project with -fno-omit-frame-pointer
to keep full frame pointer and I got almost every call-stack with outstanding allocations.(-o 3000 means just collect memory older than 30s)
memleak command is memleak -p {pid} -o 30000 -T 10 20
(when replacing malloc with jemalloc, add -O /lib64/libjemalloc.so
)
use std malloc and jemalloc have the same result.
Anyone know how to get more information? or other tools recommend to check memleak.
I have tried jemalloc with prof, but it needs too much cpu and memory, so it didn't solve my problem. Also tried LeakSanitizer, but the result is still only less than 100MB memory leak.
bcc & memleak project: https://github.com/iovisor/bcc/blob/master/tools/memleak.py
UPDATE 1:
CPPFLAGS=-O2 -Werror=return-type -Werror=uninitialized -fno-omit-frame-pointer -std=c++11 -mavx2 -mf16c
LIBADD=-lhiredis -lworkflow -lopenblas -lpugixml -lboost_regex -lboost_system -discovery -lmemcached -loffdb -lz -llzo2 -lprotobuf -lresolv -lpthread -lfmt -lACE
environment:
- memleak v0.24.0
- gcc (GCC) 8.3.1 20191121 (Red Hat 8.3.1-5)
- CentOS Linux 8.2
- kernel: Linux 4.18.0-193.28.1.el8_2.x86_64