0

I am currently interested in user-level memory allocator and TCmalloc from google.

I saw this page that shows some evaluation to comparing PTmalloc2 and TCmalloc.

I want to rebuild that experiment in my environment.

So, I installed TCmalloc from an official reference from here.

and also install golang for using profiling tool that called 'pprof' to this command

sudo snap install --classic go

then, install google-pprof tools

go install github.com/google/pprof@latest

above command get from here

finally, I just tested TCmalloc with Tensorflow MNIST python code using LD_PRELOAD

   LD_PRELOAD=/usr/lib/libtcmalloc.so.4 python3 MNIST.py

It works.

And Generated heap profile binary(.heap)

LD_PRELOAD=/usr/lib/libtcmallo.so.4 HEAPPROFILE=log python3 MNIST.py

It works too.

generated .heap file

Analyzing by using google-pprof also works.

And tried CPUPROFILE with under code

LD_PRELOAD=/usr/lib/libtcmallo.so.4 CPUPROFILE=cpu python3 MNIST.py

It didn't work. There is no output.

What am I missing?

-my system-

Ubuntu 20.04.1LTS x86-64 5.13.0-35

gcc/g++: 9.4.0

CPU: Intel i9-12900KF

MEMORY: 32GB

Thanks for your attention.

I’m looking forward to any reply.

Js Choi
  • 1
  • 1

1 Answers1

0

I found what I missed from This site!

The executable file must link by the -lprofiler option.

Then, I used LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtprofiler.so for hooking

a shared library without build.

Entire command

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc.so:/usr/lib/x86_64-linux-gnu/libprofiler.so CPUPROFILE=tcmalloc_python.prof python3 MNIST.py

(You must separate with :(colon) when using multiple libraries)

If you meet "ERROR: ld.so: object 'libpath' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored" message, that must be don't exist library file in that path.

So, just reinstall pprof another way.

try

sudo apt install google-pprof-dev

or

sudo apt-get install golang-github-google-pprof-dev

and then, retry the above command(LD_PRELOAD~). It will work.

Referenced from here.

In my case, it works.

After training is done, some .prof file is generated!

result

But, pprof didn't work at python code(only c/c++ executable file)

I'll find another way. (Any idea?)

pprof result

So, I found some test code from here and build an executable file.

Then tried pprof, It works! (pprof manual)

pprof result (success)

Js Choi
  • 1
  • 1