14

I'm baffled. I have a VM running Ubuntu 14.04. I've followed procedures here: http://clang.llvm.org/docs/LibASTMatchersTutorial.html and am at the step to run ninja. This builds llvm and clang. Now, my VM is no slouch, I gave it 6GB of RAM and 4 CPUs and a 20GB swap file. The biggest problem comes at link time - it seems to start a large number of ld processes, each using at least 3-4GB or virtual memory, and at some point a lot of CPU each. But the the swap file grew to over 12GB and the processes are all IO bound, but I don't know if they are doing something useful, or thrashing. All I know is the disk is getting hammered and the jobs run forever. I've actually just dropped the CPU count to the VM to 1, to see if it might be more efficient with less parallelism, as I surmised the issue may be thrashing.

I suppose my disk could be slow... Any ideas? Should I be using make instead of ninja? My expertise is not Linux (although I'm getting there :-) ) So I'm following the tutorial but perhaps it is not recommended the "best" way to build the clang / llvm programs.

Brian B
  • 1,509
  • 3
  • 20
  • 29

1 Answers1

15

I have been there, It's happening with the latest svn release (but not if you get clang 3.8 or older releases). What is happening is that since during development a lot of debug information is also being generated for each compilation unit the file sizes are becoming big.

The solution is to turn off all the debug info that's been attached by default. You probably are not going to debug clang, so won't need it. SO instead of just doing this 

cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON

What you should do is 

cmake -G Ninja ../llvm -DLLVM_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release

All the other steps remain the same. Now I have not tested this with ninja, but have verified it with make on ubuntu (this tutorial, I modified the same thing in step 7). This should owkr as weel.

Nishant Sharma
  • 683
  • 9
  • 16
  • I've not tried it yet, but this makes a lot of sense. I'll mark as answered. Meanwhile, I discovered you can download prebuilt images of all the code for Ubuntu 14.04 (and many other platforms) here: http://llvm.org/releases/download.html. This link (http://askubuntu.com/questions/306929/how-to-install-clang-using-precompiled-binaries)has details on how to get and install it, although I put it in my home folder instead of /usr/local. So I can just delete or get a different version easily. – Brian B Jul 09 '16 at 17:28
  • Yes, you can totally do that. For me, I had to get the latest release and build the latest from svn truck because of a bug fix that was done after releasing clang 3.8. and it was critical for my work with clang. – Nishant Sharma Jul 09 '16 at 19:10
  • 1
    Great! Solved my problem on clang 9. The funny point is, whatever sort of informations it was generating, was being done only via 1 thread!! So my CPU util was roughly 25%. All I could do was staring at monitor for around two hours even though I have 2xSSD in zfs strip config. Anyway thanks for the tip – Alireza Mohamadi Feb 01 '20 at 20:24