1

We met with segfault dump often when the traffic comes in very high. Not sure what's the problem here now?

Anyone has the experience to share with us?

(gdb) bt

#0 ... in tcmalloc::ThreadCache::ReleaseToCentralCache(tcmalloc::ThreadCache::FreeList*, unsigned long, int) () from /usr/lib64/libtcmalloc.so
#1 ... in tcmalloc::ThreadCache::ListTooLong(tcmalloc::ThreadCache::FreeList*, unsigned  long) () from /usr/lib64/libtcmalloc.so
#2 ... in tc_delete () from /usr/lib64/libtcmalloc.so
#3 ... in boost::detail::sp_counted_impl_p<OutputObject>::dispose() () at /opt/rh/devtoolset-2/root/usr/include/c++/4.8.2/bits/unique_ptr.h:67

Thanks,

Steve Lorimer
  • 27,059
  • 17
  • 118
  • 213
wenxzhen
  • 75
  • 1
  • 2
  • 12

1 Answers1

3

Most likely you have memory corruption in your program. Double free, for instance. You're saying that segfault is coming when load is high, so it's most probably race condition.

We had this problem when we used thread-unsafe object concurrently (via Singleton pattern) and thus having various problems with memory threating. You will probably find out where your problem is by going up in stacktrace and examining every frame for such bugs. It worked for me - that segfault was caused by std::string destructor, where std::string was part of thread-unsafe object structure.

prez
  • 603
  • 5
  • 7
  • as you have mentioned the "going up the stackstrace", do you have any hints/tips on how to do that? The question is not related with the original issue. – wenxzhen Dec 28 '15 at 10:24
  • If there is one, I don't know. I can just suggest you to analyze each frame. Like, if you see string destructor in trace - then you need to go upper and look where this string comes from, who owns this string, what scope is this string of, and so on. Eventually you should be able to stalk into some thread-unsafe object or code. – prez Dec 29 '15 at 10:53