2

We have a Foo server(foo.something.org), and Bar client uses a HeartBeat thread to periodically connect() and check if its alive or not. This works fine. But every 2 hours, Bar segfaults while trying to resolve the Foo's hostname.

2 hours = 7200 seconds = The default TCP Keep Alive value. But I do not know how this has anything to do with my situation. I am on Ubuntu 10.04, 32-bit.

The guilty code:

tcp::resolver res( io_service ); //This eventually calls gethostbyname2_()
res.resolve( tcp::resolver::query( foo_hostname, foo_port, resolve_err );

The stack trace:

#0  0xb481907f in ns_samename () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#1  0xb481327c in __res_nameinquery () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#2  0xb48133cc in __res_queriesmatch () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#3  0xb4813d9f in ?? () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#4  0xb48118a3 in __libc_res_nquery () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#5  0xb4811e8b in ?? () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#6  0xb48122b5 in __libc_res_nsearch () from /lib/tls/i686/cmov/libresolv.so.2
No symbol table info available.
#7  0xb4821bd6 in _nss_dns_gethostbyname3_r () from /lib/tls/i686/cmov/libnss_dns.so.2
No symbol table info available.
#8  0xb4821f2b in _nss_dns_gethostbyname2_r () from /lib/tls/i686/cmov/libnss_dns.so.2
No symbol table info available.
#9  0xb74acb0d in gethostbyname2_r () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#10 0xb746e010 in ?? () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#11 0xb746fa65 in getaddrinfo () from /lib/tls/i686/cmov/libc.so.6
No symbol table info available.
#12 0xb649b0da in boost::asio::detail::resolver_service<boost::asio::ip::tcp>::resolve(boost::shared_ptr<void>&, boost::asio::ip::basic_resolver_query<boost::asio::ip::tcp> const&, boost::system::error_code&)
    () from /home/spiderman/lib/pyfoo.so
No symbol table info available.
#13 0xb648c3b5 in boost::asio::ip::resolver_service<boost::asio::ip::tcp>::resolve (this=0x8eb9418) at /usr/local/include/boost/asio/ip/resolver_service.hpp:105
No locals.
#14 boost::asio::ip::basic_resolver<boost::asio::ip::tcp, boost::asio::ip::resolver_service<boost::asio::ip::tcp> >::resolve (this=0x8eb9418) at /usr/local/include/boost/asio/ip/basic_resolver.hpp:122
No locals.
#15 HeartBeatThread::tryConnect (this=0x8eb9418) at foo.cc:557
        err = {m_val = 0, m_cat = 0xb6141098}
        res = {<boost::asio::basic_io_object<boost::asio::ip::resolver_service<boost::asio::ip::tcp> >> = {<boost::noncopyable_::noncopyable> = {<No data fields>}, service = @0x8eb9758, implementation = {
              px = 0x0, pn = {pi_ = 0xb4590530}}}, <No data fields>}
        endpoint_iterator = {<boost::iterator_facade<boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>, boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const, boost::forward_traversal_tag, boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const&, int>> = {<No data fields>}, values_ = {px = 0x6, pn = {pi_ = 0x91a}}, 
          iter_ = {<boost::optional_detail::optional_base<__gnu_cxx::__normal_iterator<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const*, std::vector<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp>, std::allocator<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> > > > >> = {<boost::optional_detail::optional_tag> = {<No data fields>}, m_initialized = 20, m_storage = {dummy_ = {
                  data =  <incomplete sequence \370\252\202\265>, aligner_ = {<No data fields>}}}}, <No data fields>}}
        end = {<boost::iterator_facade<boost::asio::ip::basic_resolver_iterator<boost::asio::ip::tcp>, boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const, boost::forward_traversal_tag, boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const&, int>> = {<No data fields>}, values_ = {px = 0x0, pn = {pi_ = 0xa82aaac}}, 
          iter_ = {<boost::optional_detail::optional_base<__gnu_cxx::__normal_iterator<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> const*, std::vector<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp>, std::allocator<boost::asio::ip::basic_resolver_entry<boost::asio::ip::tcp> > > > >> = {<boost::optional_detail::optional_tag> = {<No data fields>}, m_initialized = false, m_storage = {
                dummy_ = {data = "\001\000\000", aligner_ = {<No data fields>}}}}, <No data fields>}}
#16 0xb648cc99 in HeartBeatThread::handleConnectCompleted (this=0x8eb9418, err=...) at foo.cc:586
No locals.
#17 0xb649015c in boost::asio::detail::handler_queue::handler_wrapper<boost::asio::detail::binder1<boost::_bi::bind_t<void, boost::_mfi::mf1<void, HeartBeatThread, boost::system::error_code const&>, boost::_bi::list2<boost::_bi::value<HeartBeatThread*>, boost::arg<1> (*)()> >, boost::system::error_code> >::do_call(boost::asio::detail::handler_queue::handler*) () from /home/sipderman/lib/pyfoo.so
No symbol table info available.
#18 0xb6499273 in boost::asio::detail::task_io_service<boost::asio::detail::epoll_reactor<false> >::run_one(boost::system::error_code&) () from /home/spiderman/lib/pyfoo.so
No symbol table info available.
Sam Miller
  • 23,808
  • 4
  • 67
  • 87
Kostolma
  • 301
  • 1
  • 4
  • 11

1 Answers1

1

Boost.Asio may be throwing an exception while attempting to resolve the query. You may wish to use the alternate call to resolve (available at least in the current version of boost):

tcp::resolver res( io_service ); //This eventually calls gethostbyname2_()
boost::system::error_code ec;
res.resolve( tcp::resolver::query( foo_hostname, foo_port, resolve_err ), ec );
if ( ec )
{
   // Some kind of error took place here.
   printf( "Error resolving address: %s", ec.message().c_str() );
}

If you do something like that, it might prevent the segfault, and with a little logging, may give you an idea why the resolve is occasionally failing.

fleebness
  • 109
  • 7