2

I just upgraded puppet to version to 3.2.1, and when running the agent, I get an error saying "Illegal instruction (core dumped)". Here is some details:

Both master and agent is run on the same machine

#cat /etc/redhat-release
CentOS release 6.2 (Final)

#uname -r
3.9.3-x86_64-linode33

#ruby --version
ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux]

RPM installed is "puppet-3.2.1-1.el6.noarch.rpm"

However when I run puppet agent with the same version on another host, everything goes fine. Problem is only while running agent on the same machine as the master. Any idea what is going on ?

Tom O'Connor
  • 27,480
  • 10
  • 73
  • 148
aqs
  • 163
  • 1
  • 6
  • Can you strace the puppet agent run command? – Tom O'Connor May 28 '13 at 08:12
  • Here is the tail of the strace log: http://pastebin.com/brYBLJgc . – aqs May 28 '13 at 08:42
  • Are you sure that the ruby, ruby-lib, nokogiri, etc files haven't been corrupted on your hd? Can you run a `rpm --verify` on your installed ruby, and nokogiri packages? Type `rpm -qa | grep ruby` to find the full package names. – robbyt May 28 '13 at 16:33
  • I ran that, and I dont see any issues with that. Packages seem to be fine – aqs May 28 '13 at 20:52

2 Answers2

1

The solution was to upgrade glibc. See https://bugzilla.redhat.com/show_bug.cgi?id=752122 for details

aqs
  • 163
  • 1
  • 6
0

If your Ruby installation and its dependent libraries are unmodified, there's a good chance that you have a library in your library search path that conflicts with the version Ruby was compiled against. This is really common if, for example, you've built a library like zlib or openssl and dropped it into /usr/local/lib instead of putting it in a custom prefix and setting an RPATH/LD_LIBRARY_PATH for the programs you need compiled against it.

Generically, the best way to diagnose a segfault is to get the kernel to dump a core file (see: ulimit and the core_pattern sysctl), then run it through a debugger like GDB. It can give you a backtrace that, if nothing else, can pretty readily point you at exactly which library's function call is to blame.

jgoldschrafe
  • 4,395
  • 18
  • 18
  • I ran the puppet agent in GDB. This is what the log says: `Program received signal SIGILL, Illegal instruction. 0x0000003864214c30 in _dl_x86_64_save_sse () from /lib64/ld-linux-x86-64.so.2` – aqs May 30 '13 at 08:12
  • Can you update your question with the full backtrace? – jgoldschrafe May 30 '13 at 12:01
  • `http://pastebin.com/Rp9rfJF6` gives the backtrace – aqs Jun 19 '13 at 10:35