Under Linux, is it possible to gcore a process whose executable has been deleted?

Question

Programming on CentOS 6.6, I deleted an executable (whoops, make clean) while it was running in a screen session.

Now, unrelated, I want to gcore the process to debug something. I have rebuilt the executable, but gcore doesn't accept the replaced file. It knows the original file was deleted and won't let me dump core.

# gcore 15659
core.YGsoec:4: Error in sourced command file:
/home/dev/bin/daemon/destinyd (deleted): No such file or directory.
gcore: failed to create core.15659

# ls -l /proc/15659/exe
lrwxrwxrwx. 1 root root 0 Mar 12 21:33 /proc/15659/exe -> /home/dev/bin/daemon/destinyd (deleted)

# ln -s /proc/15659/exe /home/dev/bin/daemon/destinyd
ln: creating symbolic link `/home/dev/bin/daemon/destinyd': File exists

# rm /proc/15659/exe
rm: remove symbolic link `/proc/15659/exe'? y
rm: cannot remove `/proc/15659/exe': Permission denied

FreeBSD's gcore has an optional argument "executable" which looks promising (as if I could specify a binary to use that is not /proc/15659/exe), but that's of no use to me as Linux's gcore does not have any such argument.

Are there any workarounds? Or will I just have to restart the process (using the recreated executable) and wait for the bug I'm tracking to reproduce itself?

http://stackoverflow.com/q/9974254/560648 looks similar but it's about attaching GDB (where you can specify executable path) rather than gcore. And, besides, none of the answers seem to match my observation that even `/proc//exe` looks broken by the deletion. — Lightness Races in Orbit, Mar 12 '15 at 22:08
I can't test this right now, but iirc you can `gdb -p /proc//exe` and run the `generate-core-file` gdb command — nos, Mar 12 '15 at 22:09
Playing around with this, I figured out that once the file was deleted, the kernel literally linked the `exe` symlink to `/file/path/file-name (deleted)`. Renaming the file to `file-name (deleted)` changed the `file` output for `exe` from "broken symbolic link" to just "symbolic link". I wasn't able to successfully run gcore even then, though. — jplatte, Mar 12 '15 at 22:10

score 4 · Accepted Answer · edited May 23 '17 at 11:59

Despite the output of ls -l /proc/15659/exe, the original executable is in fact still available through that path.

So, not only was I able to restore the original file with a simple cp (though this was not enough to restore the link and get gcore to work), but I was able to attach GDB to the process using this path as executable:

# gdb -p 15659 /proc/15659/exe

and then run the "generate-core-file" command, followed by "detach".

Then, I became free to examine the core file as needed:

# gdb /proc/15659/exe core.15659

In truth I had forgotten about the ability of GDB to generate core files, plus I was anxious about actually attaching GDB to the process because timing was quite important: generating the core file at precisely the right time to catch that bug.

But nos steered me back onto this path and, my fears apparently unfounded, GDB was able to produce a lovely core.15659 for me.

Under Linux, is it possible to gcore a process whose executable has been deleted?

1 Answers1