It is not safe to modify an executable that is running. As per What happens when you overwrite a memory-mapped executable?
Under Linux, if you replace an executable while it is running, the
results are unpredictable and it may crash.
If you delete the file and compile a new version of the program then what will happen is very well defined. The already running instance will use the previous code, and that will be held in memory by the operating system until the program terminates. Any new instances will use the new code.
The summary: You should make sure your build system deletes the old executable before recompiling, and so long as that is true then the recompile will not take effect until you rerun the program, otherwise the behaviour is undefined (read SIGSEGV).
Appendix of sorts:
JamesKanze rightly pointed out that the linker itself may delete the file before writing its output, if this is the case then it will always behave as if you'd deleted the file yourself before recompiling (the sane scenario). Looking at bfd/cache.c from the binutils cvs head:
/* Create the file.
Some operating systems won't let us overwrite a running
binary. For them, we want to unlink the file first.
However, gcc 2.95 will create temporary files using
O_EXCL and tight permissions to prevent other users from
substituting other .o files during the compilation. gcc
will then tell the assembler to use the newly created
file as an output file. If we unlink the file here, we
open a brief window when another user could still
substitute a file.
So we unlink the output file if and only if it has
non-zero size. */
So at least with GNU LD this is guaranteed to be fine. This does not necessarily extend to other linkers, however.