Don't use Eclipse to build the release version of your app. Use Ant, Maven, or any other build tool that is much more flexible than Eclipse, doesn't need a GUI, can be scripted and used by a continuous integration server.
All of these tools should easily be used to include your pre-compilation and post-compilation tasks in the build process.
That said, you could just use a public static final boolean constant FOR_TEST, and include all your testing code in
if (TestUtil.FOR_TEST) {
}
You would then have just a single place to change in the code to have all the test code removed from the compiled version. No need for a complex Java program to do that.
Or you could let all the testing code in the released version, and activate it when testing using a system property, for example. This wouldn't even need any precompilation/postcompilation phase, and would probably have a negligible cost.