After digging around in the WSSecEngine class a little bit in more depth, I found that I needed not a CallBackHandler, but a custom Processor that skips the processing of the UsernameToken object:
WSSecurityEngine secEngine = new WSSecurityEngine();
WSSConfig wsConfig = WSSConfig.newInstance();
wsConfig.setProcessor(UsernameToken.TOKEN, new Processor() {
@Override
public void handleToken(Element arg0, Crypto arg1, Crypto arg2, CallbackHandler arg3, WSDocInfo arg4, Vector arg5, WSSConfig arg6)
throws WSSecurityException {
// skip the token processing
logger.debug("Skipping processing of the username token");
}
@Override
public String getId() {
return null;
}
});
secEngine.setWssConfig(wsConfig);
Although WSS4J 1.5 is incredibly old, hopefully this may hope someone else in the future.
WSS4J 1.6+ has changed the way these processors work with Validators instead.