Validate digital signature with a self-signed certificate

Question

I have a question regarding validation of digital signatures using a self-signed certificate:

The following tutorial works for me: http://www.oracle.com/technetwork/articles/javase/dig-signature-api-140772.html

However, when a X.509 certificate is self-signed, how can a receiver trust certificate data attached to an XML message? Any one can generate a self-signed cert and claim to be the same sender. The validation in the above tutorial always returns true. Sender’s cert must be loaded to receiver’s truststore, so receiver can use whatever in the truststore to validate signed doc. I cannot find any reference for such a scenario.

Answer 1

Your understanding is correct - with self-signed certificates anyone can create a certificate and signature validation will be ok. The reason is that signature validation performs first of all cryptographic operation, which is completed successfully. The second step is to validate the certificate itself AND also it's origins. When the CA-signed certificate is used, the certificate is validated using CA certificate(s) up to trusted CA (or known root CA). With self-signed certificate validation is not possible. In the above tutorial the procedure of certificate validation was skipped for simplicity as it's quite complex and beyond the scope of tutorial.

Answer 2

The problem you're describing is usually addressed by Public Key Infrastructures (PKI).

This is the traditional model for verifying certificates for HTTPS sites, for example. It starts with a set of trusted Certification Authorities (CAs) from which you import the CA certificates as "trusted". The entity certificates that you get are then verified against this set of trusted anchors by building a certification path between the certificate to verify and a CA certificate you know (linking the certificate to a trusted issuer, perhaps via intermediate CA certificates).

The various rules to do this are described in RFC 5280. The PKI system doesn't apply only to web servers, but to any entity (there are additional rules for web servers to verify that they're the one you want to talk to, on top of having a valid certificate).

(In particular because the choice of which CA certificates to trust is often done on behalf of the user, at least by default, by the OS or browser vendor, this model isn't perfect, but it's the most common in use.)

Alternatively, there's nothing wrong with establishing a list of self-signed certificates you would trust in advance.

Either way, you need to pre-set what you trust by mechanisms out of bands (e.g. by meeting someone you trust and using the certificate they give you in person).

This PKI model goes hand-in-hand with the X.509 format thanks to the notion of Issuer DN and Subject DN. You could have other models, for example relying on PGP certificates, where you would build a web of trust; you would still need an initial set of trusted anchors.

For XML-DSig in Java, you should implement a X509KeySelector that only returns a key that you trust. In a simple scenario, where you have a pre-defined set of self-signed certificates you trust, you can iterate over a keystore containing those trusted certificates. Otherwise, use the Java PKI Programmer Guide (as linked from the tutorial you've used).