PDF digital signature verification fails on government document

Question

We're trying to validate the digital signature of a Dutch government agency (UWV Verzekeringsbericht) including the authenticity of the file.
Adobe Acrobat Reader is able to validate this file correctly.

With a small proof of concept application we're able to verify the authenticity of various kind of digitally signed PDFs:

import com.itextpdf.text.pdf.AcroFields;
import com.itextpdf.text.pdf.PdfReader;
import com.itextpdf.text.pdf.security.PdfPKCS7;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

import java.io.IOException;
import java.security.GeneralSecurityException;
import java.security.Security;
import java.util.ArrayList;

public class Verifier {

    public static void main(String[] args) throws IOException, GeneralSecurityException {

        BouncyCastleProvider provider = new BouncyCastleProvider();
        Security.addProvider(provider);

        new Verifier().run(args[0]);
    }

    private void run(String path) throws IOException, GeneralSecurityException {

        final PdfReader reader = new PdfReader(path);

        final AcroFields fields = reader.getAcroFields();

        final ArrayList<String> signatureNames = fields.getSignatureNames();

        for(String signatureName: signatureNames) {
            System.out.println("Verify signature " + signatureName);
            verifySignature(fields, signatureName);
        }
    }

    private PdfPKCS7 verifySignature(final AcroFields fields, final String name) throws GeneralSecurityException {
        System.out.println("Signature covers whole document: " + fields.signatureCoversWholeDocument(name));
        System.out.println("Document revision: " + fields.getRevision(name) + " of " + fields.getTotalRevisions());
        PdfPKCS7 pkcs7 = fields.verifySignature(name);
        System.out.println("Integrity check OK? " + pkcs7.verify());
        return pkcs7;
    }
}

Using these (Maven) dependencies:

<dependencies>
    <dependency>
        <groupId>com.itextpdf</groupId>
        <artifactId>itextpdf</artifactId>
        <version>5.5.13</version>
    </dependency>
    <dependency>
        <groupId>org.bouncycastle</groupId>
        <artifactId>bcprov-debug-jdk15on</artifactId>
        <version>1.60</version>
    </dependency>
    <dependency>
        <groupId>org.bouncycastle</groupId>
        <artifactId>bcpkix-jdk15on</artifactId>
        <version>1.60</version>
    </dependency>
</dependencies>

As you can guess, validating PDFs from this authority doesn't work.

The result of running this application is:

Exception in thread "main" java.lang.IllegalArgumentException: can't decode PKCS7SignedData object
    at com.itextpdf.text.pdf.security.PdfPKCS7.<init>(PdfPKCS7.java:214)

This is caused within the PdfPKCS7 class which is instantiating a ASN1 input stream from the signature's content (line 203):

SN1InputStream din = new ASN1InputStream(new ByteArrayInputStream(contentsKey));

Which in turn results in a IOException: DER length more than 4 bytes: 31 So the signature seems invalid.

The AcroFields' verifySignature method call tries to create a PdfPKCS7 instance. A snippet from this method:

if(!reader.isEncrypted()){
    pk = new PdfPKCS7(contents.getOriginalBytes(), sub, provider);
}else{
    pk = new PdfPKCS7(contents.getBytes(),sub,provider);
}

For some reason iTextPDF concludes the PDF is encrypted and uses the getBytes variant for the signature verification. However, the PDF is not encrypted (as far as I know), so it should use the getOriginalBytes.

When I force using this original contents, while debugging, the verification succeeds!

So it seems like a bug within iTextPDF, maybe caused by an unusual combination of factors in the pdf.

Some details from the PDF's certificate:

Version: 3
Signature algorithm: SHA256 RSA
Key usage: Digital Signature, Encrypt Keys
Public Key: RSA (2048 bits)

Unfortunately I can't share the concerning PDF because it contains personal information. As a Dutch citizen you can download your own version from UWV, see these instructions.

Any help or suggestion is appreciated.

Answer 1

The background of this issue appears to be an information missing in the ISO 32000-1 PDF specification; iText 5.5 meanwhile supports the ISO 32000-1 verbatim interpretation.

In ISO 32000-2 this has meanwhile been clarified.

---

The missing information

Before PDF became an ISO standard, PDF processor implementers followed the lead of Adobe Acrobat when documentation on PDFs were unclear or even claiming otherwise.

When Adobe Acrobat encrypted and signed a PDF, the binary string containing the signature container is not encrypted. Thus, other PDF tools in this case also did not encrypt the signature container.

In 2008 PDF became an ISO standard. According to ISO 32000-1,

Encryption applies to all strings and streams in the document's PDF file, with the following exceptions:

• The values for the ID entry in the trailer
• Any strings in an Encrypt dictionary
• Any strings that are inside streams such as content streams and compressed object streams, which themselves are encrypted

(ISO 32000-1, section 7.6 - Encryption)

According to this, in an encrypted and signed PDF the binary string containing the embedded signature container would also be encrypted.

In 2017, part 2 of ISO 32000 was published. In it the enumeration above is extended by a new entry

• Any hexadecimal strings representing the value of the Contents key in a Signature dictionary

(ISO 32000-2, section 7.6 - Encryption)

According to this, in an encrypted and signed PDF the binary string containing the embedded signature container would not be encrypted.

The code retrieving the signature container in iText

In the earliest code in iText for the retrieval of signature containers I could find, the binary string containing the signature container is assumed to never be encrypted:

pk = new PdfPKCS7(contents.getOriginalBytes(), provider);

(commit ffc70db dated November 5th, 2004, commented as "paulo version 139")

The method getOriginalBytes retrieves the bytes of a PDF string as they are in the PDF, no decryption applied ever.

Later on the code was moved two or three times without change.

When PAdES support was added, only the subfilter was added here, still the original bytes were used:

pk = new PdfPKCS7(contents.getOriginalBytes(), sub, provider);

(commit 691281c, dated August 31st, 2012, commented as "Verify a CAdES signature")

But in early 2017 it was changed to the code you found:

if(!reader.isEncrypted()){
    pk = new PdfPKCS7(contents.getOriginalBytes(), sub, provider);
}else{
    pk = new PdfPKCS7(contents.getBytes(),sub,provider);
}

(commit 0b852d7 dated February 9th, 2017, commented as "Handle encrypted content stream when verifying Signatures SUP-1783")

Apparently the support issue SUP-1783 triggered the switch to following a verbatim interpretation of ISO 32000-1.

In iText 7 we have

pk = new PdfPKCS7(PdfEncodings.convertToBytes(contents.getValue(), null), sub, provider);

(commit ae73650, dated October 11th, 2015, commented as "Added classes for support LTV, Ocsp, CRL and TSA.")

but the contents here have before marked unencrypted

contents.markAsUnencryptedObject();

(commit 6dfb206, dated April 24th, 2018, commented as "Avoid exception in SignatureUtil when a read-only document was passed")

and in iText 7 this makes contents.getValue() return the original bytes. So iText 7 supports the PDF 2.0 clarification.

What should be done?

In my opinion, considering the verbatim ISO 32000-1 interpretation, one should accept either encrypted or unencrypted signature containers but in the light of the ISO 32000-2 wording one should generate only unencrypted ones.