How to generate PKCS#7 signature from digest?

Question

I want to sign the pdf using pdf digest. I have created the hash using below code,

byte[] buffer = new byte[1024];
int numOfBytesRead =0;
MessageDigest md = null;
md = MessageDigest.getInstance("SHA256","BC");
while((numOfBytesRead = content.read(buffer)) != -1 ){
     md.update(buffer, 0, numOfBytesRead);
}
byte[] digest = md.digest();

At the end I need to attach this signature to my pdf. I have found one solution Create pkcs7 signature from file digest, but the algorithm used in the link is SHA256withRSA. My privatekey is genearted using EC algorithm and I need to use SHA256withECDSA.Is it possible to just sign the Hash using SHA256withECDSA and attach the signature to the pdf using PDFBox ExternalSigning Interface.

Have you tried using the code in the answer to the question you reference, merely changing the signature algorithms? What was the result? — mkl, Oct 11 '18 at 11:13
@Mkl-I haven't tried. Since I am new to cryptography,I don't understand what is happening in that code snippet. Do you have any reference link for cryptography ? Can you please let me know what should I change to make it SHA256withECDSA.? — Android_dep, Oct 11 '18 at 14:36

mkl · Answer 1 · 2018-10-15T15:19:21.323

There are several situations in which Adobe calls a signer's certificate invalid even though apparently it is valid; in the case at hand in particular:

Key usage or Extended key usage values not appropriate
PAdES signature misses an ESS signing-certificate-v2 attribute

Key usage or Extended key usage values not appropriate

This is based on the information the OP first published as an answer

I tried below code still the pdf says Signature is invalid. Can you please check the code,

[...]

I have attached the pdf . Pdf file created

Indeed, Adobe Reader says the signature is invalid, but look more closely:

It says "Document has not been modified since this signature was applied" - This means that the signature is mathematically correct!

The issue is that the "Signer's certificate is invalid", and the reason for this can be seen when digging into the signature property dialogues:

Thus, the problem is that your signer certificate is Not valid for usage.

This is due to the highlighted attribute, while the Key Usage Digital Signature is ok, the "Extended key usage" 1.3.6.1.5.5.8.2.2 (OID for IPSEC Protection) is not!

According to the Adobe Digital Signatures Guide for IT, Adobe Acrobat accepts only

one or more of the following Key usage values (if any)
- nonRepudiation
- signTransaction (11.0.09 only)
- digitalSignature (11.0.10 and later)
and one or more of the following Extended key usage values (if any)
- emailProtection
- codeSigning
- anyExtendedKeyUsage
- 1.2.840.113583.1.1.5 (Adobe Authentic Documents Trust)

Due to its IPSEC Protection extended key usage value, therefore, your certificate is not considered valid for signing PDF documents.

This probably only is an issue in legacy ISO 32000-1 signatures, probably not in PAdES signatures.

PAdES signature misses an ESS signing-certificate-v2 attribute

This is based on the information the OP first published as an answer

I have created 2 pdf files, PDFA is signed using the digest of the pdf content with below code,

[...]

PDFA

PDFB is created with the same private key and certificate, but instead of digest I am using pdf document content directly which gives me valid signed pdf, PDFB code below,

[...]

PDFB

I think something is missing in the signing part of PDFA which I couldn't figure out.

Here the main difference is not whether you explicitly calculate the hash yourself or allow it to be calculated implicitly, the main difference is that the signature in PDFB includes an ESS signing-certificate-v2 attribute while the one in PDFA does not. This attribute is generated between

//PAdES - PDF Advanced Electronic Signature

and

//PAdES-end

As the comments already hint, this is only necessary for PAdES signatures, not for legacy ISO 32000-1 ones. The answer the OP took his original code from referred to creating a legacy ISO 32000-1 signature (and, therefore, works alright) while the OP creates a PAdES signature.

The presence of an ESS signing certificate attribute is required by the PAdES specification ETSI EN 319 142-1:

e) Generators shall use either the signing certificate or the signing-certificate v2 attribute, depending on the hash function, in accordance with ETSI EN 319 122-1.

(ETSI EN 319 142-1, section 6.3 PAdES baseline signatures)

It references the CAdES specification ETSI EN 319 122-1 which in turn requires

h) Requirement for SPO: ESS signing-certificate. The ESS signing-certificate attribute shall be used if the SHA-1 hash algorithm is used.

i) Requirement for SPO: ESS signing-certificate-v2. The ESS signing-certificate-v2 attribute shall be used when another hash algorithms than SHA-1 is used.

(ETSI EN 319 122-1, section 6.3 Requirements on components and services)

what is "signTransaction"? I can't find it: https://tools.ietf.org/html/rfc5280#section-4.2.1.3 — Tilman Hausherr, Oct 14 '18 at 13:19
I'll have to look it up. I merely quoted the above from the a Adobe site. ;) — mkl, Oct 14 '18 at 14:02
The latest PDF has "1.3.6.1.4.1.311.10.3.12", seems that Adobe tolerates this as well. — Tilman Hausherr, Oct 14 '18 at 15:24
Yes. I meanwhile assume the main problem here actually is the absence of the `ESSCertIDv2` in the non-working cases. The examples are PAdES signed after all. — mkl, Oct 14 '18 at 22:15
Yes, problem is the absence ESSCertIDv2. This will answer https://stackoverflow.com/questions/52588527/how-to-sign-pdf-hash-using-pdfbox as well. Thanks mkl and Tilman for the support. — Android_dep, Oct 15 '18 at 12:56
@TilmanHausherr *"The latest PDF has "1.3.6.1.4.1.311.10.3.12", seems that Adobe tolerates this as well."* - That's the Microsoft "Signer of documents" OID, see [here](https://support.microsoft.com/en-us/help/287547/object-ids-associated-with-microsoft-cryptography). Apparently Adobe meanwhile also accepts this Microsoft OID (according to its description it's appropriate); or it tests entirely differently for PAdES signatures... — mkl, Oct 15 '18 at 15:45
@Android_dep If this sufficiently answers your question, please accept the answer. — mkl, Oct 31 '18 at 17:54

Android_dep · Answer 2 · 2018-10-12T09:44:40.320

I tried below code still the pdf says Signature is invalid. Can you please check the code,

System.out.println("Hash Signing started");
    List<Certificate> certList = getFormatCertificate(strCertificate);
    PrivateKey privateKey;
    CMSSignedData s = null;
    Security.addProvider(new BouncyCastleProvider());
    byte[] signature = null;
    try {
        privateKey = loadPrivateKey(strPrivatekey);
        byte[] buffer = new byte[1024];
        int numOfBytesRead =0;
        MessageDigest md = null;
        md = MessageDigest.getInstance("SHA256","BC");
        while((numOfBytesRead = content.read(buffer)) != -1 ){
            md.update(buffer, 0, numOfBytesRead);
        }
        byte[] digest = md.digest();

        // Separate signature container creation step
        JcaCertStore certs = new JcaCertStore(certList);

        CMSSignedDataGenerator gen = new CMSSignedDataGenerator();

        Attribute attr = new Attribute(CMSAttributes.messageDigest,
                new DERSet(new DEROctetString(digest)));

        ASN1EncodableVector v = new ASN1EncodableVector();

        v.add(attr);

        SignerInfoGeneratorBuilder builder = new SignerInfoGeneratorBuilder(new BcDigestCalculatorProvider())
                .setSignedAttributeGenerator(new DefaultSignedAttributeTableGenerator(new AttributeTable(v)));

        //AlgorithmIdentifier sha256withECDSA = new DefaultSignatureAlgorithmIdentifierFinder().find(signerAlgorithm);

        CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
        InputStream in = new ByteArrayInputStream(certList.get(certList.size()-1).getEncoded());
        X509Certificate cert = (X509Certificate) certFactory.generateCertificate(in);

        gen.addSignerInfoGenerator(builder.build(
                new JcaContentSignerBuilder(signerAlgorithm).build(privateKey),
                new JcaX509CertificateHolder(cert)));
        //DErse
//      gen.addSignerInfoGenerator(builder.build(
//              new JcaContentSignerBuilder(sha256withRSA,
//                      new DefaultDigestAlgorithmIdentifierFinder().find(sha256withRSA))
//                              .build(PrivateKeyFactory.createKey(privateKey.getEncoded())),
//              new JcaX509CertificateHolder(cert)));

        gen.addCertificates(certs);

        s = gen.generate(new CMSAbsentContent(), false);
        System.out.println("Hash sign completed");
        signature = s.getEncoded();// ConstructEcdsaSigValue(s.getEncoded());
    } catch (GeneralSecurityException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("GeneralSecurityException ::"+e.toString());
    } catch (OperatorCreationException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("OperatorCreationException ::"+e.toString());
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("IOException ::"+e.toString());
    } catch (CMSException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("CMSException ::"+e.toString());
    }finally{
        return signature;
    }

I have attached the pdf . Pdf file created

@Mkl/Tilman : I have created 2 pdf files, PDFA is signed using the digest of the pdf content with below code,

System.out.println("Hash Signing started");
    List<Certificate> certList = getFormatCertificate(strCertificate);
    PrivateKey privateKey;
    CMSSignedData s = null;
    Security.addProvider(new BouncyCastleProvider());
    byte[] signature = null;
    try {
        privateKey = loadPrivateKey(strPrivatekey);

        /*byte[] buffer = new byte[1024];
        int numOfBytesRead =0;
        MessageDigest md = null;
        //md = MessageDigest.getInstance("SHA256","BC");
        md = MessageDigest.getInstance("SHA-256");
        while((numOfBytesRead = content.read(buffer)) != -1 ){
            md.update(buffer, 0, numOfBytesRead);
        }
        byte[] digest = md.digest();*/
        MessageDigest md = MessageDigest.getInstance("SHA256", "BC");
        byte[] digest = md.digest(IOUtils.toByteArray(content));

        // Separate signature container creation step
        JcaCertStore certs = new JcaCertStore(certList);

        CMSSignedDataGenerator gen = new CMSSignedDataGenerator();

        Attribute attr = new Attribute(CMSAttributes.messageDigest,
                new DERSet(new DEROctetString(digest)));

        ASN1EncodableVector v = new ASN1EncodableVector();

        v.add(attr);

        SignerInfoGeneratorBuilder builder = new SignerInfoGeneratorBuilder(new BcDigestCalculatorProvider())
                .setSignedAttributeGenerator(new DefaultSignedAttributeTableGenerator(new AttributeTable(v)));

        //AlgorithmIdentifier sha256withECDSA = new DefaultSignatureAlgorithmIdentifierFinder().find(signerAlgorithm);

        CertificateFactory certFactory = CertificateFactory.getInstance("X.509");
        InputStream in = new ByteArrayInputStream(certList.get(certList.size()-1).getEncoded());
        X509Certificate cert = (X509Certificate) certFactory.generateCertificate(in);

        gen.addSignerInfoGenerator(builder.build(
                new JcaContentSignerBuilder(signerAlgorithm).build(privateKey),
                new JcaX509CertificateHolder(cert)));
        //DErse
//      gen.addSignerInfoGenerator(builder.build(
//              new JcaContentSignerBuilder(sha256withRSA,
//                      new DefaultDigestAlgorithmIdentifierFinder().find(sha256withRSA))
//                              .build(PrivateKeyFactory.createKey(privateKey.getEncoded())),
//              new JcaX509CertificateHolder(cert)));

        gen.addCertificates(certs);

        s = gen.generate(new CMSAbsentContent(), false);
        System.out.println("Hash sign completed");
        signature = s.getEncoded();// ConstructEcdsaSigValue(s.getEncoded());
    } catch (GeneralSecurityException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("GeneralSecurityException ::"+e.toString());
    } catch (OperatorCreationException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("OperatorCreationException ::"+e.toString());
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("IOException ::"+e.toString());
    } catch (CMSException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
        System.out.println("CMSException ::"+e.toString());
    }finally{
        return signature;
    }

PDFA

PDFB is created with the same private key and certificate, but instead of digest I am using pdf document content directly which gives me valid signed pdf, PDFB code below,

SignatureInterface signatureInterface = new SignatureInterface() {
                    @SuppressWarnings("rawtypes")
                    @Override
                    public byte[] sign(InputStream content) throws IOException {
                        try {
                            byte[] certificateByte = null;

                            Store certs = new JcaCertStore(certificates);

                            //PAdES - PDF Advanced Electronic Signature
                            //ESS - Enhanced Security Services
                            //ASN1 - Abstract Syntax Notation One-standard interface description language for defining data structures that can be serialized and deserialized in a cross-platform way
                            // Generating certificate hash
                            MessageDigest md = MessageDigest.getInstance("SHA-256");
                            md.update(certificates.get(certificates.size()-1).getEncoded());
                            byte[] certHash = md.digest();
                            // Generating certificate hash ends
                            System.out.println("Cert hash generated");
                            //ESSCertIDv2 identifies the certificate from the hash
                            ESSCertIDv2 essCert1 =
                                    new ESSCertIDv2(new AlgorithmIdentifier(NISTObjectIdentifiers.id_sha256), certHash);
                            ESSCertIDv2[] essCert1Arr =
                                    {
                                            essCert1
                                    };
                            SigningCertificateV2 scv2 = new SigningCertificateV2(essCert1Arr);
                            Attribute certHAttribute =
                                    new Attribute(PKCSObjectIdentifiers.id_aa_signingCertificateV2, new DERSet(scv2));
                            ASN1EncodableVector v = new ASN1EncodableVector();
                            v.add(certHAttribute);

                            AttributeTable at = new AttributeTable(v);

                            //Create a standard attribute table from the passed in parameters - certhash
                            CMSAttributeTableGenerator attrGen = new DefaultSignedAttributeTableGenerator(at){
                                protected Hashtable createStandardAttributeTable(Map parameters)
                                {
                                    Hashtable result = super.createStandardAttributeTable(parameters);
                                    result.remove(CMSAttributes.signingTime);
                                    return result;
                                }
                            };
                            //PAdES-end
                            System.out.println("CMSAttributeTableGenerator generated");
                            SignerInfoGeneratorBuilder genBuild =
                                    new SignerInfoGeneratorBuilder(new BcDigestCalculatorProvider());
                            genBuild.setSignedAttributeGenerator(attrGen);
                            //Get single certificate
                            org.spongycastle.asn1.x509.Certificate certas1 = org.spongycastle.asn1.x509.Certificate.getInstance(ASN1Primitive.fromByteArray(certificates.get(certificates.size()-1).getEncoded()));
                            // ContentSigner interface creates SHA256withECDSA signer using PvtKey
                            ContentSigner sha1Signer = new JcaContentSignerBuilder(signerAlgorithm).build(privateKey);
                            //Creates SignerInfoGenerator using X.509 cert and ContentSigner
                            SignerInfoGenerator sifGen = genBuild.build(sha1Signer, new X509CertificateHolder(certas1));

                            // CMSSignedDataGenerator generates a pkcs7-signature message 
                            CMSSignedDataGenerator gen = new CMSSignedDataGenerator();

                            gen.addCertificates(certs);
                            gen.addSignerInfoGenerator(sifGen);
                            //Creates CMS message from PDF
                            CMSProcessableInputStream msg = new CMSProcessableInputStream(content);
                            //Generate a CMS Signed Data object which can be carrying a detached CMS signature
                            //msg - content to be signed
                            CMSSignedData signedData = gen.generate(msg, false);
                            System.out.println("CMSSignedData is done");
                            return signedData.getEncoded();
                        } catch (GeneralSecurityException e) {
                            throw new IOException(e);
                        } catch (CMSException e) {
                            throw new IOException(e);
                        } catch (OperatorCreationException e) {
                            throw new IOException(e);
                        }
                }

            };
            System.out.println("CMSSignedData is done2");
            PDDocument pdDocument =  PDDocument.load(inputfile);

            System.out.println("pdDocument loaded");
            pdDocument.addSignature(signature, signatureInterface);

PDFB

I think something is missing in the signing part of PDFA which I couldn't figure out.

Thanx for providing more information but please add it to your question instead of putting it into an answer. There is an [edit](https://stackoverflow.com/posts/52757037/edit) link right underneath your question. — mkl, Oct 11 '18 at 16:10
*"still the pdf says Signature is invalid"* - yes, but look at the output more closely: It says "The document has not been modified since this signature was applied." This means that the signature is mathematically correct! The issue is that "The signer's identity is invalid." — mkl, Oct 11 '18 at 16:14
Concerning your edit: *"but instead of digest I am using pdf document content directly which gives me valid signed pdf"* - there are other differences, too, and the difference you mention is not relevant here. — mkl, Oct 12 '18 at 11:54
I have not had the time to test but the main difference I see is that your second piece of code adds an `ESSCertIDv2` signed attribute while your first one doesn't. As you create PAdES signatures, an ESS attribute may well be considered necessary by Adobe. — mkl, Oct 12 '18 at 14:31

How to generate PKCS#7 signature from digest?

2 Answers2

Key usage or Extended key usage values not appropriate

PAdES signature misses an ESS signing-certificate-v2 attribute