CCCryptorCreateWithMode ignores padding parameter

Question

I'm trying to use CommonCrypto for crypting with AES-CTR and PKCS7 padding.

I am aware that CTR does not need padding in order to work correctly, there are uses for padding beyond that. Quoting RFC 3686:

For this reason, AES-CTR does not require the plaintext to be padded to a multiple of the block size. However, to provide limited traffic flow confidentiality, padding MAY be included, as specified in ESP.

Section 2.4 of the referenced document is relevant, and lists several uses of padding.

Therefore, I figure CommonCrypto should be able to apply padding to any cipher (mode). The code seems straight forward, breaking down to something like this:

CCCryptorRef cryptor = nil;
NSData* input = [NSMutableData dataWithLength:3];
NSData* key = [NSMutableData dataWithLength:32];
NSMutableData* output = [NSMutableData dataWithLength:32];

CCCryptorCreateWithMode(
        kCCEncrypt, kCCModeCTR, kCCAlgorithmAES, ccPKCS7Padding,
        nil,
        key.bytes, key.length,
        nil, 0, 0,
        kCCModeOptionCTR_BE,
        &cryptor
);

size_t written = 0;
CCCryptorUpdate(
        cryptor,
        input.bytes, input.length,
        output.mutableBytes, output.length,
        &written
);

size_t writtenF = 0;
CCCryptorFinal(
    cryptor, 
    output.mutableBytes + written, output.length - written, 
    &writtenF);

CCCryptorRelease(cryptor);

output.length = written + writtenF;

NSLog(@"Expected: 16 bytes");
NSLog(@"Actual: %i bytes", output.length);

The output is:

Expected: 16 bytes
Actual: 3 bytes

If you check the status codes, you'll find there there are no errors.

I can decrypt the output just fine okay, so encryption itself seems to work just fine. But clearly no padding happens.

I can not debug into the implementations, so I have no idea what's going wrong. Am I using the options incorrectly?

Nota bene: I can make CCCrypt apply padding with kCCOptionPKCS7Padding, but then I don't see a way to select CTR mode.

It would be nice if the downvoters could indicate as to how I can improve the question. Silent downvotes don't help anybody. — Raphael, Jul 12 '17 at 17:49
Nota bene: Common Crypto does not support padding with CTR mode. If you feel it should file a [bug report](https://bugreport.apple.com/web/). Otherwise non mi important. — zaph, Jul 12 '17 at 18:35
@zaph A piece of documentation stating that *would* be an answer. — Raphael, Jul 12 '17 at 19:04
@zaph "Padding beyond that required for the algorithm or alignment reasons cited above, may be used to conceal the actual length of the payload, in support of (partial) traffic flow confidentiality" — Raphael, Jul 12 '17 at 21:26
@zaph That said, I tire of this. You have no idea what my purpose with this is, and it's not necessary to answer the question, either. If I wanted to discuss whether padding is necessary, I'd be over on [security.SE]. This is about how to make CommonCrypto do what I want, if possible. If you don't want to address this, why waste both of our times? — Raphael, Jul 12 '17 at 21:28
You are well aware that Common Crypto does not support padding with AES CTR mode so there is really no question, the answer is already known. If you need "padding" you can add it to the plain-text data as you are aware. When a spec states "MAY be included" it is optional. — zaph, Jul 12 '17 at 21:35
"You are well aware that Common Crypto does not support padding with AES CTR mode" -- I was most certainly not when I wrote the question; I thought I was doing something wrong. You keep insisting this is the case, and I tend to believe you. However, I'd like to see definite proof. Is there matrix of which ciphers are supported with which feature, or something similar? If I haven't overlooked something, the code doc in headers and source does *not* document the behaviour seen here. — Raphael, Jul 12 '17 at 21:38
Common Crypto is poorly documented so what you see is what you get. Common Crypto is lacking many things including CTS and GCM mode and I have been informed directly by Apple in writing: "At this point we do not have a plan to further develop CommonCrypto". I also know directly that Apple is aware of the short comings such as lack of GCM mode and more secure encryption on a system thread. So my guess is that Apple is working on something else for iOS, possible a port of the macOS transform encryption library, perhaps we will see that when iOS 11 is finally released. — zaph, Jul 12 '17 at 22:51
Nota bene (funny I would have expected Deutsch): Most of the crypto libraries are very short on documentation including missing attributes, default attributes and confusing API naming. PKCS#5 for AES instead of PKCS#7, "256" variously meaning Rijndael block size (mcrypt) or key size. Many examples by major distribution showing DES and ECB mode that should have updates a decade ago. Given that, while Common Crypto is bad so is most of the documentation. That is to be expected when the developers also write the documentation. — zaph, Jul 12 '17 at 23:00
@zaph Thanks for your help. It's unfortunate that the documentation sucks as much as it does, and that there's no roadmap for crypto libraries from Apple. (Good libraries exist, they could just wrap one up as a framework...) "That is to be expected when the developers also write the documentation." -- you state that as an absolute truth, which makes me said. Imho, a developer that can't write documentation another developer can comprehend can not be a *good* developer. I'd have thought that Apple had good people. Maybe they work on other things. — Raphael, Jul 17 '17 at 08:49
(FWIW, I still think this is a fair question, especially due to the lack of proper documentation. I hope future googlers are saved of wasting as much time on this as I've had to.) — Raphael, Jul 17 '17 at 08:51

score 2 · Accepted Answer · edited Oct 07 '21 at 11:04

Quoting RFC 3686:

For this reason, AES-CTR does not require the plaintext to be padded to a multiple of the block size. However, to provide limited traffic flow confidentiality, padding MAY be included, as specified in ESP.

This is a different use of padding. It is not PKCS#7 padding used in block ciphers. Its is RFC 2406 padding placed in the ESP packet behind the payload data (see below).

Therefore, I figure CommonCrypto should be able to apply padding to any cipher (mode)...

The confusion has taken you down a rabbit hole. It does not intersect with CommonCrypto like you are thinking. Stop now :)

From page 2 of RFC 2406:

2.  Encapsulating Security Payload Packet Format

   The protocol header (IPv4, IPv6, or Extension) immediately preceding
   the ESP header will contain the value 50 in its Protocol (IPv4) or
   Next Header (IPv6, Extension) field [STD-2].

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ----
|               Security Parameters Index (SPI)                 | ^Auth.
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov-
|                      Sequence Number                          | |erage
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | ----
|                    Payload Data* (variable)                   | |   ^
~                                                               ~ |   |
|                                                               | |Conf.
+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |Cov-
|               |     Padding (0-255 bytes)                     | |erage*
+-+-+-+-+-+-+-+-+               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |   |
|                               |  Pad Length   | Next Header   | v   v
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ ------
|                 Authentication Data (variable)                |
~                                                               ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

"This is a different use of padding." -- I see, thanks for clarifying that. (However, looking and the schematic you quote, applying PKCS#7 padding to the payload (without caring about what lower levels do) would serve pretty much the same purpose, wouldn't it?) — Raphael, Jul 17 '17 at 08:45
"It does not intersect with CommonCrypto like you are thinking." -- Thanks for confirming. It's unfortunate that apparently all documentation there is about this aspect of CommonCrypto is hearsay. — Raphael, Jul 17 '17 at 08:46
@Raphael - Yeah, Apple docs are frustrating at times. MSDN is not much better. Its hard to find relevant information there, too. — jww, Jul 17 '17 at 12:32

Raphael · Answer 2 · 2017-07-12T18:24:31.663

^{This is an attempt at an answer, ultimately unsuccessful, in the hopes that someone else can use the pointers and make more sense of the situation.}

In the implementation of ccSetupCryptor (which gets called by CCCryptorCreateWithMode) we find:

case ccPKCS7Padding:
        if(mode == kCCModeCBC)
            ref->padptr = &ccpkcs7_pad;
        else
            ref->padptr = &ccpkcs7_ecb_pad;
        break;

This seems to suggest that PKCS7 is only available for CBC and ECB.

But the padding should do something. Digging deeper, ccpkcs7_ecb_pad is defined in corecryptoSymmetricBridge:

const cc2CCPaddingDescriptor ccpkcs7_ecb_pad = {
    .encrypt_pad = ccpkcs7_encrypt_ecb_pad,
    .decrypt_pad = ccpkcs7_decrypt_ecb_pad,
    .padlen = ccpkcs7_padlen,
    .padreserve = ccpkcs7_reserve,
};

ccpkcs7_encrypt_ecb_pad leads to an actual encryption routine, which I'm sure is not executed in CTR mode.

Tracking ccpkcs7_padlen leads me to this:

static size_t ccctr_mode_get_block_size(const corecryptoMode modeObject) { 
    return modeObject.ctr->block_size; 
}

Now, this corecryptoMode is defined in corecryptoSymmetricBridge but I can not find the definitions of block_size there, or anywhere else.

It should be 128bit, as for all AES modes, but then maybe it's set to zero or one since CTR can deal with arbitrary-length inputs? That would explain why zero padding bytes get added, but I can not confirm that that is what happens.

*"This is an attempt at an answer, ultimately unsuccessful..."* - You should probably delete this answer. I suspect its mostly going to confuse other users and attract downvotes. — jww, Jul 14 '17 at 20:55

CCCryptorCreateWithMode ignores padding parameter

2 Answers2