How to port OpenSSL-based DES decryption method to CommonCrypto?

Question

I have the following method, which I would like to port to a CommonCrypto-based approach, but I am completely stuck. Any help is highly appreciated.

- (NSString*) decryptCiscoPassword:(NSString*) encPassword
{
    NSUInteger i, k = 0;
    NSData *rawData = nil;
    int rawInt, rawDataLength = [encPassword length] >> 1;
    const char *encString = [encPassword cStringUsingEncoding:NSUTF8StringEncoding];
    const char *tmpString;
    unsigned char *rawDataBytes = (unsigned char*) malloc(rawDataLength*sizeof(unsigned char));

    for(i = 0; i < [encPassword length]; i = i+2)
    {
        tmpString = &encString[i];
        sscanf(tmpString,"%2x",&rawInt);
        rawDataBytes[k++] = (unsigned char) rawInt;
    }
    rawData = [NSData dataWithBytesNoCopy:rawDataBytes length:rawDataLength freeWhenDone:YES];

    /* now let's start the decryption */

    NSString *returnString = nil;

    /* --- calculate 3DES-Key --- */
    DES_cblock desKey1, desKey2, desKey3;

    unsigned char tempHash[20];
    [rawData getBytes:tempHash length:20];
    tempHash[19]++;

    unsigned char hashV1[20];
    CC_SHA1(tempHash,20,hashV1);

    unsigned char hashV2[20];
    tempHash[19] += 2;
    CC_SHA1(tempHash,20,hashV2);

    memcpy(&desKey1, hashV1, 8);
    memcpy(&desKey2, &hashV1[8], 8);
    memcpy(&desKey3, &hashV1[16], 4);
    memcpy(&desKey3[4], hashV2, 4);

    /* --- Decryption (8 byte blockSize, 24 byte Keylength) --- */
    DES_cblock iv;
    [rawData getBytes:iv length:8];
    NSInteger length = [rawData length];
    unsigned char *rawDataChar = (unsigned char*) malloc(length * sizeof(unsigned char));
    [rawData getBytes:rawDataChar length:length];

    NSInteger blockCount = (length - 40) / 8;
    if(blockCount < 1)
    {
        free(rawDataChar);
        return nil; // encPassword is no valid DES-encrypted password!
    }
    NSUInteger pwLength = blockCount*8;
    char* output = (char*) malloc(pwLength*sizeof(char));

    DES_key_schedule key_schedule1, key_schedule2, key_schedule3;

    DES_set_key_unchecked(&desKey1, &key_schedule1);     // generate key_schedule
    DES_set_key_unchecked(&desKey2, &key_schedule2);     // generate key_schedule
    DES_set_key_unchecked(&desKey3, &key_schedule3);     // generate key_schedule
    DES_ede3_cbc_encrypt((unsigned char*)&rawDataChar[40],(unsigned char*)output, pwLength,&key_schedule1,&key_schedule2,&key_schedule3,&iv,0);

    unsigned char len = output[pwLength-1];

    /* create null-terminated c-string */
    output[pwLength-len] = 0;

    /* rescue decrypted string */
    returnString = @(output);

    /* free allocated memory */
    free(rawDataChar);
    free(output);

    return returnString;

}

I already found the CommonCrypto function

CCCryptorStatus CCCrypt(CCOperation op, CCAlgorithm alg, CCOptions options, const void *key, size_t keyLength, const void *iv, const void *dataIn, size_t dataInLength, void *dataOut, size_t dataOutAvailable, size_t *dataOutMoved);

but I am not sure on how to port the OpenSSL parameters that I derive in my original method to the parameters required by CCCrypt(...).

You will need to look at the parameters for the CommonCrypto API, look at what each is and what you need, and then go through the OpenSSL API looking for things that correspond. This is a quite specific question, so I doubt anyone will be able to answer it offhand... — roelofs, Aug 16 '14 at 11:30
Alright. To be more specific here regarding the unclear parameters: What is the difference between the `key_schedule` in `DES_ede3_cbc_encrypt` and the `key` parameter of `CCCrypt`? How could I get the one from the other? — Fabian Jäger, Aug 16 '14 at 11:44
The code above looks to be using DES3. Just to be sure - are you using plain DES, or DES3 in CCCrypt? — roelofs, Aug 16 '14 at 11:45
I am using 3DES, but the more important question is how to generate the `key` parameter based on the information that I have. — Fabian Jäger, Aug 16 '14 at 12:13
I'll have to look at it in a bit more detail and get back to you... — roelofs, Aug 16 '14 at 13:27
For debugging: The encrypted string `CEEB491A40B430D29BE5A17A2C7141E0E0ED17D0DB6A17AA82B690D9524F7F25064B7C44058D87DD11A90A7020F5C037D09E90F863E7802C` should decode to `vpn-secret-key` — Fabian Jäger, Aug 16 '14 at 13:36
The clear text is 14 bytes. Given that DES (or 3DES) has a block size of 8 bytes and with added padding the resultant encrypted data would be 16 bytes. The encrypted data is 57 bytes (114 hex characters) so there are an unaccounted 41 bytes. So there is something else going on here such as some added preamble. You are going to have to find that out. Aside: Don't use 3DES on new work, use AES. — zaph, Aug 16 '14 at 14:13
As can be seen in the original source code, the actual encrypted string is taken from the converted hex string starting at byte 40 (`(unsigned char*)&rawDataChar[40]`). I guess this matches your comment then, right? — Fabian Jäger, Aug 16 '14 at 14:19
@FabianJäger Then you might want to remove your "For debugging" comment and add a correct comment. — zaph, Aug 16 '14 at 17:48
*"DES_ede3_cbc_encrypt"* - that's 3-key Triple DES using Encrypt-Decrypt-Encrypt (EDE). CBC is the block cipher mode of operation. Why not use `kCCAlgorithm3DES` instead of plain DES? Given that information, this might be helpful from Github: [Triple DES encryption example using the CommonCrypto Library](https://gist.github.com/marsepu/3854078). — jww, Aug 16 '14 at 22:53
The point is that I am not doing the encoding, but just the decoding part. The encoding is a very simple method used by Cisco for storing of VPN passwords on their users' computer. You can see examples of the decoding here: https://www.unix-ag.uni-kl.de/~massar/bin/cisco-decode — Fabian Jäger, Aug 17 '14 at 06:30

score 0 · Accepted Answer · answered Aug 20 '14 at 18:12

I solved the task by implementing the attached replacement method. Thanks again for all the helpful and inspiring comments.

NSString* CSDecryptCiscoPassword(NSString* encPassword)
{
    // convert to C
    NSUInteger rawDataLength = [encPassword length] >> 1;
    const char *encString = [encPassword cStringUsingEncoding:NSUTF8StringEncoding];

    // convert hex to bytes
    NSUInteger k = 0;
    unsigned char rawDataBytes[rawDataLength];
    for(NSUInteger i = 0; i < [encPassword length]; i = i+2)
    {
        const char *tmpString = &encString[i];
        int rawInt;
        sscanf(tmpString,"%2x",&rawInt);
        rawDataBytes[k++] = (unsigned char) rawInt;
    }

    const char *h1  = (const char*)rawDataBytes;
    const char *h4  = (const char*)rawDataBytes + 20;
    const char *enc = (const char*)rawDataBytes + 40;

    unsigned char ht[20], h2[20], h3[20], key[24];
    const char *iv = h1;

    if (rawDataLength < 48)
        return nil;
    rawDataLength -= 40;

    memcpy(ht, h1, 20);

    ht[19]++;
    CC_SHA1(ht, 20, h2);

    ht[19] += 2;
    CC_SHA1(ht, 20, h3);

    memcpy(key, h2, 20);
    memcpy(key+20, h3, 4);

    CC_SHA1(enc, rawDataLength, ht);

    if (memcmp(h4, ht, 20) != 0)
        return nil;

    size_t outLength = 0;
    NSMutableData *outputData = [NSMutableData dataWithLength:(rawDataLength+kCCBlockSize3DES)];

    CCCryptorStatus result = CCCrypt(kCCDecrypt, // operation
                     kCCAlgorithm3DES, // Algorithm
                     0, // options
                     key, // key
                     24, // keylength
                     iv,// iv
                     enc, // dataIn
                     rawDataLength, // dataInLength,
                     outputData.mutableBytes, // dataOut
                     outputData.length, // dataOutAvailable
                     &outLength); // dataOutMoved

    NSString* outString = [[NSString alloc] initWithData:outputData encoding:NSUTF8StringEncoding];

    return [outString autorelease];
}

How to port OpenSSL-based DES decryption method to CommonCrypto?

1 Answers1