Additional characters in RC4 encryption

Question

I am facing an issue when trying to encrypt a string using Open SSL RC4 function. Somme additionnal characters where added at the end of the encrypted string.

Here is my code :

RC4_KEY rc4_key;
QString key= "612c207468652066";

QByteArray keyData=key.QString::toUtf8();
unsigned char * mykey=reinterpret_cast<unsigned char *>(keyData.data());

QString input = "7447806cc997966f6b8b59a57c01bfe6fe8381d4fed5628b531fbeb1c4629151722ee712fa906dfea1c68b7015243bfcdd42d3e990c1bd6daa56df620a9d1e441f4ba308da5584e032f06fe925ee9df328fadab0ad2a5869fdea366189a397f72b320a4f11b0cc41cbfaa3b6923f20cc8c4c80d9c9b69eb5e2a6cc2fb9ee2e72ca79f08617b3b5812eddb3f58b68";
qDebug()<< "input"<<input << input.size();
QByteArray inputData=input.QString::toUtf8();
unsigned char * ciphertext1= (unsigned char*)malloc(input.size());
memset(ciphertext1, 0, input.size());
ciphertext1=reinterpret_cast<unsigned char *>(QByteArray::fromHex(inputData).data());
printf("ciphertext1: %s\n", ciphertext1);

RC4_set_key(&rc4_key,keyData.size() , mykey);

unsigned char * firsthashJson= (unsigned char*)malloc(input.size());
firsthashJson[input.size()-1]=0x00;
memset(firsthashJson, 0, input.size());
RC4(&rc4_key,input.length(),ciphertext1, firsthashJson);
printf("Decrypted: %s\n", firsthashJson);

QString str1 = QString::fromUtf8((char*)firsthashJson);

I obtained the following result: b036fe3eae7881fe135562335f9a##5265705250##b2a475f7a99cf400d34da436d522d9bc5566d8b7a015c7a16ee2d5e730199acf1f61fe2f3d6b9525e1c45acb36383253b4fc< Ú«Í`ã¿rßÖQÕ§ìj‗¤Ì&Öâ¤#>XÏ¸0õ+▲®╩Ä=╚líUm"ód☻Ô1ı┘©ÜLÔ»âH÷/o┐öñ▄|▄ÐÄ┤N ☻§@ı2w5▒KTâUÆB¢ÏdÔxÚ⌂ê☺°¹╔«▓J)→8èê╔┤║ù@│k ↕▬↨[║GtUË(♂D

the expected one is : b036fe3eae7881fe135562335f9a##5265705250##b2a475f7a99cf400d34da436d522d9bc5566d8b7a015c7a16ee2d5e730199acf1f61fe2f3d6b9525e1c45acb36383253b4fc

Do you have any idea of the source cause of this issue.

Thanks in advance for your help.

###############################################################

@paddy Thank you for your help.

Unfortunately i have the same issue here is the new code:

RC4_KEY rc4_key;
QString key= "612c207468652066";

QByteArray keyData=key.QString::toUtf8();
unsigned char * mykey=reinterpret_cast<unsigned char *>(keyData.data());

QString input = "7447806cc997966f6b8b59a57c01bfe6fe8381d4fed5628b531fbeb1c4629151722ee712fa906dfea1c68b7015243bfcdd42d3e990c1bd6daa56df620a9d1e441f4ba308da5584e032f06fe925ee9df328fadab0ad2a5869fdea366189a397f72b320a4f11b0cc41cbfaa3b6923f20cc8c4c80d9c9b69eb5e2a6cc2fb9ee2e72ca79f08617b3b5812eddb3f58b68";
qDebug()<< "input"<<input << input.size();
QByteArray inputData=input.QString::toUtf8();
unsigned char * ciphertext1= (unsigned char*)malloc(input.size());
memset(ciphertext1, 0, input.size());
ciphertext1=reinterpret_cast<unsigned char *>(QByteArray::fromHex(inputData).data());
printf("ciphertext1: %s\n", ciphertext1);

RC4_set_key(&rc4_key,keyData.size() , mykey);

unsigned char * firsthashJson= (unsigned char*)malloc(input.size()+1);

memset(firsthashJson, 0, input.size()+1);
firsthashJson[input.size()]=0;

RC4(&rc4_key,input.size(),ciphertext1, firsthashJson);
printf("Decrypted: %s\n", firsthashJson);

firsthashJson[input.size()]=0;
QString str1 = QString::fromUtf8((char*)firsthashJson);

qDebug()<<"final string "<<str1<< str1.size();

You must allocate enough storage for your NUL-terminator, if you want NUL-terminated strings. This looks like a classic case of unterminated strings. It looks like you_tried_ with `firsthashJson[input.size()-1]=0x00;` but that's not the right position. Besides, you immediately clear the entire memory on the next line with `memset` so this one is pointless. You probably need `input.size() + 1` bytes, and to terminate you can then do `firsthashJson[input.size()] = 0;`. Consider doing that after the call to `RC4`. — paddy, Jul 06 '22 at 08:29

Botje · Answer 1 · 2022-07-06T13:37:15.700

Your core problem is that you are passing input.size() as the len parameter to RC4, but input.size() is actually double the size of your ciphertext1 because of hex encoding.

Here is a correct version without leaving the comfort and safety of Qt:

QString key = "612c207468652066";
QString input = "7447806cc997966f6b8b59a57c01bfe6fe8381d4fed5628b531fbeb1c4629151722ee712fa906dfea1c68b7015243bfcdd42d3e990c1bd6daa56df620a9d1e441f4ba308da5584e032f06fe925ee9df328fadab0ad2a5869fdea366189a397f72b320a4f11b0cc41cbfaa3b6923f20cc8c4c80d9c9b69eb5e2a6cc2fb9ee2e72ca79f08617b3b5812eddb3f58b68";

RC4_KEY rc4_key;
RC4_set_key(&rc4_key, key.size(), key.toUtf8().data()); // Or qUtf8Printable(key)
QByteArray ciphertext = QByteArray::fromHex(input.toUtf8());

QByteArray firsthashJson(ciphertext.size(), 0);
RC4(&rc4_key, ciphertext.size(), ciphertext.data(), firsthashJson.data());

QString decrypted = QString::fromUtf8(firsthashJson);
qDebug()<<"final string " << decrypted;

Additional characters in RC4 encryption

1 Answers1