0

When I run this code in C++ (shown), C#, or VB6, I get different results for pbBuffer in CryptEncrypt! The only thing I can think of is CryptEncrypt is language-based or the parameters are wrong for the other functions. What other reason would this happen? Am I passing the wrong parameters to CryptDeriveKey?

if (CryptAcquireContext(&hProv, NULL, NULL, PROV_RSA_FULL, 0))     
{   
    if (CryptCreateHash(hProv, CALG_MD5, 0, 0, &hHash))        
    {   
        if (CryptHashData(hHash, (BYTE *)szLocalPassword, _tcslen(szLocalPassword), 0))            
        {
            if (CryptDeriveKey(hProv, CALG_RC4, hHash, CRYPT_EXPORTABLE, &hKey))                   
            {   
                dwLength = sizeof(TCHAR)*_tcslen(sUnencryptedString);                      
                BYTE *pbBuffer = (BYTE *)malloc(dwLength);                     
                if (pbBuffer != NULL)                      
                {   
                // Convert TCHAR to BYTE
                char c_szText[128] = {0};       // Any char
                SIZE_T so = wcslen(sUnencryptedString) + 1;
                wcstombs(c_szText, sUnencryptedString, so);
                memcpy(pbBuffer, c_szText, dwLength);

                if (CryptEncrypt(hKey, 0, TRUE, 0, pbBuffer, &dwLength, dwLength))                         
JeffR
  • 765
  • 2
  • 8
  • 23
  • 1
    Describe what "different results" means. – Robert Harvey Apr 23 '14 at 21:40
  • You would have to show us how you have converted it into the other languages. Perhaps the issue is character-set-related, for example. (String types in VB6 and C# contain characters, not bytes, so they could be interpreted differently when marshaled for the `CryptEncrypt()` call.) – cdhowie Apr 23 '14 at 21:41
  • 2
    My guess would be related to character encodings. C# uses UTF-16 to encode strings by default, for example. – Martin Costello Apr 23 '14 at 21:48
  • 1
    Add to above VB6 also uses UTF-16 internally and when using objects. But in API calls VB6 converts UTF-16 to ANSI. So you need to call A versions of functions or pass byte arrays instead of strings to W versions. All API functions have an A or W version if they take or return a string. – tony bd Apr 23 '14 at 22:02
  • Or pass `StrPtr(stringvar)` to a `Long` API parameter if you really want to pass the UTF-16. I don't believe most of the Crypto API entrypoints come in A and W flavors. – Bob77 Apr 24 '14 at 02:47
  • @JeffR You really must provide more information - particularly the C# and VB code. – Mark Bertenshaw Apr 24 '14 at 16:28
  • CryptoAPI uses little-endian byte order. .NET Framework API uses big-endian byte order. – Raj Apr 25 '14 at 08:29
  • @martin_costello and "tony bd" got me pointed in the right direction. – JeffR Apr 29 '14 at 01:13

1 Answers1

0

This is the VB code that shows how to pass the correct info to the Crypto APIs to encrypt a string in VB6. Hopefully someone can use this info when searching!

...
Dim bKeyRoot() As Byte
bKeyRoot = StrConv(sKeyRoot, vbFromUnicode)
lResult = CryptHashData(hHash, bKeyRoot(0), Len(sKeyRoot), 0)
...
Dim bText() As Byte
bText = StrConv(sText, vbFromUnicode)
lResult = CryptEncrypt(lKey, 0, 1, 0, bText(0), lClear, lData)
sText = StrConv(bText, vbUnicode)
JeffR
  • 765
  • 2
  • 8
  • 23