First, I apologize if my title isn't technically accurate. I'm not sure exactly what's happening and that describes it about as well as anything.
I am attempting to decode an SSL certificate for a specific use in a program. I p/invoked all of the necessary CryptoAPI functions and structs and in debug mode, everything is working as expected. However, when the program is run in release mode as a service (that's the purpose of this program), I get an access violation when attempting to decode the extensions. In order to make extensions easier to decode, I have created a generic class that can be used to represent any extension. This class contains an object of type TStruct that represents the underlying structure that the extension is based on. It has an EncryptedValue field of type byte[] that will encrypt/decrypt the extension data and place it in the structure as appropriate. The base class is as follows:
public abstract class ExtensionBase<TStruct> : IExtensionBase where TStruct : new()
{
//The underlying struct
protected TStruct objectData = new TStruct();
//The identifier of the struct, ie: szOID_BASIC_CONSTRAINTS
public string Identifier { get; protected set; }
//An enum representing the structure type, ie: X509_BASIC_CONSTRAINTS
public CertStructType StructureType { get; protected set; }
//Determines if the extension is critical
public bool IsCritical { get; protected set; }
//Overridden in any child class to determine if that extension actually contains
//data that should be encoded
public abstract bool HasData { get; }
//Encrypts/decrypts the data from/to the underlying structure
public virtual byte[] EncryptedValue
{
get
{
uint encodedSize = 0;
//PinnedHandle is a class that I wrote to wrap a GCHandle.
//It has an implicit cast to IntPtr that returns GCHandle.AddrOfPinnedObject
//The finalizer of the class releases the GCHandle if it is a valid handle
IntPtr dataPtr = new PinnedHandle(objectData);
byte[] retVal = null;
if (StructureType != CertStructType.None)
{
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
(uint)StructureType,
dataPtr,
0,
IntPtr.Zero,
null,
ref encodedSize))
throw new Win32Exception();
retVal = new byte[encodedSize];
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
(uint)StructureType,
dataPtr,
0,
IntPtr.Zero,
retVal,
ref encodedSize))
throw new Win32Exception();
}
else
{
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
Identifier,
dataPtr,
0,
IntPtr.Zero,
null,
ref encodedSize))
throw new Win32Exception();
retVal = new byte[encodedSize];
if (!Crypt32.CryptEncodeObjectEx((uint)CertEncoding.X509Asn,
Identifier,
dataPtr,
0,
IntPtr.Zero,
retVal,
ref encodedSize))
throw new Win32Exception();
}
return retVal;
}
set
{
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if(StructureType != CertStructType.None)
decodedData = Crypt32.CryptDecodeObjectEx(StructureType, value);
else
decodedData = Crypt32.CryptDecodeObjectEx(Identifier, value);
TStruct data = (TStruct)Marshal.PtrToStructure(decodedData, typeof(TStruct));
objectData = data;
Marshal.FreeHGlobal(decodedData);
}
}
public ExtensionBase(string id)
{
Identifier = id;
StructureType = CertStructType.None;
}
public ExtensionBase(string id, CertStructType structType)
{
Identifier = id;
StructureType = structType;
}
}
One of the child classes that is giving me problems is the CertKeyUsage class which uses a CRYPT_BIT_BLOB struct to represent its data:
public class CertKeyUsage : ExtensionBase<CertKeyUsageFlags, CRYPT_BIT_BLOB>
{
public override bool HasData
{
get { return Value != CertKeyUsageFlags.None; }
}
public override unsafe byte[] EncryptedValue
{
get
{
CertKeyUsageFlags flags = Value;
objectData.cbData = 2;
objectData.cUnusedBits = 0;
objectData.pbData = new IntPtr(&flags);
return base.EncryptedValue;
}
set
{
try
{
//The following code was taken directly from Microsoft's implementation
//of X509Certificate
base.EncryptedValue = value;
if (objectData.cbData > 4)
objectData.cbData = 4;
byte[] keyUsage = new byte[4];
//This if statement returns true, and the following Marshal.Copy statement
//is where the program crashes with the Access Violation. As it is an unmanaged
//exception, the try/catch block doesn't do anything and the app dies
if (objectData.pbData != IntPtr.Zero)
Marshal.Copy(objectData.pbData, keyUsage, 0, (int) objectData.cbData);
Value = (CertKeyUsageFlags) BitConverter.ToUInt32(keyUsage, 0);
}
catch
{
}
}
}
public CertKeyUsage()
: base(CertOid.szOID_KEY_USAGE, CertStructType.X509KeyUsage)
{
IsCritical = true;
}
}
What about this code would be different from debug to release that would cause an access violation at the spot noted above? What could I do differently to get it working properly. This particular extension isn't critical and I could just skip over it, but after commenting out the code above that is causing the exception, another of the extensions will crash. I can't comment out all extensions, and the fact that it's moving to a different location tells me that there is some underlying problem with my code that I'm missing. Any help or suggestions would be greatly appreciated.
These are the p/invoked functions I'm calling:
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptEncodeObjectEx(uint certEncodingType,
[MarshalAs(UnmanagedType.LPStr)]
string structType,
IntPtr structInfo,
uint flags,
IntPtr encodePara,
byte[] encodedData,
[In, Out] ref uint encodedSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptEncodeObjectEx(uint certEncodingType,
uint structType,
IntPtr structInfo,
uint flags,
IntPtr encodePara,
byte[] encodedData,
[In, Out] ref uint encodedSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptDecodeObjectEx(uint certEncodingType,
[MarshalAs(UnmanagedType.LPStr)]
string structType,
byte[] encodedData,
uint encodedDataSize,
uint flags,
IntPtr encodePara,
IntPtr decodedData,
[In, Out] ref uint decodedDataSize);
[DllImport("crypt32.dll", CharSet = CharSet.Ansi, SetLastError = true)]
public static extern bool CryptDecodeObjectEx(uint certEncodingType,
uint structType,
byte[] encodedData,
uint encodedDataSize,
uint flags,
IntPtr encodePara,
IntPtr decodedData,
[In, Out] ref uint decodedDataSize);
And the function that wraps CryptDecodeObjectEx:
public static IntPtr CryptDecodeObjectEx(string structType, byte[] encodedData)
{
uint encodedSize = (uint)encodedData.Length;
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
decodedData = Marshal.AllocHGlobal((int)decodedSize);
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
return decodedData;
}
public static IntPtr CryptDecodeObjectEx(uint structType, byte[] encodedData)
{
uint encodedSize = (uint)encodedData.Length;
uint decodedSize = 0;
IntPtr decodedData = IntPtr.Zero;
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
decodedData = Marshal.AllocHGlobal((int)decodedSize);
if (!CryptDecodeObjectEx((uint)CertEncoding.X509Asn,
structType,
encodedData,
encodedSize,
0,
IntPtr.Zero,
decodedData,
ref decodedSize))
throw new Win32Exception();
return decodedData;
}
On a whim, I decided to compile the program as X86 rather than AnyCPU and everything works as expected. This leads me to believe that the problem is actually in this section of code:
public static IntPtr Next<T>(this IntPtr val)
{
T retVal = (T)Marshal.PtrToStructure(val, typeof(T));
if (Environment.Is64BitProcess)
return (IntPtr)((long)val + Marshal.SizeOf(retVal));
return (IntPtr)((int)val + Marshal.SizeOf(retVal));
}
This is the code that I use to enumerate through a pointer to an array of structures. The structure type is passed in and an IntPtr to the next structure in the array should be returned. The 32 bit version is working just as it should, but apparently my logic in the 64 bit version is somewhat lacking.
