InstallShield calling advapi32.dll method type mismatch error

Question

I am trying to call Advapi32.LsaOpenPolicy() from basic MSI InstallShield code. I've successfully called other avdapi32.dll methods; But LsaOPenPolicy is throwing a mismatched type error.

My prototype is:

prototype INT Advapi32.LsaOpenPolicy(POINTER, POINTER, INT, POINTER);

The windows definition is:

NTSTATUS LsaOpenPolicy(
  _In_     PLSA_UNICODE_STRING SystemName,
  _In_     PLSA_OBJECT_ATTRIBUTES ObjectAttributes,
  _In_     ACCESS_MASK DesiredAccess,
  _Inout_  PLSA_HANDLE PolicyHandle
);

I've noted in C++ samples that the ObjectAttriibute structure is zeroed out. So I do something similar here in the InstallShield code -- pArray points to the array contents.

    for i = 0 to 11
        array(i) = 0;
    endfor;
    array(0) = 24;

    // current error is 80020005 type mismatch.
    try
        i = POLICY_CREATE_ACCOUNT | POLICY_LOOKUP_NAMES;
        pArray = array; 
        pPolicy = NULL;
        nvOSResult = LsaOpenPolicy(NULL, pArray, i, pPolicy);
    catch
        Sprintf(errString, "0x%08x", Err.Number);
        _Logger(hMSI, methodName, "LsaOpenPolicy Exception "+errString, INFORMATION, FALSE);
        nvOSResult = Err.Number;
    endcatch;

There not much other information I can find other than the 80020005 error thrown; I've tried a few different argument constructions, but I can't get past this.

I've posted this in an flexera and microsoft forum -- but I have gotten no traction there. (references for posterity: flexera-link, microsoft-link)

Any help or suggestions are welcome!

I'm sorry, 10 years ago I would have been all over this but these days I don't see any point in investing this amount of effort into InstallScript. I'd either do it as a C++ custom action or a C# custom action. If you are trying to do what I think you are trying to do ( grand logon as service to an account ) there are far, far easier ways of doing this in a WiX authored merge module that then gets consumed by InstallShield. — Christopher Painter, Mar 31 '13 at 01:00
A custom action using C++ sounds like an interesting proposition. I am unfamiliar with WiX. — user2097370, Apr 01 '13 at 16:48
To complete the loop, I put everything in a C++ DLL (w/ static linking -- as I ran into a run-time mismatch, but that's another story). I buried the DLL as binary data in the binary table. — user2097370, Apr 05 '13 at 05:10

score 0 · Answer 1 · answered Sep 18 '13 at 19:55

The answer to this question was to actually work-around the interface between installshield and the system DLLs by moving all the workings into a C++ DLL. As installation got more complex, I ended up with two separate DLL functions, one executed at dialog (non-admin) mode and one at deferred execution (admin) mode.

In order to pass information I used the MsiGetProperty() API using MSI properties for both input and output variables.

Note that for deferred execution, I needed a CAD function on the installshield side to marshal data into the custom action data location, and on the DLL side extract the data, again by using MsiGetProperty() but getting the "CustomActionData" property and then parse the resulting string which contained the marshaled data.

InstallShield calling advapi32.dll method type mismatch error

1 Answers1