VB6 - How to pass Strings to a DLL written in C through a .tlb File

Question

I'm working on a graphical interface written in VB6, where I have to call function contained in a certain DLL written in C. Because of a known limitation I had to implement a trick that allows me to load this DLL in a implicit way.

This is possible creating an IDL file, compile it with MIDL and reference the resulting .tlb file in the VB6 project.

The problem is that VB6 strings and C arrays of char do not match, so I can't pass (and get back) them to the DLL.

The prototype of the C function is:

int __stdcall myFunc(char filename_in[], char filename_out[], char ErrMsg[]);

What should I write in the IDL file and how should I call it from VB6?

Thanks.

@GolezTrol Yep, you are right, it's a mistake call them string, I did it because in VB6 I just know strings and not arrays of char. — Beppe, Mar 08 '11 at 11:44
I think you'd either need to use the `Declare Function myFunc lib "MyDLL"` syntax or write a proper COM interface using BSTRs (unicode strings with a length prefix at word -1) not char[]. — Rup, Mar 08 '11 at 11:50
@Beppe As you know I've been following this thread. Please explain why you can't just use `Declare Function`? As far as I am aware, you will be able to do this in addition to your tlb trick to workaround the TLS issue. — David Heffernan, Mar 08 '11 at 11:51
@David Do you mean I can use `Declare Function` combined with the tlb trick? I didn't try it out yet. I thought that declaring functions in that way would have brought the problem back. I should check it. Thanks — Beppe, Mar 08 '11 at 11:57
@Beppe That's the idea. Can you confirm to me that the DLL is in the import table for the VB6 produced .exe? Check this with MS Dependency Walker. If so then `Declare Function` will be fine. — David Heffernan, Mar 08 '11 at 12:13
@David Yes I can confirm it. It works, the only problem left are the strings -> char* conversions — Beppe, Mar 09 '11 at 08:39

score 3 · Answer 1 · answered Mar 08 '11 at 14:15

3

You must use BSTR to use VB6 compatible strings. It is the standard COM string type, it stores Unicode strings in utf-16 encoding, just like the Win32 api.

 int __stdcall myFunc(BSTR filename_in, BSTR filename_out, BSTR* ErrMsg);

You can cast the in args to WCHAR* directly, use WideCharToMultiByte() if you need to convert to char* (best avoided). Use SysFreeString if *ErrMsg is not null to release an existing string before assigning it. Use SysAllocString to allocate the ErrMsg string. It must be a utf-16 string as well, MultiByteToWideChar() if necessary again to convert from char*. Or use a string literal that's prefixed with L, like L"Oops".

answered Mar 08 '11 at 14:15

Hans Passant

922,412
146
1,693
2,536

@Hans I cannot change the DLL code. So I have to find a solution that involves only VB6 and IDL changes (so I can't use WideCharToMultiByte and SysAllocString) – Beppe Mar 08 '11 at 17:33
Not sure if that's possible, rather doubt it. The ErrMsg argument is very problematic. Why don't you write C code that you can change that simply calls the C code that you cannot change. – Hans Passant Mar 08 '11 at 17:54
@Hans It would imply to change the DLL and I cannot do it, because there are several applications/platforms/services that uses it. – Beppe Mar 08 '11 at 19:44
No it doesn't. You write another one. Talk to the members of your team, surely somebody can help you. – Hans Passant Mar 08 '11 at 19:59
Standard API functions don't take BSTRs as params, just plain LPSTR or LPWSTR but are still callable through Declare or a typelib. How does this fit with *You must use BSTR to use VB6 compatible strings*? Do you mean that CHAR* and WCHAR* params are not callable by VB6? – wqw Mar 08 '11 at 22:33
@wqw If I declare the char arrays using char* or wchar*, VB6 return me an error. VB6 recognize the BSTR type as String, but seems that on the C side it is not readable. – Beppe Mar 09 '11 at 08:46
@wqw A BSTR is generally an UTF-16 nul-terminated string - i.e. it works as an LPWSTR - but that also has its length stored in the word before the string pointer. You can pass arbitrary (<32k) binary blobs as BSTRs too but here we're not. If you're calling through COM (i.e. the function in the IDL) you need to use BSTRs so the marshaller can use the length value to copy the data exactly; if you're just calling standard APIs (i.e. through declare function) you don't need it. VB strings are (I think) BSTRs contained in VARIANTs. – Rup Mar 09 '11 at 10:12
@Beppe really if you have a DLL with a Windows API it should be using Unicode nowadays not plain chars. You can ask your team to add a parallel function MyFuncW (same as all the API calls have -A and -W variants) that you can call using wide strings; if necessary they can just write a wrapper around the original MyFunc that does the string conversions, although really it ought to be the other way around: you should be processing all string data in Windows as Unicode. – Rup Mar 09 '11 at 10:20
@Rup: When using a `Declare` an LPSTR param is declared as `ByVal As String` and the run-time does the BSTR<->LPSTR conversion (both ways). BSTR length prefix is 4 bytes so size limit is at 2GB. OP is using a typelib to declare API functions where both LPSTR and LPWSTR params appear 'As String' to VB6 and are callable with automatic BSTR<->LP[W]STR converstion -- no need for an extra MyFuncW effort. – wqw Mar 09 '11 at 15:59
@wqw True - not sure why I remembered word length for BSTRs. I'm not sure he really is using a typelib, though - that's for some TLS trick, whereas the function he has isn't part of a COM object (which is what I thought typelibs were for?). Anyway glad it sounds like he got it working, but I still think a fixed 256 characters in his string declaration is wrong. (And so's non-Unicode on Windows, on principle if not technically.) – Rup Mar 09 '11 at 16:09
@wqw - focus on the ErrMsg argument, there is a nasty memory management problem with it. The C code must allocate it so that the VB6 runtime can release it. Only BSTR qualifies. – Hans Passant Mar 09 '11 at 16:12
@Hans: ErrMsg must be allocated by the caller. Obviously the function is missing a parameter specifying buffer size or size must be hardcoded (256?). WinAPI handles output structs or strings -- lpszBuffer + cbSize, buffer allocated by caller. Shell API is using CoTaskMemXxx allocator. Anyway, in his case he would have to use `sOutFile = String(257, 0) : sErrMsg = String(257, 0) : Call myFunc(sInFile, sOutFile, sErrMsg)` and after that "massage" output strings with `sOutFile = Left$(sOutFile, InStr(sOutFile, Chr$(0)) - 1)` – wqw Mar 09 '11 at 18:41

score 2 · Answer 2 · answered Mar 08 '11 at 14:15

2

VB6 has no problems consuming stdcall functions with ANSI strings params. Just use [in] LPSTR filename_in in IDL and the run-time does the UNICODE<->ANSI conversion automagically.

The "magic" works for [out] params too.

answered Mar 08 '11 at 14:15

wqw

11,771
1
33
41

score 0 · Accepted Answer · edited May 23 '17 at 11:55

0

Thanks to GSerg and wqw I found the solution to this problem:

In the IDL file the char arrays should be declared as LPSTR, so the prototype of the function looks like:

int _stdcall myFunc(LPSTR file_name_in, LPSTR file_name_out, LPSTR ErrMsg)

note that ErrMsg is declared exactly as the other arrays, even if it will contains an output message (readable on the VB6 side).

On the VB6 side the strings should be allocated as:

Dim file_name_in As String * 256
Dim file_name_out As String * 256
Dim ErrMsg As String * 256

Doing so these strings are allocated with a limited size of 256, thus being compatible with the char arrays in the C DLL.

Hope this will help someone else.

Regards,

G.B.

edited May 23 '17 at 11:55

Community

1
1

answered Mar 08 '11 at 14:44

Beppe

381
2
7
15

You should use the `[in]` and `[out]` hints against the arguments in your IDL and then you shouldn't have to set a fixed length. You are using the proper [SysAllocString](http://msdn.microsoft.com/en-us/library/ms221458.aspx) functions to generate the BSTRs, aren't you, so that they get the string length word set too? – Rup Mar 08 '11 at 14:57
I only tried to use [in] [out] in the IDL but without using SysAllocString. I'll try it out as soon I get solved onther problem I encountered. Thank you very much. – Beppe Mar 08 '11 at 15:21
@Beppe: This should work without the * 256 part too. Did you try declaring string params as `[in] LPSTR` and `[out] LPSTR *`? – wqw Mar 09 '11 at 16:03
@wqw Actually I didn't tried it,but I'm going to test this solution too. – Beppe Mar 10 '11 at 13:41
@wqw I tried your solution but it crashes, while if I take off the pointer from the [out] parameter, the application does not crash, but the returned string is empty. – Beppe Mar 10 '11 at 13:59

VB6 - How to pass Strings to a DLL written in C through a .tlb File

3 Answers3