Delphi DLL (in XE) must handle TStringList (D2007, Ansi)

Question

The DLL was originally written in D2007 and needed a quick, panic TStringList call (yes, it was one of those “I’m sure to regret”; though all the calls to the DLL, made by several modules, are all made by Delphi code and I wrongly presumed/hoped backwards compatibility when XE came out).

So now I’m moving the DLL to XE5 (& thus Unicode) and must maintain the call for compatibility. The worst case is I simply write a new DLL only for XE while keeping the old one for legacy, but feel there should be no reason why XE couldn’t deconstruct/overrride to an {ANSI} TStringList parameter. But my Delphi behind-the-scenes knowledge is not robust and a couple of attempts have not succeeded.

Here is the DLL call – it takes a list of file paths and in this stripped-down code, simply adds each string to an internal list (that is all the DLL does with the parameter, a single read-only reference):

function ViewFileList ( lstPaths: TStringList): Integer; Export; Stdcall;
begin
      for iCount := 0 to lstPaths.Count - 1 do
         lstInternal.Add(lstPaths.strings[iCount]);
end;

What I found is that when I compiled this in XE5, that lstPaths.Count is correct, so the basic structure aligns. But the strings were garbage. It seems the mismatch would be two-fold: (a) the string content naturally is being interpreted as two-bytes per character; (b) there is no Element size (at position -10) and code page (at position -12; so yes, garbage strings). I am also vaguely aware of behind-the-scenes memory management, though I only do read-only access. But the actual string pointers themselves should be correct (??) and thus is there a way to coerce my way through?

So, regardless of whether I have any of that right, is there any solution? Thanks in advance.

Answer 1

What you perhaps don't yet realise is that your code has always been wrong. In general, it is not supported to pass Delphi objects across module boundaries. You can make it work so long as you understand the implementation very well, so long as you don't call virtual methods, so long as you don't do memory allocation, so long as you use the same compiler on both sides, and probably many other reasons. Either use runtime packages (also requires same compiler on both sides), or use interop safe types (integers, floats, null terminated character arrays, pointers, records and arrays of interop safe types, etc.)

There's really no simple solution here. It should never have worked in the first place and if it did then you have been very unlucky. Unlucky because a much better outcome would have been a failure that would have led you to doing it properly.

Perhaps the best thing you can do is make an adapter DLL. The architecture goes like this, from bottom to top:

• Original Delphi 2007 DLL at the bottom, with the bogus export that requires D2007 string list to be supplied.
• New adapter Delphi 2007 DLL in the middle. It calls the bogus export, and is able to supply a D2007 string list. The adapter DLL exposes a proper interface that does not require Delphi objects to be passed across the module boundary.
• New XE5 executable at the top. This talks to the adapter, but does so using valid interop types.

Answer 2

David and Jerry already told you what you should do - re-write the DLL to do the right thing when it comes to passing interop-safe data across module boundaries. However, to answer your actual question:

the actual string pointers themselves should be correct (??) and thus is there a way to coerce my way through?

So, regardless of whether I have any of that right, is there any solution?

You can try the following. It is dangerous, but it should work, if a re-write is not an option for you at this time:

// the ASSUMPTION here is that the caller has been compiled in D2007 or earlier,
// and thus is passing an AnsiString-based TStringList object.  When this DLL is
// compiled in Delphi 2009 or later, TStringList is UnicodeString-based instead,
// so we have to re-interpret the data a little.
//
// The basic structure of TStringList itself should be the same, just the string
// content is different.  For backwards compatibility, the refcnt and length
// fields of the StrRec record found in every AnsiString/UnicodeString payload
// are still at the same offsets. Delphi 2009 added some new fields, but we can
// ignore those here.
//
// Of course, XE is the version that removed the RTL support code for the {$STRINGCHECKS}
// compiler directive, which handled all of these details in Delphi 2009 and 2010
// when users were first migrating to Unicode.  But in XE, we'll have to deal with
// it manually.
//
// These assumptions may change in future versions, but lets deal with that if/when
// the time comes...

function ViewFileList ( lstPaths: TStringList): Integer; Export; Stdcall;
{$IFDEF UNICODE}
var
  tmp: AnsiString;
{$ENDIF}
begin
  for iCount := 0 to lstPaths.Count - 1 do
  begin
    {$IFDEF UNICODE}

    // the DLL is being compiled in Delphi 2009 or later...
    //
    // the Length(String) function simply returns the value of the string's
    // StrRec.length field, which fortunately is in the same location in
    // both pre-2009 AnsiString and 2009+ AnsiString/UnicodeString, and in
    // this case will reflect the number of AnsiChar elements in the source
    // AnsiString.  We cannot simply typecast a "UnicodeString" directly to
    // a PAnsiChar, nor can we typecast a PWideChar to a PAnsiChar, but we
    // can typecast a string to a Pointer first and then cast that to a
    // PAnsiChar.  This code is assuming that it can safely get a pointer to
    // the source AnsiString's underlying character data to make a local
    // copy of it that can then be added to the internal list normally.
    //
    // Where this MIGHT fail is if the source AnsiString contains a reference
    // to a string literal (StrRec.refcnt=-1) for its character data, in
    // which case the RTL will try to copy the character data when assigning
    // the source string to a variable, such as the one the compiler is
    // likely to generate for itself to receive the TStringList.Strings[]
    // property value before it can be casted to a Pointer.  If that happens,
    // this is likely to crash when the RTL tries to copy too many bytes from
    // the source AnsiString!  You can use the StringRefCount() function to
    // detect that condition and do something else, if needed.
    //
    // But, if the source AnsiString is a normal allocated string (the usual
    // case), then this should work OK.  Even with the compiler-generated
    // variable in play, the compiler should simply bump the reference count
    // of the source AnsiString, without affecting the underlying character
    // data, just long enough for this code to copy the data and release the
    // reference count...
    //
    SetString(tmp, PAnsiChar(Pointer(lstPaths.strings[iCount])), Length(lstPaths.strings[iCount]) * SizeOf(AnsiChar));
    lstInternal.Add(tmp);

    {$ELSE}

    // the DLL is being compiled in Delphi 2007 or earlier, so just add the
    // source AnsiString as-is and let the RTL do its work normally...
    //
    lstInternal.Add(lstPaths.strings[iCount]);

    {$ENDIF}
  end;
end;