How can I cope with 32bit/64bit mismatches when doing IPC via SendMessage?

Question

I have a piece of C++ code which reads out the text of a tree item (as contained in a plain Common Controls Tree View) using the TVM_GETITEM window message. The tree view which receives the mesage is in a different process, so I'm using a little bit of shared memory for the structure which is pointed to by one of the arguments to the window message. I have to do this work since the remote process is not under my control (I'm writing an application similiar to Spy++).

This works well in principle, but fails in case the target process is substantially different:

1. If the code of the target process was built with UNICODE defined but my own code wasn't, the two processes will have different ideas about the structure of the string members in the TVITEM structure. I solved this already using an IsWindowUnicode call and then explicitely sending either TVM_GETITEMA or TVM_GETITEMW (recoding the result if necessary).

2. If the calling process was built in 32bit mode and the target process is 64bit (or the other way round), the layout (and size) of the TVITEM structure structure is different since pointers have a different size.

I'm currently trying to find a good way to solve the second issue. This particular use case (getting the tree item text) is just an example, the same issue exists for other window messages which my code is sending. Right now, I'm considering two approaches:

1. Build my code twice and then execute either the 32bit or the 64bit code depending on what the target process does. This requires some changes to our build- and packaging system, and it requires factoring the code which is architecture specific out into a dedicated process (right now it's in a DLL). Once that is done, it should work nicely.
2. Detect the image format of the target process at runtime and then use custom structs instead of the TVITEM structure structure which explicitely use 32bit or 64bit wide pointers. This requires writing code to detect the architecture of a remote process (I hope I can do this by calling GetModuleFileName on the remote process and then analyzing the PE header using the Image Help Library) and hardcoding two structs (one with 32bit pointers, one with 64bit). Furthermore, I have to make sure that the shared memory address is in the 32bit address space (so that my own code can always access it, even if it's compiled in 32bit mode).

Did anybody else have to solve a similiar problem? Are there easier solutions?

Answer 1

I ended up checking whether the remote process is 32bit or 64bit at runtime, and then writing the right structure to the shared memory before sending a message.

For instance, here's how you can use the TVM_GETITEM message even if there's a 32bit <-> 64bit mixup between the caller and the receiver of the message:

/* This template is basically a copy of the TVITEM struct except that
 * all fields which return a pointer have a variable type. This allows
 * creating different types for different pointer sizes.
 */
template <typename AddrType>
struct TVITEM_3264 {
  UINT      mask;
  AddrType  hItem;
  UINT      state;
  UINT      stateMask;
  AddrType  pszText;
  int       cchTextMax;
  int       iImage;
  int       iSelectedImage;
  int       cChildren;
  AddrType  lParam;
};
typedef TVITEM_3264<UINT32> TVITEM32;
typedef TVITEM_3264<UINT64> TVITEM64;

// .... later, I can then use the above template like this:
LPARAM _itemInfo;
DWORD pid;
::GetWindowThreadProcessId( treeViewWindow, &pid );
if ( is64BitProcess( pid ) ) {
    TVITEM64 itemInfo;
    ZeroMemory( &itemInfo, sizeof( itemInfo ) );

    itemInfo.mask = TVIF_HANDLE | TVIF_TEXT;
    itemInfo.hItem = (UINT64)m_item;
    itemInfo.pszText = (UINT64)(LPTSTR)sharedMem->getSharedMemory( sizeof(itemInfo) );
    itemInfo.cchTextMax = MaxTextLength;
    _itemInfo = (LPARAM)sharedMem->write( &itemInfo, sizeof(itemInfo) );
} else {
    TVITEM32 itemInfo;
    ZeroMemory( &itemInfo, sizeof( itemInfo ) );

    itemInfo.mask = TVIF_HANDLE | TVIF_TEXT;
    itemInfo.hItem = (UINT32)m_item;
    itemInfo.pszText = (UINT32)(LPTSTR)sharedMem->getSharedMemory( sizeof(itemInfo) );
    itemInfo.cchTextMax = MaxTextLength;
    _itemInfo = (LPARAM)sharedMem->write( &itemInfo, sizeof(itemInfo) );
}

The sharedMem->getSharedMemory function is a little helper function to get a pointer to the shared memory region; the optional function argument specifies an offset value. What's important is that the shared memory region should always bee in the 32bit address space (so that even a 32bit remote process can access it).

Answer 2

IMHO there is a design problem. I don't know why your are doing this way, maybe you don't have total control of all parts. But in a basic MVC perspective, you are peeking values from a view instead of asking it to the model.

Answer 3

I'm not familiar with this particular message, but if the Windows TVM_GETITEM message is supposed to function correctly across processes, then Windows should fill in the TVITEM struct in the caller's address space and handle any needed conversions for you, without needing you to supply shared memory. If it isn't, then I doubt the problem you're seeing here is easily solvable without some uncomfortable contortions.

The shared memory bit confuses me; generally you have to make both processes explicitly aware of the shared memory segment, and you didn't mention DLL injection or anything like that. How exactly is the callee made aware of the shared memory section in its address space, and how are you using it? Are you sure it's needed for this API?