How can I use shared_ptr using PostThreadMessage?

Question

I would like to upgrade my MFC production code to use the std::shared_ptr smart pointer when calling other windows or threads. Such calls are SendMessage, PostMessage and PostThreadMessage which pass wparam and lparam and which respectively are an unsigned int and long. Currently, I create a class object, new an object, make the call passing a pointer to the object, use the object on the receiving end and then delete it.

Since shared_ptr works so well in the rest of my code I wanted to at least explore the reasons why I can't do the same for windows calls.

Current call:

auto myParams = new MyParams(value1, value2, value3);
PostThreadMessage(MSG_ID, 0, reinterpret_cast< LPARAM >( myParams );

ReceivingMethod::OnMsgId( WPARAM wParam, LPARAM lParam)
{
  auto myParams = reinterpret_cast< MyParams * >( lParam );
  ... // use object
  delete myParams;
}

to a C++11-like smart pointer call:

std::shared_ptr< MyParams > myParams( new MyParams( value1, value2, value3 ) );
PostThreadMessage( MSG_ID, 0, ???myParams??? );

ReceivingMethod::OnMsgId( WPARAM wParam, LPARAM lParam )
{
  auto myParams = ???lParam???;
  ... // use object
}

Edit 1:

@Remy Lebeau: Here is the sample code revised to use the unique_ptr passing approach, however, there are leaks in my code when passing the object:

struct Logger
{
  Logger()
  {
    errorLogger = ( ErrorLogger * )AfxBeginThread( RUNTIME_CLASS( ErrorLogger ), THREAD_PRIORITY_BELOW_NORMAL );
  }

  ~Logger()
  {
    // gets properly dtor'ed upon app exit
  }

  void MakeLogMsg( ... );

  ErrorLogger * errorLogger;
  std::unique_ptr< LogParams > logParams;
};

Logger logger;
std::recursive_mutex logParamsRecursiveMu; // because of multiple requests to lock from same thread

struct ErrorLogger : public CWinThread
{
  ErrorLogger()
  {
  }

  ~ErrorLogger()
  {
    // gets properly dtor'ed before logger upon app exit
  }

  afx_msg void OnLog( WPARAM wParam, LPARAM lParam );
};

void Logger::MakeLogMsg( ... )
{
  // construct msg from logparams 

  // make msg smart object using unique ptr and send to errorlogger thread queue
  logParams = std::make_unique< LogParams >();

  // set logparams

  // with the addition of the mutex guard, the leaks are gone
  logParamsRecursiveMu.lock();
  logger.errorLogger->PostThreadMessage( ONLOG_MSG, 0, reinterpret_cast< LPARAM >( logParams.get() ) );
  logParams.release(); // no longer owns object
  logParamsRecursiveMu.unlock();
}

void ErrorLogger::OnLog( WPARAM wParam, LPARAM lParam )
{
  std::unique_ptr< LogParams > logParams( reinterpret_cast< LogParams * >( lParam ) );
}

Notice that when I comment out the passing of the unique_ptr, the leaks disappear. How does my code differ from your code that uses this approach and works?

Edit2:

With regard to @Remy Lebeau‘s answer showing how std::unique_ptr could be used instead of std::shared_ptr, I stated in a comment below that there were “…no extra objects to implement. No obvious cons.”. Well, that is not quite true. The MyParams object has to be created for each different type of message. Some apps might only have a few types, but some might have 100s or more. Each time I want to execute a function on the other side I have to craft a new struct which has a constructor that takes all the arguments of the destination call. Very tedious to implement and hard to maintain if there are many.

I think that it would be possible to eliminate that struct-building phase by merely passing the arguments.

Apparently there are new C++1x constructs that can help accomplish this. One is perhaps the std::forward_as_tuple which “Constructs a tuple of references to the arguments in args suitable for forwarding as an argument to a function.”

For my app I solved the problem by templatizing MyParams, but for anyone whose wants to avoid adding a lot of structs, he may want to look at using tuples and the like.

Edit 3: Both @RemyLebeau's answer number 1 and @rtischer8277’s answer number 5 are correct. Unfortunately, StackOverflow doesn’t recognize multiple correct answers. This StackOverflow limitation reflects a flawed PsychoLinguistic assumption that linguistic context is universal for the same language group, which it is not. (see “Introduction to Integrational Linguistics” by Roger Harris on the fixed-code language myth, p.34). In response to my original posting, @RemyLebeau answered the question based on the context described by the posted code that shows a newed MyParams (see Edit 2: for more explanation). Much later in Answer 5 (rtischer8277), I answered the question myself based on the original wording of the question which asked if std::shared_ptr could be used across threads using PostThreadMessage. As a reasonable consequence, I have re-assigned the correct answer back to @RemyLebeau it being the first correct answer.

EDIT4: I added a third legitimate answer to this posting. See the 6th Answer beginning with @Remy Lebeau and @rtischer8277 have so far submitted two answers to my original posting…. The effect of this solution is to turn cross-thread access into the conceptually simple RPC (remote procedure call). Although this Answer shows how to use a future to control ownership and synchronization, it does not show how to create a message object with an arbitrary number of parameters that can safely be used on either side of the PostThreadMessage call. That functionality is addressed in StackOverflow’s Answer to issue passing a parameter pack over a legacy function signature using forward_as_tuple.

Answer 1

Since the message parameters have to outlive the calling scope, it does not make much sense to use shared_ptr in this example, as the source shared_ptr is most likely to go out of scope before the message is processed. I would suggest using unique_ptr instead, so that you can release() the pointer while it is in flight, and then obtain new ownership of it once the message is processed:

SendingMethod::SendMsgId( ... )
{
    ...

    std::unique_ptr<MyParams> myParams( new MyParams(value1, value2, value3) );
    if (PostThreadMessage(MSG_ID, 0, reinterpret_cast<LPARAM>(myParams.get()))
        myParams.release();

    ...
}

ReceivingMethod::OnMsgId( WPARAM wParam, LPARAM lParam)
{
    std::unique_ptr<MyParams> myParams( reinterpret_cast<MyParams*>(lParam) );
    ... // use object
}

Answer 2

One way:

1. Derive your Params from std::enable_shared_from_this<Params> or from a shared-from-this enabled base class.

2. Pass a raw pointer in the message.

3. At the receiving end, call p->shared_from_this() to obtain a new shared_ptr.

Note that for posting a message, as opposed to sending, you need to ensure that the object is still valid when the message is processed.

One way to do that can be use a static shared_ptr. To ensure correct protocol you can then limit the accessibility of shared_from_this and wrap it in a getter that nulls out the static shared_ptr. Disclaimer: I haven't done that so there may be issues.

Answer 3

If this is not a speed-critical part of your application, then a solution could be to create a singleton that would hold every shared_ptr you want to pass.

When you need to pass a shared_ptr, you would store it in this singleton and get in return an ID that refers to it. This is this ID that you would transmit with PostThreadMessage.

On the other side, when the message is received, you can retrieve the shared_ptr using this ID. Of course, the singleton must delete its reference to the shared_ptr.

Pros:

• this technique is quite easy to implement and should be compatible with your existing code,
• using a singleton to store the shared_ptr will ensure that they cannot be destroyed before the message is transmitted.

Cons:

• what happens if a message is never received? (is it even possible with PostThreadMessage?) You have to make sure the shared_ptr won't stay forever in your singleton.
• accessing a shared_ptr requires to find it in a std::map (it could become slow if you have lots of messages at the same time),
• since this is a multithreaded code, you need to use mutexes to secure the access to the map (shared_ptr are thread-safe),
• you will need an object that can store any kind of shared_ptr and that can be stored in a std::map. Think about having a template class that can store any shared_ptr, and that would derive from a non-template base class (I don't remember the name of this trick).
• obviously this technique won't work if you need to send messages between different processes.

Answer 4

Problem resolved. I wrapped the std::make_unique to logParams.release() statements in the revised sample code above in a mutex. No leaks reported after adding those two statements.

What the statements do is keep the errorLogger thread from deleting the unique object before its ownership is released.

There is still a problem though.

When commenting out the added 2 mutex statements, there are still no leaks. They should have re-appeared. Not only did the leaks get fixed by adding the statements, but it fixed permanently the leaks in both instances on my dev machine. The second instance's code had not been altered. This problem looks like a compiler bug. Fortunately, I have a nightly backup on another machine and it still exhibits the leak behavior.

What is clear though, is, Remy Lebeau's unique_ptr approach to passing smart ptrs via windows message calls works as long as the objects are wrapped in a mutex. He warned that the objects would leak if there were lifetime issues.

Answer 5

@ Remy Lebeau and @rtischer8277 have so far submitted two answers to my original posting. The former used std:unique_ptr and the latter used std::shared_ptr. Both answers work but have the same limitation. @Rem Lebeau puts it like this: …so that you can release() the pointer while it is in flight, and then obtain new ownership of it once the message is processed…, which breaks the spirit of the smart pointer idiom. What is needed is a solution that works like the Remote Procedure Call (RPC) which has been around since the beginning of computing (RPC, DCE, CORBA, SQL and databases, and GOOBLE/BING/YAHOO searches, not to mention all Web page links and queries to date). It is obvious that there is clear motivation for an easily programmed cross-thread RPC MFC functionality. SendMessage is RPC for same-thread calls on objects derived from CWnd (aka, “window”). The solution I am providing here is just such a non-workaround solution which I am calling “SendThreadMessage”.

Below you will see a project that demonstrates this capability. Background: PostThreadMessage has two “overloads”. One that works with windows and uses the called thread’s m_hThreadID in its signature, and the other is CWinThread::PostThreadMessage and does not contain any windows (read: CWnd derived) classes. A practical example and clear explanation of the former overload is shown in CodeProject’s PostThreadMessage Demystified by ThatsAlok.

A second workaround for the illusory “SendThreadMessage” functionality is to simply send a message to the other thread and when that procedure is finished, send one back. Gorlash: … I used a two-message system for this, where the main thread sent a message to the task-handler thread, and when the task-handler was done it sent another message back to the caller (both of these being user-defined messages)…, and OReubens: … a PostThreadMessage, and a Post back (either to window or as thread message) is the safer/better/more controllable way out. Note: SendMessageCallback is not a solution because the callback is not synchronous for cross threads.

The two “overloads” have caused much confusion and there have been no definitive code examples of the CWinThread::PostThreadMessage overload. I have seen no solution that is not tedious compared to a clean synchronous RPC call.

So here is my solution: don’t ever release ownership of the unique_ptr object until it naturally goes out of scope. Use <future> to create a promise that is automatically passed over to the other thread via the LPARAM parameter. In the receiving thread, execute the code (ie, the remote procedure) using whatever the data is needed that was sent in the members of the passed-in object. When the remote thread is finished processing and setting its output data in that same object, use promise’s set_value to signal the waiting future object in the first thread that the call has completed and the results are stored in the unique_ptr object. This effectively simulates SendMessage synchronous RPC functionality using the simple modern C++ future construct and without having to work around or defeat C++’s normal ownership semantics.

Here is a link to the commented VS2015 project that demonstrates how this works: SendThreadMessage.sln

SendThreadMessage.sln is a VS console project with ATL and MFC checked. The console output shows unique_ptrs being created and going out of scope called pingThread and pongThread. The mainfrm thread is blocked while the ping and pong threads alternate sending a message to each other (with one second intervals). Each PostThreadMessage with futures demonstrates the “SendThreadMessage” synchronous RPC functionality using plain MFC and modern C++.

Answer 6

In Edit2: I identified a con with @Remy Lebeau’s answer: the MyParams object needs to be std::unique_ptr-created and then transferred and then re-owned on the destination thread. Also, my original motivation of wanting to use std::shared_ptrs across the PostThreadMessage remained in my application. In my OP code I did new the MyParams object, which is what @Remy Lebeau’s answer solved. The original code should have been:

struct MyParams { MyParams() {} int nn { 33 }; }; // doesn’t go out of scope w/r/t receiving method
...
std::shared_ptr< MyParams > myParams = std::make_shared< MyParams >();
...
PostThreadMessage( MSG_ID, 0, reinterpret_cast< LPARAM >( myParams.get() );

And here is the receiving code:

void ReceivingMethod::OnMsgId( WPARAM wParam, LPARAM lParam )
{
  std::shared_ptr< MyParams > myParamsX( reinterpret_cast< MyParams* >( lParam ) ); // error
  myParamsX->nn++; // ok, nn: 35
} // throws exception when going out of scope

Since weak_ptrs do not affect ownership, then I should be able to transfer its pointer across threads using PostThreadMessage’s lParam, I reasoned. There were several weak_ptr constructors that were available. Use the std::weak_ptr< A >&(ref) constructor. Here is the receiving code:

void ReceivingMethod::OnMsgId( WPARAM wParam, LPARAM lParam )
{
  std::weak_ptr< MyParams > myParamsX( reinterpret_cast< std::weak_ptr< MyParams >& >( lParam ) ); // ok: nn:34 but strong and weak refs reporting are random numbers
  //std::weak_ptr< MyParams > myParamsX( reinterpret_cast< std::weak_ptr< MyParams >& >( lParam ) ); // ok: also works
  myParamsX.lock()->nn++; // ok: nn: 35
  int nnX = myParamsX.lock()->nn; // ok: nnX: 35
} // ok: weak_ptr releases resource properly, but intellisense strong and weak ref counts are still bad

I tested incrementing myParam’s and myParamX’s nn members and both the myParamsWptr.lock()->nn++ and myParams->nn++ could increment the smart object. This time releasing the myParams object didn’t fail. I assume because myParamsWptr locks the object, no thread access problems will occur.

As expected there were no leaks in the program.

Why continue using PostThreadMessage at all? To communicate across windows user-interface (message pump) threads you use PostThreadMessage. I am still looking for Modern C++ techniques that are as good as that. std::promise, std::future and std::get_future work fine with worker threads which don’t have message pumps. In the meantime, I use PostThreadMessage in my MFC C++ apps.