1

I'm working on a python extension which spawns threads with pthread. In this minimal example the spawned threads don't join back the main thread in python. On the contrary I have no trouble with running the library when linked with a C++ executable. What is a safe way to pthread_create and pthread_joinfor python extensions?

My python_extension.cpp is:

#include <stdio.h>
#include <stdlib.h>
#include <pthread.h>

#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>


class myTestClass{

 public:
  myTestClass(){};
  void *print_message_function(void *ptr);
  void wrapperf();

}

void *myTestClass::print_message_function( void *ptr ){
  char *message;
  message = (char *) ptr;
  printf("%s \n", message);
}


void myTestClass::wrapperf()
{
  pthread_t thread1, thread2;
  char *message1 = "Thread 1";
  char *message2 = "Thread 2";
  int  iret1, iret2;

  /* Create independent threads each of which will execute function */

  iret1 = pthread_create( &thread1, NULL, print_message_function, (void*) message1);
  iret2 = pthread_create( &thread2, NULL, print_message_function, (void*) message2);

  /* Wait till threads are complete before main continues. Unless we  */
  /* wait we run the risk of executing an exit which will terminate   */
  /* the process and all threads before the threads have completed.   */

  pthread_join( thread1, NULL);
  pthread_join( thread2, NULL);

  printf("Thread 1 returns: %d\n",iret1);
  printf("Thread 2 returns: %d\n",iret2);

}


// C Wrappers for the above C++ classes                                                                                                                                
#ifdef __cplusplus
extern "C" {
#endif
  myTestClass *newcase(){ return new myTestClass};
#ifdef __cplusplus
}
#endif


// Boost python                                                                                                                                                        
BOOST_PYTHON_MODULE(mylib_py)
{
  using namespace boost::python;
  class_<myTestClass, boost::noncopyable>("myTestClass", init<std::string>())
    .def("wrapperf", &myTestClass::wrapperf)


  def("newcase", &newcase, return_value_policy<manage_new_object>());
}

My python script is as follows:

import mylib_py 
obj = newcase()
obj.wrapperf()
x = 2 # Dummy line

Sometimes the output to this is:

Thread 1
Thread 2
Thread 1 returns: 0
Thread 2 returns: 0

Sometimes it is just:

Thread 1
Thread 2

And the threads don't join at all! How could I resolve this? Could this be because my python script has already proceeded to the next line x=2 before obj.wrapperf() completes?

newkid
  • 1,368
  • 1
  • 11
  • 27
  • 1
    Your example code joins its threads before control is returned to Python. So what is your question? Are you trying to create long running threads that will overlap with Python? – Adam Aug 26 '19 at 19:32
  • 1
    Why use C headers instead of C++ headers? And why use `pthread` (unless you use a pre C++11 compiler)? – Ted Lyngmo Aug 26 '19 at 19:46
  • @TedLyngmo True, I built this minimal example from a [c example](https://www.cs.cmu.edu/afs/cs/academic/class/15492-f07/www/pthreads.html). `pthreads` because its minimal and all that I need as my threads are asynchronous. – newkid Aug 26 '19 at 19:51
  • 1
    I see. Even so, I'd jump at `` and its friends asap though. It should make your code even more minimal :-) and safer (RAII mutex locks etc). – Ted Lyngmo Aug 26 '19 at 19:57
  • do u have something between pthread_create() and pthread_join() that may throw an exception?? – TonySalimi Aug 26 '19 at 20:30
  • @Gupta No. It works fine sometimes and sometimes not! Nothing in-between generates an execution. This am confident of. – newkid Aug 26 '19 at 20:31
  • or maybe the threads throw exceptions and the main thread cannot join them. – TonySalimi Aug 26 '19 at 20:36
  • If the threads are throwing an exception would it be after creation or during? How do I systematically check when its throwing an exception if it is? – newkid Aug 26 '19 at 20:37
  • After creation. Put the whole thread body (i.e. in your case print_messsage_function() body) inside a try block and see if any exceptions are thrown or not. – TonySalimi Aug 26 '19 at 20:42
  • I confirm that no exceptions thrown – newkid Aug 26 '19 at 22:49
  • Does "the threads don't join at all" mean control does not return to Python, or are you judging that based on observed output? Or some other way? – John Bollinger Aug 27 '19 at 00:02
  • Yeah it doesn't return to python and I'm judging that based on the observed output – newkid Aug 27 '19 at 00:12

0 Answers0