Python 3.6 - How to start a thread from within a daemon thread and have sub-thread exit once the function it is running completes

Question

I am learning Python (3.6) and I am writing some code in a functional style (not using classes) - I have functions which are started as daemon threads by a manager() function.

I need to start an additional function in its own thread, that is passed arguments from within one of the functions that is started by the manager function as a daemon thread, as it throws an exception if it is started from within the manager function, as the argument which it expects does not exist when it is started...

Here is a representation of my code and the problem I am having - I want to be able to start func4 in its own thread and pass it an argument from within func3... func3 will call func4 a large number of times, so I need the thread in which func4 runs to die as soon as soon as the code completes...

import threading

threads = []


def func1():
    // runs in its own thread and receives messages from a message queue

def func2():
    // runs in its own thread and sends messages in a loop to a message queue

def func3():
    // runs in its own thread and receives messages from a message queue, does some
    // processing, assigns values to a variable then passes variable to func4


def func4(passed_variable_from func3):
    // performs actions on variable passed from func3 and I would like it to 
    // run in its own thread... IS THIS POSSIBLE?

def manager():

# Thread t1 
t1 = threading.Thread(target=func1)
t1.daemon = True
threads.append(t1)

# Thread t2
t2 = threading.Thread(target=func2)
t2.daemon = True
threads.append(t2)
t2.start()

# Thread t3 
t3 = threading.Thread(target=func3)
t3.daemon = True
threads.append(t3)
t3.start()

t1.start()
for t in threads: 
    t.join()

manager()

mhsmith · Answer 1 · 2018-01-11T01:00:00.100

1

func4, like all the other thread functions, will cause its thread to die as soon as the function returns, so nothing special is required to make that happen.

However, if func3 will call func4 a large number of times, then starting a new thread for each call may be inefficient. Especially if you want to write in a functional style, it may be better to use a higher-level API such as ThreadPoolExecutor to dispatch your calls. This also gives you a clean way to retrieve each call's return value, using the result or add_done_callback method as appropriate.

edited Jan 11 '18 at 01:00

answered Jan 11 '18 at 00:52

mhsmith

6,675
3
41
58

Thanks I will check out your suggestion and come back to you - in the meantime, func3 process messages from a queue and passes an argument to func4 for each message (up to 20 per second...) I'm not sure that func4 i running in its own thread as I havent started it as such and cant run it as a daemon thread as the fact it takes an argument causes an exception if I try to start func4 from manager... (please forgive my newbie-ness) – Mark Smith Jan 11 '18 at 01:16

score 1 · Accepted Answer · answered Jan 12 '18 at 15:22

After much searching and investigation I was able to find a solution to the issue by using information in this stackoverflow answer on threading

Basically I found that I was able to start a new thread from within a thread using the code below:

t = threading.Thread(target=target_function,args=(args))
t.daemon = True
t.start()

I kind of already knew about using this code to start threads as its more or less the same code I use to start my func1, func2 and func3 functions listed in my original question, however what I didn't realise is that this code can be called from within a function that has been started in its own thread...

Here is the code from my original question with the code directly above inserted so that func4 runs in its own thread after being started from within func3, which is also running in its own thread...

import threading

threads = []


def func1():
    // runs in its own thread and receives messages from a message queue

def func2():
    // runs in its own thread and sends messages in a loop to a message queue

def func3():
    // runs in its own thread and receives messages from a message queue, does some
    // processing, assigns value to variable_from_func3

    t = threading.Thread(target=func4,args=(variable_from_func3))
    t.daemon = True
    t.start()

    // Code above starts func4 in its own thread passing variable_from_func3 as an arg


def func4(variable_from func3):
    // performs actions on variable passed from func3  
    // This now runs in its own thread which dies once complete

def manager():

# Thread t1 
t1 = threading.Thread(target=func1)
t1.daemon = True
threads.append(t1)

# Thread t2
t2 = threading.Thread(target=func2)
t2.daemon = True
threads.append(t2)
t2.start()

# Thread t3 
t3 = threading.Thread(target=func3)
t3.daemon = True
threads.append(t3)
t3.start()

t1.start()
for t in threads: 
t.join()

manager()

I have tested this and found that it works as needed in as much as func1, 2 and 3 are started by the manager function and stop when the manger function is terminated and func4 runs in its own thread and stops when the function completes or when the manager function terminates.

I hope this will be useful to others...

Python 3.6 - How to start a thread from within a daemon thread and have sub-thread exit once the function it is running completes

2 Answers2