python3 how to delete thread object from list

Question

I'm writing a script that should run indefinitely and put some things in a db every few seconds, using threading. This works, but I see the process' memory slightly increasing every few seconds and I think this is because the list that holds all the Thread objects is never emptied. How would I do that? The join is put under the is_alive condition so it doesn't take any time away from spawning the next threads. The below example results in an

AttributeError: 'Thread' object has no attribute 'kill'

import threading
import time

def dicthing(dic, x):
    dic.update({x: x*x})
    time.sleep(0.01)

dic = {}
threads = []
x = 100
while x > 0:
    t = threading.Thread(target = dicthing, args = (dic, x))
    threads.append(t)
    t.start()
    x -= 1

    if x % 50 == 0:
        print(len(threads), len(threading.enumerate()))

    for t in threads:
        if t.is_alive() == False:
            t.join()
            t.kill = True
    threads = [t for t in threads if not t.kill]

I'd like the output to be:

1 1

1 1

Use loop or list comprehension to filter threads that aren't alive anymore — Michael Butscher, Oct 09 '17 at 22:01
Rather than create a new thread for every `x` you might want to make a fixed pool of threads (ThreadPool) and sharing the work between them. — 101, Oct 09 '17 at 22:02
@cᴏʟᴅsᴘᴇᴇᴅ Won't doing that delay the creation of following threads? — r0edie, Oct 09 '17 at 22:09
@MichaelButscher I tried that, following this #1 answer: https://stackoverflow.com/questions/4067786/python-checking-on-a-thread-remove-from-list but it didn't work for me. I assumed it was a python2 thing? — r0edie, Oct 09 '17 at 22:09
@101 As an inexperienced (python) programmer they looked scarier to me than this, plus I once timed it vs this method and saw this was faster. — r0edie, Oct 09 '17 at 22:09
You should add to your question what exactly you tried and how it failed. — Michael Butscher, Oct 09 '17 at 22:11

score 1 · Accepted Answer · answered Oct 11 '17 at 14:00

Last lines (for-loop and on) can be written either simple as:

    threads_alive = []
    for t in threads:
        if t.is_alive() == False:
            t.join()
        else:
            threads_alive.append(t)

    threads = threads_alive

or if you have to process the already dead threads somehow yet:

    threads_alive = []
    threads_dead = []
    for t in threads:
        if t.is_alive() == False:
            t.join()
            threads_dead.append(t)
        else:
            threads_alive.append(t)

    threads = threads_alive
    for t in threads_dead:
        ... postprocess dead threads here ...

I stored the dead threads by index with enumerate because I needed that to remove it from the list, but my solution was more or less the same. Thanks! — r0edie, Oct 20 '17 at 13:43

python3 how to delete thread object from list

1 Answers1