How to run class in separated thread Python?

Question

I have two classes in Python script. One of is Main() and the second is Loading()

class Main:
   pass

class Loading:
   pass

At first, works Main() that return was filled dictionary

Then is created instance of Loading() that iterates all images and downloads them:

## LOAD IMAGES ##
imageLoader = Loading()
imageLoader.save()

So, the problem is when I call this script it creates one major thread that is waiting for the end of imageLoader = Loading().

As a result, a major thread works so long, it invokes 502 Server error.

How to run imageLoader = Loading() in a separate background thread that to release major thread?

What will be launched first in this code:

LOADED_IMAGES = {}
IMAGES_ERRORS = []
IMAGES = {"A": "https://images.aif.ru/009/299/3378e1a1ab2d1c6e6be6d38253dd3632.jpg", "B": "http://static1.repo.aif.ru/1/77/623957/b99ee5f894f38261e4d3778350ffbaae.jpg"}

    excel = Excel()
    excel.readExcel(file_path, 'Main')

    imageLoader = ImageLoader()
    Thread(target=imageLoader.run().save()).start()

Does it work line by line or Thread will be created immediately?


**This is full code:**

    class ImageLoader:
        def run(self):
            for article, image in IMAGES.items():
                if image is None or image == '':
                    continue
                LOADED_IMAGES[article] = self.loadImage(self.replaceHttpsProtocol(image), '/home/o/oliwin4/jara/public_html/image/catalog/s/')

        def replaceHttpsProtocol(self, url):
            return url.replace("https:", "http:")

        def nameNameGenerate(self):
            return int(round(time.time() * 1000))

        def extention(self, path):
            ext = path.split(".")[-1]
            return '.' + ext if ext else "jpg"

        def save(self):
            for article, image in LOADED_IMAGES.items():
                self.add(article, image)

        def add(self, article, image):
            Products.update(image=image).where(Products.sku == article).execute()

        def loadImage(self, path, path_folder):
            try:

                filename = str(self.nameNameGenerate()) + str(self.extention(path))
                wget.download(url=path, out=path_folder + filename)
                return 'catalog/s/' + filename

            except BaseException as e:
                IMAGES_ERRORS.append(str(e))

Using:

def runOnThread():
    imageLoader = ImageLoader()
    imageLoader.run()
    imageLoader.save()

if __name__ == "__main__":
     Thread(target=runOnThread, daemon=True).start()

After you used the threaded code, what is the error you getting? — forayer, Feb 18 '18 at 18:12
Also in your code you have this piece of code above your class definition, isn't that making the code run twice? " imageLoader = ImageLoader() Thread(target=imageLoader.run().save()).start()" — forayer, Feb 18 '18 at 18:13

forayer · Accepted Answer · 2018-02-21T05:30:03.543

0

You need to look for which line is blocking your code to run it in a separated thread, usually the blocking line is some kind of I/O or expensive computation.

To do it you can use the threading module.

So, assuming that your blocking line is the

imageLoader.save()

Try to run it in a separated thread with this code.

from threading import Thread

Thread(target=imageLoader.save()).start()

edited Feb 21 '18 at 05:30

answered Feb 18 '18 at 16:07

forayer

367
2
10

How to put this to `Thread`? `imageLoader = Loading()` and then call `imageLoader.save()`? – Feb 18 '18 at 16:11
I don't know the insides of your Loading() class, does it get blocked only on the line imageLoader.save()? If so, you can use Thread(target=imageLoader.save()).start(). It will run this line in another thread. – forayer Feb 18 '18 at 16:14
Be careful, because this will run separately, so if you wan't to do something after this thread finish its job, you need to wait for it to finish. To do that you use the method join(). (ex.: t1 = Thread(target=some_function()).start() and them t1.join() -> this wait the thread to finish. So usually you start a bunch of threads and them wait them all to finish to do some work. – forayer Feb 18 '18 at 16:19
Can you modify your answer for my case? – Feb 18 '18 at 16:24
OK, I did it :) – forayer Feb 18 '18 at 16:28
Yes it is. You can use it without downloading anything. – forayer Feb 18 '18 at 16:34
I dont see where you create instance `imageLoader()`? – Feb 18 '18 at 16:43
You can create it just before you run the thread. It is ok. Instance it normally. imageLoader = Loading() – forayer Feb 18 '18 at 16:44
Move to chat please – Feb 18 '18 at 16:45
Is it possible to run like this: `Thread(target=imageLoader.run().save()).start()`? – Feb 18 '18 at 16:49
I mean first method from class then second `save()`? – Feb 18 '18 at 16:49
Also, will be it run after finished `Main()`? – Feb 18 '18 at 16:52
It's difficult to tell without seeing your code. I don't know what is blocking you, if it's when you instantiate your Loading class, or when you run the method "save". – forayer Feb 18 '18 at 16:53
Look this example from stackoverflow https://stackoverflow.com/questions/15365406/run-class-methods-in-threads-python – forayer Feb 18 '18 at 16:54
Try to do it as a function. create a new function which instantiate Loading and them save. And use this function as a target to thread, it should work. – forayer Feb 18 '18 at 17:02

Omar Einea · Answer 2 · 2018-02-18T17:09:51.597

0

As mentioned, you can use Python's threading module for this. Though, a thread takes a reference to a function (passing target a function call is useless / wrong).

In your case, if you want to both instantiate then run a function on an object in a separate thread, you should put these two in a function:

def runOnThread():
    imageLoader = Loading()
    imageLoader.save()

Then pass a reference of this function to a new thread, like so (notice no ()):

from threading import Thread
Thread(target=runOnThread).start()

If you don't want the main thread to wait for the new thread to finish, you could make it a Daemon thread, like so:

Thread(target=runOnThread, daemon=True).start()

Shorter version of all the above:

from threading import Thread
Thread(target=lambda: Loading().save(), daemon=True).start()

edited Feb 18 '18 at 17:09

answered Feb 18 '18 at 16:50

Omar Einea

2,478
7
23
35

I get twice call of ` Loading()` I dont know why – Feb 18 '18 at 17:29
@Amely show the code you've just tested (on pastebin or something) – Omar Einea Feb 18 '18 at 17:30
Does not work, it call twice `run()` also it falls in second iteration in loop: `for article, image in IMAGES.items():` – Feb 18 '18 at 17:48
I think problem is in `wget.download` it calls exception and thread is falled – Feb 18 '18 at 17:49
this will deferentially only call whatever function you pass it once, your loop might be iterating twice. Anyway, that's out of your original question. – Omar Einea Feb 18 '18 at 17:54

How to run class in separated thread Python?

2 Answers2