0

I am unsure if I should use "celery" or "rq".

I am looking for a light-weight solution and my gut feeling told me that importing celery will be much slower than importing rq.

But the opposite is true. At least on my device:

> time python -c 'import rq'

real 0m0,115s
user 0m0,101s
sys  0m0,014s

> time python -c 'import celery'

real 0m0,035s
user 0m0,034s
sys  0m0,001s

I measured this several times - same result.

My prejudice was wrong. Why is importing rq three times slower?

Since somebody asked for it in a comment:

time python -c 'import huey'

real    0m0,096s
user    0m0,083s
sys     0m0,014s
guettli
  • 25,042
  • 81
  • 346
  • 663
  • 2
    It depends on what the module does when you import it. But importing is only done once when you start your script. Does this minimal time difference during import really matter? – Matthias Apr 08 '21 at 19:06
  • why answer now when you can wait for the bounty ;) – Chris_Rands Apr 08 '21 at 19:08
  • @Matthias during developing I start python (or to precise pytest) over and over again (unfortunately (AFAIK) it is not possible to just reload the code in a long running interpreter). This means the interpreter starts again and again. I want this to be fast. – guettli Apr 08 '21 at 19:30
  • Why does the import speed matter to you? It is done once when the process starts anyway... – DejanLekic Apr 09 '21 at 10:40
  • @DejanLekic see my above comment. – guettli Apr 09 '21 at 11:07
  • 1
    Since you are comparing Celery and rq, I suggest you try [Huey 2](https://huey.readthedocs.io/en/latest/) as well. – DejanLekic Apr 09 '21 at 11:22
  • @DejanLekic I added it. – guettli Apr 09 '21 at 11:32

1 Answers1

2

Roughly speaking, it has to do with the "for free" instantiation of a LocalStack in rq.local. There isn't a similar complex thread-safe object that is created when importing celery. That said you may see different times when you actually construct the application objects for each framework.

2ps
  • 15,099
  • 2
  • 27
  • 47