0

Suppose I need to retrieve a job status, and I spawn a new thread to handle the job computation:

def post_view(request):
     g = Greenlet.spawn(heavy_job, arg1, arg2)
     return json.dumps(default_response)

and then in heavy_job

def heavy_job(...):
   # call a 3rd party service to start doing its job
   job_id = (....)

   request.session['heavy_job_status'] = 'Running'
   status = 'Running'
   while status == 'Running':
       # ask the 3rd party guy "are you still running"
       resp = 3rdparty-service.query(job_id, ....)
       if resp != 'Running':
          return resp
       time.sleep(5)  # we will monkey patch the time with gevent's timer so no locking

Is this while loop approach really bad for scaling? Or would I rather query the 3rd party service as each ajax request comes in?

User007
  • 1,519
  • 6
  • 22
  • 35
  • In python, `for` loops are faster than `while` loops. – Ashwini Chaudhary Aug 29 '12 at 19:23
  • You are suggesting a generator? I can write a simple generator that has `3rdpart-servier.query(...)` inside, and the generator takes in the `job_id`. But can it handle 2 users at the same time? – User007 Aug 29 '12 at 19:26
  • 8
    @AshwiniChaudhary: `>>> import timeit >>> tfor = timeit.Timer("for i in l: t += i", "t, l = 0, range(10)") >>> twhile = timeit.Timer(""" while i < 10: t += i i += 1""", "t = i =0") >>> tfor.timeit() 0.676494836807251 >>> twhile.timeit() 0.05109286308288574 >>> ` – Joel Cornett Aug 29 '12 at 19:34
  • Polling is usually not good at all for scaling. You cannot get a callback from this 3rd party ? – gbin Aug 29 '12 at 21:39

1 Answers1

1

Sorry, I can't write comments yet, so I write here a note about the comment by Joel Cornett with example of timing for 'while' and 'for'.

There is a mistake in this example: timeit() methods executes initializing code only once, but the main code -- many times, so with 'while' loop every execution except the first one does nothing because the value of 'i' is already 10. You can change 10 for 100 or 1000 and se no difference in timing (but not with 'for' loop). You can also add 'i = 0' before 'while' and see very different result.

But, I think, it doesn't matter for your code since 3rdparty-service.query() probably takes significantly more time than the difference between 'for' and 'while' loops.

arilou
  • 465
  • 3
  • 6