0

I have a model (PurchaseOrder - abbreviated PO) holding a time budget. Users can add hour records to this budget, where each hour record reduces the remaining budget.

I implemented signals for updating the remaining budget. After adding an hour record the budget is reduced accordingly. Because the calculation is potentially time consuming I used a thread for this task. 

def update_po_remaining_value(sender, instance, **kwargs):
    CalculatePOThread(sender, instance).start()   
post_save.connect(update_po_remaining_value, sender=HourRecord)       
post_delete.connect(update_po_remaining_value, sender=HourRecord)

The thread CalculatePOThread is calculating the remaining PO budget value via getting the hour record set and deducting the total hour record set from the budget

hr_set = HourRecord.objects.filter(purchase_order = po)

At my dev.workspace this works perfectly fine. In production the post_save connect also works fine, but I'm experiencing a strange issue with the post_delete signal. It happens quite often that the sum of hour records returned by the query HourRecord.objects.filter(purchase_order = po) still includes the deleted hour record which has triggered the CalculatePOThread thread.

Anyhow I circumvented the behavior with adding a delay of 6 seconds to the thread before executing the query. time.sleep(6).

Does anybody has an idea why this situation can occur? it seems like the post_delete signal is triggered before the record is really deleted from the database..!? But this would be a bug in Django and this would be my last guess.

Thomas Kremmel
  • 14,575
  • 26
  • 108
  • 177

1 Answers1

1

Hard to say, but my guess would be you've hit a thread-safety issue. When you spawn off a thread to handle a long-running task, you must be aware that while it's trying to complete, a similar thread could easily be fired off tasked with the same. Generally, when working with threads, you want to keep their footprint minimal, or in other words, not make them highly dependent on database state and such.

If database access is required, which it is in this case, you need to institute locks to prevent the database from being simultaneous messed with. This is going to be much more difficult though, given you're running Django 1.1. How to achieve table-level locking in Django 1.1 is going to be dependent on the database server you're running and warrants its own question.

Chris Pratt
  • 232,153
  • 36
  • 385
  • 444
  • Thanks for your answer. One additional information: I made sure that I m the only one messing around in the prod.env. and triggered just one by one - post.delete signal. Therefore I assured that no other threads are conflicting with my tests. Thus the lock should be set by the object delete command and the post_delete signal should only be triggered after the lock has been released / the object deleted. But that's Django internal stuff that I cannot influence, I guess. PS: DB = postgres, webserver = apache. – Thomas Kremmel May 07 '12 at 21:05
  • seems as this issue is also discussed here: http://groups.google.com/group/django-haystack/browse_thread/thread/c452da9acfc04c64?pli=1 – Thomas Kremmel May 07 '12 at 21:26