Limiting Update Frequency / Purging Update Cache

Question

I have a User Interface that receives asynchronous notifications from a Service that prompts it to re-read information from a Database. The async notifications contain a primary key that can be used to retrieve the modified record. Under load, I might get 10 or 15 notifications a second, but frequently with duplicated IDs. Something like this:

{u'callback': u'job.modify', u'job-id': 1090, u'timestamp': u'2012-01-26 09:50:04.766'}
{u'callback': u'job.modify', u'job-id': 1091, u'timestamp': u'2012-01-26 09:50:04.767'}
{u'callback': u'job.modify', u'job-id': 1090, u'timestamp': u'2012-01-26 09:50:04.780'}
{u'callback': u'job.modify', u'job-id': 1091, u'timestamp': u'2012-01-26 09:50:04.808'}
{u'callback': u'job.modify', u'job-id': 1090, u'timestamp': u'2012-01-26 09:50:04.812'}
{u'callback': u'job.modify', u'job-id': 1090, u'timestamp': u'2012-01-26 09:50:04.829'}
{u'callback': u'job.modify', u'job-id': 1088, u'timestamp': u'2012-01-26 09:50:04.831'}
{u'callback': u'job.modify', u'job-id': 1088, u'timestamp': u'2012-01-26 09:50:04.836'}
{u'callback': u'job.modify', u'job-id': 1091, u'timestamp': u'2012-01-26 09:50:04.846'}

It would seem worthwhile to skip a few of the database reads in cases like this. I'm working with a class, so my thought was to configure some sort of refresh interval. This is what I have so far:

class myClass():
  def __init__(self):
    self.modified = set([])
    self.lastrefresh = datetime.datetime.now()
    self.refreshinterval = datetime.timedelta(milliseconds = 250)

  def onModify(self, data):
    self.modified.add(data['job-id'])
    if datetime.datetime.now() - self.lastrefresh < self.refreshinterval:
      return
    self.doModify()

  def doModify():
    ids = list(self.modified)
    self.lastrefresh = datetime.datetime.now()
    self.modified.clear()

Which works (mostly) but carries the potential to have a few updates left over. Because these updates are received async I never know how many or how often they'll show up. Any pending refreshes are handled on the next notification, but if the next notification doesn't come in a timely manner then ids sit in the modified set for much longer than the 250ms interval I want. Any suggestions are much appreciated.

g.d.d.c · Accepted Answer · 2012-01-26T20:16:38.190

I think I found a way to make this work. It requires the following modifications to the Class:

class myClass():
  def __init__(self):
    self.modified = set([])
    self.lastrefresh = datetime.datetime.now()
    self.refreshinterval = datetime.timedelta(milliseconds = 250)

  def onModify(self, data):
    self.modified.add(data['job-id'])
    if datetime.datetime.now() - self.lastrefresh < self.refreshinterval:
      return
    self.doModify()

  def doModify():
    if not self.modified:
      return
    ids = list(self.modified)
    self.lastrefresh = datetime.datetime.now()
    self.modified.clear()
    wx.CallAfter(self.purgeModifies)

  def purgeModifies(self):
    wx.CallLater(250, self.doModify)

It feels a little bit hokey to use wx.CallAfter and then immediately call wx.CallLater, but the onModify is happening in a background thread, which means that wx.CallLater throws a C++ Assertion Error. wx.CallAfter returns control to the Main Thread, which allows wx.CallLater to work. This gets me my "one more run" to handle any pending updates.

Limiting Update Frequency / Purging Update Cache

1 Answers1