0

I want to periodically (every hour?) query the Python Package Index API from Plone. Something equivalent to:

$ for i in `yolk -L 24 | awk '{print $1}'` # get releases made in last 24 hours
do
  # search for plone classifier
  results=`yolk -M $i -f classifiers  | grep -i plone`
  if [ $results ]; then
    echo $i
  fi
done

Results:

collective.sendaspdf
gocept.selenium
Products.EnhancedNewsItemImage
adi.workingcopyflag
Products.SimpleCalendarPortlet
Products.SimpleCalendar

Then I want to display this information in a template. I would love to, at least initially, avoid having to persist the results.

How do I display the results in a template without having to wait for the query to finish? I know there are some async packages available e.g.:

But I'm not sure what the general approach should be (assuming I can schedule an async task, I may need to store the results somewhere. If I have to store the results, I'd prefer to do it "lightweight" e.g. annotations)

aclark
  • 4,345
  • 1
  • 19
  • 31

1 Answers1

1

How about the low, low tech version?

Use a cron-job to run the query, put this in a temp file, then move the file into a known location, with a timestamp in the filename.

Then, when someone requests the page in question (showing new packages), simply read the newest file in that location:

filename = sorted(os.listdir(location))[-1]
data = open(os.path.join(location, filename)).read()

By using a move, you guarantee that the newest file in the designated location is always a complete file, avoiding a partial result being read.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Clever! Thanks. I'm not ruling out low tech, but I think I'm leaning towards something AJAX-ish. – aclark Nov 02 '12 at 14:28
  • @aclark: You can dress this up as much as you like, but if what you were looking for is hourly updates, then for the underlying retrieval the cron-job approach with a file-move is still best. The AJAX-y frontend just needs to poll a view that'll return data that is newer than the last-polled value passed in, for example. – Martijn Pieters Nov 02 '12 at 14:30
  • Perhaps. I'm still thinking about hourly updates vs. only updating when a request is received. – aclark Nov 02 '12 at 14:59
  • @aclark: In which case you need something asynchronous. Perhaps kick of a task that does the same thing as I described, but instead of a cron-job it's a background script instead. In any case, using the filesystem for an async storage is nice and simple. Beyond that, it'll require a `plone.app.async` task and ZODB storage for essentially transient data. – Martijn Pieters Nov 02 '12 at 15:02