14

Suppose I have 2 servers.

The first is a service that provides some computations, which can last long time (minutes to hours).

The second server will use this service to have some data computed.

I'm trying to design a REST API for the first server and so far so good. But I'd like to hear some opinion on how to model notifications when the long lasting task is finished.

I considered 2 approaches so far:

  1. Polling - the second server will ask every now and then about the result.
  2. Callback - Second server will setup an uri for the first one to call after it is done. But this smells a bit in REST API.

What do you think?

Kugel
  • 19,354
  • 16
  • 71
  • 103

3 Answers3

7

For your situation I would choose polling. When the second server makes the initial request to create the job on the first server, it should get a response that has the url of the eventual status page. The second server then polls that url every 5-15 minutes to check the status of the job. If the first server makes that url an RSS or Atom feed, then users could also point their RSS readers at the same url and find out themselves if the job is done. It's a real win when both people and machines can get information out of a single source.

Hitesh
  • 1,413
  • 1
  • 11
  • 11
4

In addition to what I've already answered in this similar question, I'd suggest using the Atom Publishing Protocol for the notification (you could publish to your second server).

Community
  • 1
  • 1
Bruno
  • 119,590
  • 31
  • 270
  • 376
  • Would it be inherently wrong to use the callback approach? I have a very similar problem (except my service can take anything between 2 and 500 seconds to respond) and using a callback seems much simpler than polling – Kritz Jun 25 '16 at 08:09
  • 1
    A brief explanation and a link to an explanation of the Atom Publishing Protocol would greatly improve this answer. – Geerten Nov 27 '17 at 16:11
1

If you use Python, you can take advantage of RabbitMQ and Celery to do the job. Celery lets you create an item in a queue and then pause execution of whatever you're running it through (i.e.: Django) such that you can consume the output of the queue processor as it becomes available. No need for polling OR callbacks.

mattbasta
  • 13,492
  • 9
  • 47
  • 68
  • Unless you understand celery (i.e. you are the author) I'd just use Rabbit MQ + pika directly and save yourself a shedload of pain (disappearing exceptions, chaining simply not working, weird compatibility issues with gevent etc.). You might get it working but if you are doing anything remotely complex celery is hiding you from loads of real work implementation details that you might need. – pip Feb 08 '16 at 12:39