2

I am using python twisted for my real time project.

Explanation: I am pulling the html pages for the url to which i give request to my remote machine.So,I am maintaining scheduler in my database machine which sends request to the remote machines and fetches the result asynchronously in twisted by posting the url param.

So, In twisted i run the main method in "LoopingCall service" for every two minutes whether we get the result success callback chain or err call back chain.I will start my main method every 2 mins.

In this scenario i get error when one request gets failed to reach back.The other remote machines are still available for the service.So, here whatever case happened my main method should be called every two minutes.It is not happening due to some interrupted error.

     from twisted.internet.task import LoopingCall
     from twisted.internet import reactor
     from twisted.internet import defer
     from twisted.web import client

    def service_avail_check():
        ip_list = get_all_ws_hosts()
        for i in ip_list:
           url = "http://" + i[0] + ":8080/dm/"
           print url
           server_ip = i[0]
           client\
             .getPage(url)\
             .addCallback(service_avail_handler, server_ip)\
             .addErrback(service_avail_error, server_ip)

     def service_avail_handler(data, server_ip):
         if data.strip() == 'pong':
             mark_the_service_avail(server_ip)
             crawler_avail_check(server_ip)
         else:
            mark_the_service_unavail(server_ip)


     def service_avail_error(failure, server_ip):
          print >> sys.stderr, "Error:", failure.getErrorMessage()
          print 'server ping error sever ip: '+ server_ip
          mark_the_service_unavail(server_ip)

    lc = LoopingCall(service_avail_check)
    lc.start(120)
    reactor.run()

Explanation:

1.service_avail_check is my main method.it fetches the remote machine and sends the request
2.based on the response the corresponding call back is called
3.I am calling other methods to update into memory

I get the error sometimes if the client doesn't respond well.

 crawl_handle Error: Connection was closed cleanly.
 Traceback (most recent call last):
 Failure: twisted.internet.error.ConnectionDone: Connection was closed cleanly.
 None

Even though the service_avail_check should be called again because the remote machine is flushed and again it could be ready to provide service after few minutes.So, I have call it frequently.

But it doesn't get called instead it gets hung up very much time.

could you please help to this problem?

Nava
  • 6,276
  • 6
  • 44
  • 68

1 Answers1

4

Add an errback to the Deferred returned by LoopingCall.start which at least logs whatever error is happening. LoopingCall stops once its callable raises an exception. Adding an errback will show you what the exception is and give you a place to put some code to restart the LoopingCall.

Jean-Paul Calderone
  • 47,755
  • 6
  • 94
  • 122
  • sorry i beg your pardon,We have to assign the deferred call back in each call back incase any error the restarting logic has to ported in each call back wherever we get error.am i correct? – Nava Jan 03 '13 at 16:32
  • 4
    To start with, you just have to do `lc.start(120).addErrback(twisted.python.log.err)`. If that reveals something interesting, then you just need to replace `twisted.python.log.err` with an errback that has a bit more logic in it, like (for example) calling `lc.start` again (and again adding an errback). – Jean-Paul Calderone Jan 03 '13 at 16:44