Unable to close a stream opened with pycurl

Question

I am working on a client for a web service using pycurl. The client opens a connection to a stream service and spawns it into a separate thread. Here's a stripped down version of how the connection is set up:

def _setup_connection(self):
    self.conn = pycurl.Curl()
    self.conn.setopt(pycurl.URL, FILTER_URL)
    self.conn.setopt(pycurl.POST, 1)
    .
    .
    .
    self.conn.setopt(pycurl.HTTPHEADER, headers_list)
    self.conn.setopt(pycurl.WRITEFUNCTION, self.local_callback)

def up(self):
    if self.conn is None:
        self._setup_connection()
    self.perform()

Now, when i want to shut the connection down, if I call

self.conn.close()

I get the following exception:

error: cannot invoke close() - perform() is currently running

Which, in some way makes sense, the connection is constantly open. I've been hunting around and cant seem to find any way to circumvent this problem and close the connection cleanly.

see http://stackoverflow.com/questions/6031921/how-to-halt-kill-stop-or-close-a-pycurl-request-on-a-stream-example-given-using — , Oct 02 '11 at 13:59

score 1 · Answer 1 · answered Sep 24 '10 at 16:42

1

It sounds like you are invoking close() in one thread while another thread is executing perform(). Luckily, the library warns you rather than descending into unknown behavior-ville.

You should only use the curl session from one thread - or have the perform() thread somehow communicate when the call to perform() is complete.

answered Sep 24 '10 at 16:42

Jeremy Brown

17,880
4
35
28

I am using a signal sent to the thread containing the pycurl connection to initiate the self.conn.close() call. The issue is, barring errors and timeouts, perform() never is complete, information will keep streaming until either party closes the connection (I am interfacing with the twitter stream api, if that helps at all). – boldfield Sep 24 '10 at 18:00

score 0 · Accepted Answer · answered Nov 12 '10 at 01:42

You obviously showed some methods from a curl wrapper class, what you need to do is to let the object handles itself.

def __del__(self):
    self.conn.close()

and don't call the closing explicitly. When the object finishes its job and all the references to it are removed, the curl connection will be closed.

Unable to close a stream opened with pycurl

2 Answers2