Wrapping libcurl with std::streambuf for uploads?

Question

I've implemented the read half of the std::streambuf API (i.e., underflow() and xsgetn()) around libcurl so that an ordinary istream can have such a streambuf plugged into it and retrieve the contents from HTTP or FTP servers. It works great.

Now I'd like to implement the write half of the API (i.e., overflow() and xsputn()) for uploading files to an FTP server to do something like:

ostream os( my_curl_streambuf );
curl_easy_setopt( my_curl_handle, CURLOPT_UPLOAD, 1L );
os << is->rdbuf(); // copy entire contents of istream "is" to ostream "os"

The problem is that libcurl uses a "pull" API via a read-callback function set via CURLOPT_READFUNCTION whereas the last code line above is a "push" way of doing it.

Can this be made to work? I.e., wrap libcurl with the streambuf API for both reading and writing?

It can be done, but no matter how you do it, you're stuck with the fact that sockets are asynchronous, and streams are synchronous, so no matter *how* you do it, it's probably not going to work well enough to justify the work. — Jerry Coffin, Dec 28 '13 at 00:04
Care to elaborate on the "It can be done" part? I'll be the one to decide whether it's worth the work. — Paul J. Lucas, Dec 28 '13 at 00:06
One possible starting point would be: http://codereview.stackexchange.com/q/14389/489. — Jerry Coffin, Dec 28 '13 at 00:12
That code is *only* for reading -- which my code already does (and does it better since it handles time-outs). I specifically asked about *writing*. — Paul J. Lucas, Dec 28 '13 at 00:20
Hmm...I seemed to recall its having a `write` and a `writecallback` (or something on that order) that looked like they were for writing. If I mis-remembered (or misread), I apologize. — Jerry Coffin, Dec 28 '13 at 00:25

score 0 · Answer 1 · answered Dec 28 '13 at 01:12

I don't know enough about the libcurl API but I would expect that it pulls data until all data is exhausted. Since IOStreams have a push interface which won't be married easily with a pull interface it seem you'd best implement a stream which buffers all output until it is complete and at this point have libcurl pull for the readily available buffer. You'd probably have sync() do nothing and instead provide a custom function, say, send() on your stream which is the trigger to hand over control to libcurl until it is down with its business.

The alternative approach would be to use a thread for libcurl to pull on and another thread to write to the stream and feed the bits which are written to libcurl when it pulls for data.

Wrapping libcurl with std::streambuf for uploads?

1 Answers1