Problem description
I have an executable which repeatedly posts data to an HTTPS endpoint, using libcurl with libcares. This is getting occasional DNS resolution timeouts on some clients (not all). If I run the equivalent command in command-line curl, I never see any timeouts.
What makes this even more confusing is that the host is explicitly specified in /etc/hosts, so there shouldn't be any DNS resolution required.
The error from libcurl (with verbose mode) is:
* Adding handle: conn: 0xcbca20
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 88 (0xcbca20) send_pipe: 1, recv_pipe: 0
* Resolving timed out after 2002 milliseconds
* Closing connection 88
My libcurl executable is sending 2-3 queries a second, and I see this error about once every 300 requests. Using command-line curl, I have run 10000 queries without a single timeout.
Can anyone suggest anything I can try to resolve these errors from libcurl? Are there any settings which I need to add to my libcurl setup, or system configuration I might be missing?
I wasn't sure whether to put this in Stack Overflow, Server Fault, or Ask Ubuntu; apologies if it is in the wrong place.
Thanks for your time!
More detailed information
The client is Ubuntu 12.04, 64-bit. The same problem has been observed on several clients, all with the same OS.
Usernames/passwords/urls have been obfuscated in the following snippets.
Command-line curl tester (using v 7.22.0):
while true; do curl -v -u username:password "https://myhost.com/endpoint" -X POST --data "a=x&b=y" >> /tmp/commandLine.log 2>&1; sleep 0.1; done &
Libcurl source code (using curl 7.30.0, with c-ares 1.10.0):
#include <curl/curl.h>
#include <unistd.h>
#include <string>
#include <iostream>
using namespace std;
int main(int argc, char** argv) {
while (1) {
// Initialise curl
CURL *curl = curl_easy_init();
// Set endpoint
string urlWithEndpoint = "https://myhost.com/endpoint";
curl_easy_setopt(curl, CURLOPT_URL, urlWithEndpoint.c_str());
// Set-up username and password for request
curl_easy_setopt(curl, CURLOPT_USERPWD, "username:password");
// Append POST data specific stuff
string postData = "a=x&b=y";
long postSize = postData.length();
curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, postSize);
curl_easy_setopt(curl, CURLOPT_COPYPOSTFIELDS, postData.c_str());
//set timeouts
curl_easy_setopt(curl, CURLOPT_TIMEOUT, 10);
curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 2);
curl_easy_setopt(curl, CURLOPT_DNS_CACHE_TIMEOUT, 60);
cout << endl << endl << "=========================================================" << endl << endl;
cout << "Making curl request to " << urlWithEndpoint << "(POST size " << postSize << "B)" << endl;
// Set curl to log verbose information
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1);
// Perform the request
CURLcode curlRes = curl_easy_perform(curl);
// Handle response
bool success = false;
if (curlRes == CURLE_OK) {
long httpCode;
curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
success = (httpCode==200);
cout << "Received response " << httpCode << endl;
} else if ( curlRes == CURLE_OPERATION_TIMEDOUT ) {
cout << "Received timeout" << endl;
} else {
cout << "CURL error" << endl;
}
curl_easy_cleanup(curl);
if (success) {
cout << "SUCCESS! (" << time(0) << ")" << endl;
usleep(0.1*1e6);
} else {
cout << "FAILURE!!!! (" << time(0) << ")" << endl;
usleep(10*1e6);
}
}
}
