1

I use curl in php to make whois requests. Most of the time this works fine, but the occasional whois server rejects curl requests because they use an invalid format.

This is a working example:

curl -X "who-is.ga" whois.my.ga:43

This on the other hand does not work:

curl -X "ikea.eu" whois.eu:43

The EU whois server rejects the format, because (it appears) to include all the headers in the request, ie:

WHOIS ikea.eu / HTTP/1.1User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2Host: whois.eu:43Accept: /

While most whois servers parse the request until \r\n, some take the entire request into account.

So I thought to remove the additional headers, but was only able to remove the user agent and host.

curl -X "ikea.eu" --header "Host:" --header "Accept:" --user-agent "" whois.eu:43

But the headers "/" and "HTTP/1.1" remain.

% WHOIS ikea.eu / HTTP/1.1 -7: %Invalid pattern

Any suggestions of how to mute all headers, effectively only sending the contents of "-X" or how to individually drop unwanted headers?

NB: I use curl for whois requests, because it allows me to use proxies. I intentionally left this part out of the above examples.

clund
  • 99
  • 1
  • 2
  • 8
  • 1
    Note that `whois` is a protocol transported over TCP, not HTTP. So using an HTTP tool to query it, like `curl` will come with difficulties. See also https://stackoverflow.com/a/45286777/6368697 – Patrick Mevzek Jan 02 '18 at 16:49

1 Answers1

4

The first server, whois.eu appears to work because it stops reading when it encounters whitespace. The other, whois.eu reads all the input from the socket and uses that as your query.

Best bet is probably to use cURL in telnet mode to send a single domain and read the response.

This works for me on most WHOIS servers I tried:

echo ikea.eu | curl telnet://whois.eu:43

It should be noted, that cURL writes one byte at a time to the socket in telnet mode rather than all data at once. This might cause problems with some servers that don't wait for a newline to terminate input (e.g. whois.tucows.com:43) and instead execute the query once they've recv'd any data.

In those cases you can use netcat:

echo yahoo.com | nc whois.tucows.com 43

It also supports proxies:

echo yahoo.com | nc -X5 -x127.0.0.1:1080 whois.tucows.com 43
drew010
  • 68,777
  • 11
  • 134
  • 162
  • Thanks for the tip, netcat seems like the way to go and it works like a charm without the proxy, however, when I add an external proxy I get: nc: Permission denied. Any ideas on how to fix this? – clund Aug 01 '16 at 20:49
  • Is it a SOCKS proxy? You might need to try `sudo` depending on where the permission denied error is coming from. – drew010 Aug 01 '16 at 21:51
  • Connecting using netcat (no proxy) works fine, but with a proxy I get no response back. I added the verbose switch and got this "Connection to whois.tucows.com 43 port [tcp/nicname] succeeded!". Any idea what could be going wrong? – clund Sep 22 '16 at 17:28
  • It appears that NC terminates before getting any feedback from the remote host (when using proxies). So adding the -i (interval switch) and setting it to 1 sec. helps, but slows down the lookup considerably. My version of NC does not seem to accept MS (milliseconds) as input. Is there another way I can force NC to wait for the response? – clund Sep 22 '16 at 17:57
  • 1
    This has been changed if anyone is curious. cURL telnet mode now sends multiple bytes at a time so these servers with such a request can now work as of version 7.54.1 – richardhsu Jul 24 '17 at 17:24
  • 1
    @richardhsu Thanks for the heads up and for contributing that change to cURL. Much better! – drew010 Jul 24 '17 at 23:19
  • Still valid in 19? – Jackie Degl'Innocenti Apr 23 '19 at 11:12
  • 1
    @JackDegl'Innocenti Talking to the servers hasn't changed, but post-GDPR, information returned by most registrars will be protected. – drew010 Apr 23 '19 at 15:06