Status checker for hundreds IP addresses

Question

I wonder how to make a status checker, checking about 500 addresses in a few minutes? (it'll check a certain port if its listening).

But I care about the performance... so I don't know if it can be done with PHP. Waiting for your suggestions guys.

Also please give me some examples, if you think that the best way for this thing will be PHP or C#.

Ofc. I meant the TCP connection but not http, since I need to check open port for example: 11740

Edit:

Added third bounty for this question! Please post some better answer than those already posted.

Important point here - is this a TCP socket or datagram based, like UDP? — DaveRandom, Jan 21 '12 at 21:29
TCP, fe. I want to check if a IP address if alive on port 80. — Cyclone, Jan 21 '12 at 21:42
Your performance bottleneck is going to be the network request, not the language you use for checking. — Brent Baisley, Feb 21 '12 at 11:20

Brent Baisley · Answer 1 · 2012-02-21T11:19:06.760

This is very doable in PHP and you could check 500 IP in a few seconds. Use mutli-curl to send out many requests at once (i.e. 100 or all 500). It will take only as long as the slowest IP to respond. But you may want to set a reasonable curl connect timeout of a few seconds. Default network timeout is 2 minutes as I recall. http://php.net/manual/en/function.curl-multi-exec.php

Note that I'm not saying PHP is your best choice for something like this. But it can be done fairly quickly and easily in PHP.

Here is a full code example. I tested it with real URLs and it all worked. The $results array will contain just about all the stats you can get from a curl request. In your case, since you just care if the port is "open" you do a HEAD request by setting CURLOPT_NOBODY to true.

$urls    = array(
    'http://url1.com'=>'8080',
    'http://url2'=>'80',
    );
$mh        = curl_multi_init();
$url_count    = count($urls);
$i        = 0;
foreach ($urls as $url=>$port) {
    $ch[$i] = curl_init();
    curl_setopt($ch[$i], CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch[$i], CURLOPT_NOBODY, true);
    curl_setopt($ch[$i], CURLOPT_URL, $url);
    curl_setopt($ch[$i], CURLOPT_PORT, $port);
    curl_multi_add_handle($mh, $ch[$i]);
    $i++;
}
$running    = null;
do {
    $status    = curl_multi_exec($mh,$running);
    $info     = curl_multi_info_read($mh);
} while ($status == CURLM_CALL_MULTI_PERFORM || $running);

$results= array();
for($i = 0; $i < $url_count; $i++) {
    $results[$i]     = curl_getinfo($ch[$i]);
    // append port number request to URL
    $results[$i]['url']    .= ':'.$urls[$results[$i]['url']];
    if ($results[$i]['http_code']==200) {
        echo $results[$i]['url']. " is ok\n";
    } else {
        echo $results[$i]['url']. " is not ok\n";
    }
    curl_multi_remove_handle($mh, $ch[$i]);
}

curl_multi_close($mh);
print_r($results);

Nice point as well, I always forget about the curl_multi functions! — drew010, Jan 21 '12 at 21:43
That is completely incorrect. For example, curl has a number of FTP specific options (CURLOPT_FTPSSLAUTH). The CURLOPT_PROTOCOLS option alone lists LDAP, SCP, TELNET, DICT, FILE. — Brent Baisley, Feb 01 '12 at 02:23
@BrentBaisley Sorry for such a long reply delay. Could you please show me an example to check status with addresses and ports from array? It should look like this: `$data = array('ip_address' => 'port');`, thank you! — Cyclone, Feb 14 '12 at 14:45
It's already there, check the first line of the code: `$urls = array( 'http://url1.com'=>'8080', 'http://url2'=>'80', );` — Rizky Ramadhan, Feb 21 '12 at 11:17

score 13 · Answer 2 · edited Jan 21 '12 at 21:32

13

Your best bet might be to use a dedicated port scanner like nmap. I don't know the command-line options off the top of my head, but it should be possible to output a results file and just parse it from PHP.

edited Jan 21 '12 at 21:32

DaveRandom

87,921
11
154
174

answered Jan 21 '12 at 21:29

CoderMD666

997
1
8
16

DaveRandom · Answer 3 · 2012-01-21T22:33:20.097

Here is a way to do this very quickly in PHP using the sockets extension, by setting all the sockets to non blocking. From a purely networking point of view, this is the most efficient way, since this simply tests the TCP connectivity and does not exchange any actual data. Tested on PHP/5.2.17-Win32.

<?php

  // An array of hosts to check
  $addresses = array(
    '192.168.40.40',
    '192.168.5.150',
    '192.168.5.152',
    'google.com',
    '192.168.5.155',
    '192.168.5.20'
  );
  // The TCP port to test
  $testport = 80;
  // The length of time in seconds to allow host to respond
  $waitTimeout = 5;

  // Loop addresses and create a socket for each
  $socks = array();
  foreach ($addresses as $address) {
    if (!$sock = socket_create(AF_INET, SOCK_STREAM, SOL_TCP)) {
      echo "Could not create socket for $address\n";
      continue;
    } else echo "Created socket for $address\n";
    socket_set_nonblock($sock);
    // Suppress the error here, it will always throw a warning because the
    // socket is in non blocking mode
    @socket_connect($sock, $address, $testport);
    $socks[$address] = $sock;
  }

  // Sleep to allow sockets to respond
  // In theory you could just pass $waitTimeout to socket_select() but this can
  // be buggy with non blocking sockets
  sleep($waitTimeout);

  // Check the sockets that have connected
  $w = $socks;
  $r = $e = NULL;
  $count = socket_select($r, $w, $e, 0);
  echo "$count sockets connected successfully\n";

  // Loop connected sockets and retrieve the addresses that connected
  foreach ($w as $sock) {
    $address = array_search($sock, $socks);
    echo "$address connected successfully\n";
    @socket_close($sock);
  }

/* Output something like:

Created socket for 192.168.40.40
Created socket for 192.168.5.150
Created socket for 192.168.5.152
Created socket for google.com
Created socket for 192.168.5.155
Created socket for 192.168.5.20
4 sockets connected successfully
192.168.40.40 connected successfully
192.168.5.150 connected successfully
google.com connected successfully
192.168.5.20 connected successfully

*/

Sorry for the delay in answer, but it doesn't work as expected: when I add fe. `i-does-not-exist.com.uk` the script returns: `1 sockets connected successfully i-does-not-exist.com.uk connected successfully` , no matter what `port` I use - 1 or 99999 , it still returns that socket is connnected... — Cyclone, Jan 25 '12 at 16:55
@Cyclone Port `99999` does not exist - the maximum valid TCP port is `65535` - the port number is represented in IPv4 by a 16-bit integer. I only tested this code briefly on my local network and found that it worked as expected, I will do some more playing around with it and see if I can repeat/rectify your problem with it when working over the internet. — DaveRandom, Jan 25 '12 at 18:32
@DaveRandom Could you try to fix your code please? it seems not work correctly, even if the port is off, the codes always return successfully connect, i really appreciate your help. just please try, that's very important to me and maybe for other. thank you. — user2203703, Dec 02 '13 at 17:31
@DaveRandom could you check [this Q](http://stackoverflow.com/questions/20360167/tcp-connect-by-using-socket-return-false-results) please? it's about your answer and +100 bounty are there! thanks — user2203703, Dec 06 '13 at 05:35

cha0site · Accepted Answer · 2012-02-22T11:42:44.790

6

The best way to this would be nmap, as mentioned in other answers.

You'd want to run it like this (-PN is don't ping, -sP means to skip the port scan and just check if the host is up, -PS80 means to check port 80, -n is not to do reverse DNS lookup, -oG - is to output in machine readable format, and the other arguments are IP addresses or hostnames):

nmap -PN -sP -PS80 -n -oG - --send-ip IP1 IP2 ...

And it would look like this:

$ nmap -n -PN -sP -PS80 -oG -  209.85.147.104 87.248.122.122 4.4.4.4
# Nmap 5.21 scan initiated Tue Feb 21 01:07:20 2012 as: nmap -n -PN -sP -PS80 -oG - 209.85.147.104 87.248.122.122 4.4.4.4 
Host: 209.85.147.104 () Status: Up
Host: 87.248.122.122 () Status: Up
Host: 4.4.4.4 ()    Status: Down
# Nmap done at Tue Feb 21 01:07:21 2012 -- 3 IP addresses (2 hosts up) scanned in 0.95 seconds

You could run and parse this from PHP with no trouble. I'm not very experienced in PHP, and haven't tested this, but here's some example code:

<?php
$output = shell_exec('nmap -n -PN -sP -PS80 -oG - --send-ip ' . implode(" ", $ips));
$result = array();
foreach(preg_split("/\r?\n/", $output) as $line) {
    if (!(substr($line, 0, 1) === "#")) {
        $info = preg_split("[\t ]", $line);
        $result[$info[2]] = ($info[5] === "Up");
    }
}
?>

Mind you, writing PHP code or C# code or whatever that does this isn't a big deal, it's just that nmap is very very good at what it does, and extremely versatile, that writing code that does this is reinventing the wheel. If you do decide to go that route, make sure you make your code asynchronous, otherwise one slow server would slow down your entire sweep.

edited Feb 22 '12 at 11:42

answered Feb 20 '12 at 23:01

cha0site

10,517
3
33
51

O_o , nice and its pretty fast also! But there is a problem with it: I guess `-PS80` and `80` is a port number - if so, then its not working properly since no matter what port I set, it will show that the host is up. – Cyclone Feb 22 '12 at 11:37
@Cyclone: Are the servers on your local LAN by chance? If so, then nmap defaults to using ARP for checking if the host is up as it's much faster. You need to use `--send-ip` to tell it to always try to connect. I'll edit my answer to include this. – cha0site Feb 22 '12 at 11:39
No, I'm creating an game servers list, and want to check if each is online or not. My test mode: `1021 IP addresses (248 hosts up) scanned in 39.95 seconds`, its a pretty good performance also, I think 40 second isnt that much for such request? – Cyclone Feb 22 '12 at 11:50
I have used your edited shell command: `nmap -n -PN -sP -PS12345 -oG --send-ip xxx.xxx.xxx.xxx` , please note I've set the port to the: `12345` which is not up on the host, but it still shows its `Up`. – Cyclone Feb 22 '12 at 11:54
@Cyclone: The port is closed, but the _host_ is up. Meaning, it replied to the connection request with a `RST`. Try something like `nmap -p 12345 -PS12345 -n -oG - --send-ip xxx.xxx.xxx.xxx` and see what that tells you =) – cha0site Feb 22 '12 at 12:02
Used `nmap -p 12345 -PS12345 -n -oG - --send-ip my-ip` and it still printed that the host is `Up`. – Cyclone Feb 22 '12 at 12:06
@Cyclone: Yes, but on the next line it printed something like `12345/closed/tcp//netbus///`. Meaning, the host is up but the port is _closed_. If you try it with 80, you'll likely get something like `80/open/tcp//http///`, meaning the host is up and the port is _open_. – cha0site Feb 22 '12 at 12:09
ah, I see, but isnt there a way to check the ports only? Because I would like to use your example. – Cyclone Feb 22 '12 at 12:12
@Cyclone: You can still use the example, you'll just have to improve it a bit. Specifically, you can check if `$info[4] === "Ports:"`, then do a `explode` on slashes, and check if the second array member is `"open"`. – cha0site Feb 22 '12 at 12:16
Last question: is it possible to check two ports status at once in one request? Fe. `1700` and `80`. – Cyclone Feb 22 '12 at 12:49
@Cyclone: Sure, use `-p80,1700` and `-PS80,1700` in the commandline, but notice how it changes the output. – cha0site Feb 22 '12 at 19:09

score 4 · Answer 5 · answered Jan 21 '12 at 21:37

As mentioned by kz26, nmap from command-line would be your best option. With PHP functions like system, exec, shell_exec, etc to capture the results for processing.

This guide should help you to get started http://www.cyberciti.biz/tips/linux-scanning-network-for-open-ports.html

score 2 · Answer 6 · answered Jan 21 '12 at 21:33

2

C# would be a better option as you can use threads to speed up the process.

In PHP, you can essentially check if a TCP port is open by using fsockopen().

$host = 'example.com';
$port = 80;
$timeout = 30;

$fp = @fsockopen($host, $port, $errno, $errstr, $timeout);
if (!$fp) {
    echo "Port $port appears to be closed on $host  Reason: $errno: $errstr.\n";
} else {
    echo "Port $port is open on $host\n";
    fclose($fp);
}

As kz26 said, you could also get something like nmap to check the ports on a bunch of hosts and use php to call it and process the results as well.

answered Jan 21 '12 at 21:33

drew010

68,777
11
134
162

In order for this to be true, you need to have a multi core procesor. – JanLikar Jan 21 '12 at 21:36
8

Even on a single core processor, creating several threads will speed the process up as you can initiate multiple connections simultaneously and not have to wait for one to finish before checking the next. – drew010 Jan 21 '12 at 21:38
1

You're right, I'm sorry for my mistake. I don't know how it happened, but I forgot that the question is about networking. – JanLikar Jan 21 '12 at 21:42
3

drew010: The usual way to do this is with asynchronous APIs, not threads! Anyway, nmap already contains the appropriate logic, so... – SamB Jan 31 '12 at 20:28
I used this script to create an application, it worked very fine with a bit of modification. https://portscanner.standingtech.com/ – Akam Mar 02 '17 at 21:14

score 1 · Answer 7 · answered Jan 21 '12 at 21:35

PHP should be able to check 500 ips in pretty short amount of time, but it really depends on your internet connection and other specs. Other tools written in lower languages would be a little faster, but I think that php is good for the job. You must just set php execution limit to a suitable value before trying

score 1 · Answer 8 · answered Jan 29 '12 at 13:35

They keyword in this is non-blocking. You would want to have a certain callback when an attempt is succesful and be able to run more calls in the meantime.

Some functions like sockets from DaveRandom's answer allow for a non-blocking mode to do this in regular php installations. You can also use different programming languages such as node.js to do this very efficiently because they are designed to be non-blocking.

score 1 · Answer 9 · answered Feb 01 '12 at 13:39

The by far most efficient way to do this is to use raw sockets, however this requires administrative privileges and some boilerplate coding. You choose a local port as the source, send a SYN packet to port 80 of each target address, and collect the return packets on the source port. If they have an ACK flag, the port is open for connections. If they have RST or they don't return, the port isn't open/the host is dead.

Obviously you will need to manually wait for whatever your "timeout" of choice is as this method goes around the OS network stack. This also requires you to construct the TCP and IP headers manually, though, and I doubt it can be done from PHP. It can certainly be done in C, and I believe C# also supports raw sockets (haven't done it in C# myself).

This is extremely fast and doesn't pollute the system with a bunch of open connections or threads.

Nice solution, it its impossible for me without any example :/. — Cyclone, Feb 22 '12 at 10:58

score 0 · Answer 10 · answered Feb 21 '12 at 12:16

The best thing you can do is start a number of threads (you decide the number; probably more than 10 and less than a 100). Doing scanning on different threads will speed up your check. I am not sure if you can do multi-threading in php. If not, then c# would be better.

Status checker for hundreds IP addresses

10 Answers10

Linked