PHP: read a remote file (ideally using fopen)

Question

I'd like to read a remote text file (ideally using fopen) using PHP. My script works using fopen when I'm using this function on a local file.

I've tried:

$file = fopen ("http://abc.abc.abc", "r");
if (!$file) {
    echo "<p>Unable to open remote file.\n";
    exit;
}

and I got:

Warning: fopen(http://abc.abc.abc): failed to open stream: No connection could be made because the target machine actively refused it. in C:\xampp\htdocs\NMR\nmrTest5.php on line 2 Unable to open remote file.

I've read that phpseclib could be a good option and since I can access to my files using WinSCP (SFTP) or by using Puttyfor I tried this (after copying all the files from phpseclib to my directory) hoping that I could copy locally the file and then read it with fopen (not the best thing for met but I could live with that):

include('Net/SFTP.php');

$sftp = new Net_SFTP('abc.abc.abc');
if (!$sftp->login('username', 'pass')) {
    exit('Login Failed');
} 

and I got:

Notice: No compatible server to client encryption algorithms found in C:\xampp\htdocs\NMR\Net\SSH2.php on line 1561 Login Failed

Interstingly, I got a different message if I was connected to the server (using WinSCP):

Notice: Error reading from socket in C:\xampp\htdocs\NMR\Net\SSH2.php on line 3362

Notice: Connection closed by server in C:\xampp\htdocs\NMR\Net\SSH2.php on line 1471 Login Failed

Any idea on how I could get it to work? Ideally I would use fopen but I'm open to other solution.

Answer 1

I've just been working through this exact problem myself and couldn't find any good documentation in any one single place for how to accomplish this.

I have just made a logging service that uses Monolog and basically makes a custom stream handler based on the log files that are being written to/created. As such it requires a resource (such as one created by fopen) in order to write the log files to an SFTP server.

I had it working using the ssh2 library like this:

$connection = ssh2_connect($this->host, 22);
ssh2_auth_password($connection, $this->user, $this->password);

$sftp = ssh2_sftp($connection);

//some stuff to do with whether the file already exists or not

$fh=fopen("ssh2.sftp://$sftp".ssh2_sftp_realpath($sftp,".")."/$this->logName/$this->fileName", 'a+');

return new StreamHandler($fh);

Everything was working beautifully until I went to integrate the service into a different project and realised this was only working on my development machine because it has the libssh2 library installed as outlined in this question.

Unfortunately, the production server is not so easy to add libraries to. I therefore found myself looking for a different solution.

I have used phpseclib in other projects but only for basic get(), put() and some nlist() calls.

In order to get this working I had to use a Stream object. Not very well documented but there is a good discussion here.

Based on the info there, plus some digging around in the SFTP class, particularly the get() function, this is how I managed to achieve the same functionality using phpseclib

SFTP\Stream::register();
$sftpFileSystem = new SFTP($this->host);
if (!$sftpFileSystem->login($this->user, $this->password)) {
    throw new Exception("Error logging in to central logging system. Please check the local logs and email for details", 1);
}

$context = [
    'sftp' => [
        'sftp' => $sftpFileSystem
    ],
];

//some stuff to do with whether the file already exists or not
$remote_file = $sftpFileSystem->realpath('test.txt');

$sftpStream = fopen("sftp://.{$remote_file}", 'a+', null, stream_context_create($context));
if (!$sftpStream) {
    exit(1);
}

return new StreamHandler($sftpStream);

note the dot (.) after the sftp:// in the call to fopen(). It wasted me a good half an hour!

Answer 2

This is how I ended fixing my problem usng phpseclib as suggested by @neubert in the comments of my question.

I first added the phpseclib folder on my server. Then I used this code in my PHP file to get access to my file on a remote server:

//needed for phpseclib
set_include_path(get_include_path() . PATH_SEPARATOR . 'phpseclib');
include_once('Net/SFTP.php');

//connection to the server
$sftp = new Net_SFTP('abc.abc.abc');
if (!$sftp->login('my_login', 'my_password')) {
  exit('Login Failed');
}

//dump entire file in a string, convert to an array and check number of lines

else {
  $text = $sftp->get('full_path_to_my_file');
}
$myArray = explode("\n", $text);
$nbline = count($myArray);

Answer 3

I have faced similiar issues with fopen. Curl is useful for these purposes.

Please check with the following basic example function(if the url is https, please uncomment the CURLOPT_SSL_VERIFYPEER = FALSE line).

$url = '***THE URL***';
$result = get_web_page_by_curl($url);

if ($result['errno'] != 0) echo 'error: bad url, timeout, redirect loop ...';
if ($result['http_code'] != 200) echo 'error: no page, no permissions, no service ...';
else {
    $page = $result['content'];
    echo $page;
}

function get_web_page_by_curl($url) {
    $agent = "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)";

    $options = array(
        CURLOPT_RETURNTRANSFER => true,     // return web page
        CURLOPT_HEADER         => false,    // don't return headers
        CURLOPT_FOLLOWLOCATION => true,     // follow redirects
        CURLOPT_ENCODING       => "",       // handle all encodings
        CURLOPT_USERAGENT      => $agent,   // who am i
        CURLOPT_AUTOREFERER    => true,     // set referer on redirect
        CURLOPT_CONNECTTIMEOUT => 120,      // timeout on connect
        CURLOPT_TIMEOUT        => 120,      // timeout on response
        CURLOPT_MAXREDIRS      => 10,       // stop after 10 redirects
        //CURLOPT_SSL_VERIFYPEER => FALSE   // this line makes it work under https 
    );

    $ch = curl_init($url);
    curl_setopt_array($ch, $options);
    $content = curl_exec($ch);
    $err     = curl_errno($ch);
    $errmsg  = curl_error($ch);
    $header  = curl_getinfo($ch);
    curl_close($ch);

    $header['errno']   = $err;
    $header['errmsg']  = $errmsg;
    $header['content'] = $content;
    return $header;
}