0

I am trying to download a large binary file via SCP using ssh2 crate v0.3.3.

extern crate ssh2;

use std::net::TcpStream;
use ssh2::Session;
use std::path::Path;
use std::fs::File;
use indicatif::ProgressBar;

fn main() -> Result<(), Box<std::error::Error>> {
    let ssh_host_port = "prokerala.com:22";
    let ssh_user = "prokeral";
    let remote_temp_file = "/tmp/dbimport_Rk6Iwwm5.sql.bz2";

    let tcp = TcpStream::connect(&ssh_host_port).unwrap();
    let mut sess = Session::new().unwrap();
    sess.handshake(&tcp).expect("SSH handshake failed");
    let _ = sess.userauth_agent(ssh_user);

    let path = Path::new(&remote_temp_file);
    let (mut remote_file, stat) = sess.scp_recv(path).unwrap();

    let stream:ssh2::Stream = remote_file.stream(1);

    // Update: solved by using io::copy as suggested by @apemanzilla below
    // let mut target = File::create("/tmp/done.txt").unwrap();
    // let pb = ProgressBar::new(stat.size());
    // std::io::copy(&mut pb.wrap_read(remote_file), &mut target)?;
    Ok(())
}

I want to write stream to a file, and show a progress bar. Is it possible to write stream to a file directly, without reading the data into memory with stream.read() within a loop?

Shepmaster
  • 388,571
  • 95
  • 1,107
  • 1,366
Joyce Babu
  • 19,602
  • 13
  • 62
  • 97
  • I don't think it's possible you need to copy at least one time the data to the user space cause that how ssh2 is working, it must probably do some transformation on the data from the socket and so need a buffer to write encrypted data on it. – Stargateur Jul 21 '19 at 15:04
  • @Stargateur I am using the latest version `v0.3.3`. The question is not really about an issue with the above code, but on how to do something in Rust. i.e. how to write `stream:ssh2::Stream` to a file without using a loop. I included the above code only to show the type of `stream` variable. – Joyce Babu Jul 21 '19 at 15:35
  • 2
    @Stargateur Thank you. I fixed the error while creating the minimal reproducible example as you suggested :) `io::copy` had to be called with `remote_file` (`ssh2::Channel`) and not `remote_file.stream()` (`ssh2::Stream`). – Joyce Babu Jul 21 '19 at 16:19
  • It's great that you have the solution to your question! You should post it as an answer rather than an edit to your question and then potentially accept that answer. That way, the question shows up as solved in search results, people can vote on your answer, and your solution can be more helpful to future people with the same problem. – Shepmaster Jul 22 '19 at 23:17

1 Answers1

3

Since Stream implements Read, you should be able to just use std::io::copy(&mut stream, &mut File::create(...).unwrap()) to copy all the data into a file. If you want a progress bar, I'd recommend using the indicatif crate, which has a method to wrap a Read instance, automatically updating a progress bar as data is read.

apetranzilla
  • 5,331
  • 27
  • 34
  • This isn't doing what the question ask "without reading the data into memory with stream.read() within a loop?" `copy` use a buffer, https://doc.rust-lang.org/src/std/io/util.rs.html#43-63 – Stargateur Jul 21 '19 at 14:59
  • @Stargateur You're going to need to buffer the data somewhere if you want to efficiently copy data, and it won't get much easier or faster than what the standard library already provides. What exactly is the issue with having data in memory? – apetranzilla Jul 21 '19 at 15:03
  • I don't have one, but actually your answer is wrong, it doesn't answer the OP question. I'm not an expert in ssh2, but just know that the answer "no, you can't" is perfectly acceptable if it's true. copy is probably the way to go but that need a buffer data. – Stargateur Jul 21 '19 at 15:06
  • @apemanzilla The code hangs at `std::io::copy(&mut stream, &mut file);`. There is nothing written to the target file. – Joyce Babu Jul 21 '19 at 15:26
  • @apemanzilla Thank you. `io::copy` and `indicatif::ProgressBar` worked perfectly for me. – Joyce Babu Jul 21 '19 at 16:39