0

I'm dealing with a tough one:

For testing purposes I use Apache Mina SSHD and JIMFS to mock a remote sftp server - only locally and in test environment. Never in production.

In my Spring Boot application I start a Apache Mina sshd during application startup. I create a virtual filesystem with jimfs (tried unix and osX) for the FileSystemFactory. My business code calls a remote sftp server in production and in test environment it calls the mina sshd server on localhost instead. It does nothing else than uploading a file to the sftp server.

This works very well on my machine locally but i get a "file or directory not found" error message when I run this in our test environment (a docker container).

com.jcraft.jsch.SftpException: No such file or directory
    at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2873)
    at com.jcraft.jsch.ChannelSftp._put(ChannelSftp.java:594)
    at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:475)
    at com.jcraft.jsch.ChannelSftp.put(ChannelSftp.java:365)
    at our.service.core.client.sftp.SftpClient.upload(SftpClient.java:89)

Creation of JIMFS file system during Application startup:

public class MockSftpServerInitializer implements ApplicationListener<ApplicationReadyEvent> {

  @Override
  public void onApplicationEvent(ApplicationReadyEvent event) {
    
    SshServer sshd = SshServer.setUpDefaultServer();

    sshd.setHost("localhost");
    sshd.setPort(1234);
    sshd.setCommandFactory(new ScpCommandFactory());
    sshd.setKeyPairProvider(new SimpleGeneratorHostKeyProvider(new File("host-key.ser")));
    sshd.setPublickeyAuthenticator(getPublickeyAuthenticator());
    sshd.setFileSystemFactory(getInMemoryFileSystemFactory());
    List<NamedFactory<Command>> sftpCommandFactory = new ArrayList<>();
    sftpCommandFactory.add(new SftpSubsystemFactory());
    sshd.setSubsystemFactories(sftpCommandFactory);
    
    try {
      sshd.start();
    } catch (Exception e) {
      throw new RuntimeException("Unable to start sshd", e);
    }
  }
  
  private FileSystemFactory getInMemoryFileSystemFactory() {
    return session -> getInMemoryFileSystem();
  }

  private FileSystem getInMemoryFileSystem() {
    return Jimfs.newFileSystem(
        Configuration.osX()
            .toBuilder()
            .setWorkingDirectory("/")
            .setMaxSize(1024*1024*4)
            .setDefaultAttributeValue("posix:permissions", "rwxrw-rw-")
            .setAttributeViews("basic", "owner", "posix", "unix", "acl", "user")
            .build());
  }
  

}

File upload:

public class SftpClient {

  public boolean upload(String source, String destination) {

    Session session = null;
    ChannelSftp sftpChannel = null;

    try {
      sftpProvider.addIdentity("Identity", privateKey.getBytes(), publicKey.getBytes(), null);
      session = sftpProvider.getSession(username, host, port);
      session.setConfig("StrictHostKeyChecking", "no");

      session.connect();

      Channel channel = session.openChannel("sftp");
      channel.connect();

      sftpChannel = (ChannelSftp) channel;
      sftpChannel.put(source, destination);

      log.info("Successful upload of file {} to SFTP server {}.", source, host);

      return true;
    } catch (SftpException | JSchException ex) {
      log.error("Failed upload of file {} to SFTP server {}.", source, host);
      log.error("{}", ex);
      return false;
    } finally {
      if (sftpChannel != null) {
        sftpChannel.disconnect();
      }
      if (session != null) {
        session.disconnect();
      }
    }
  }
}

I'm grateful for any hint on why this works locally but not in our docker container.

Markus
  • 1
  • 3
  • That's very hard to diagnose with the info you're giving. However, to me this looks like the source file for the PUT is not found. Did you verify the file exists (or is created in the right location) inside the container? – Paul Georg Podlech Mar 15 '22 at 08:24
  • Thank you for your answer! I'll do my best to provide all information necessary. The file is created by the spring boot application and uploaded right after creation. The upload (to remote sftp, not the mina sshd) worked for years and works locally fine - so yes, the file exists in the container ("real" filesystem, not in the jimfs). Also, the error message is generated by the sftp server, not our sftp client. – Markus Mar 15 '22 at 08:41
  • It would be best, if you could try to boil down your code to a minimal example. There's no way to understand how you start the sshd, create the file or call the PUT command. – Paul Georg Podlech Mar 16 '22 at 06:18
  • You are right. I added the server startup code (applicationListener) and the sftpClient – Markus Mar 16 '22 at 13:43
  • Which values do you use for destination? Maybe the directory you try to upload to is missing. – Paul Georg Podlech Mar 17 '22 at 06:35
  • destination is /temp/filename.csv. I also thought of this and wasn't sure if the directory has to be created before uploading a file. But it works locally on my machine but not in the docker container. A missing directory or file would make sense if the file system wasn't a virtual one. As far as I understood, it should make no difference if it runs on my machine or in the docker container. – Markus Mar 17 '22 at 08:18
  • I would try to create the directory beforehand nontheless to be sure. – Paul Georg Podlech Mar 18 '22 at 05:42
  • 1
    That actually worked! I can't really see why creating the directory manually is necessary in the container but not locally (especially as the file system is virtual). Maybe the sftp server's privileges are restricted in the container... whatever the reason is: thank you for your help!! – Markus Mar 21 '22 at 07:21

0 Answers0