Commons FTPClient InputStream from multiple files in a single connection

Question

My requirement is to get list of files of required file extension from different directories and while getting the files itself i should get the input stream and do some processing.... I am trying to get the list of files with required file extension, then I am trying to get input stream from all the selected files in that list. For the first iteration, i am getting the required files and also i am getting the stream for one file but for the next file, i get a null pointer exception. I am also unable to get the list of files from the second iteration

Sample code that i used to test:

        System.out.println(ftp.printWorkingDirectory());
        boolean status = ftp
                .changeWorkingDirectory("mydirectory");
        System.out.println("Status of Change Directory:" + status);
        System.out.println(ftp.printWorkingDirectory());
        InputStream is = null;

        System.out.println(ftp.printWorkingDirectory());
        System.out.println(ftp.isConnected());

        FTPFile[] list2 = ftp.listFiles();
        System.out.println("Number of files in this directory:"
                + list2.length);
        for (int i = 0; i < list2.length; i++) {
            System.out.println("-------[" + list2[i].getName()
                    + "]---------------------");
            is = ftp.retrieveFileStream(list2[i].getName());
            int bytesRead = 0;
            byte[] buffer = new byte[1024];
            while ((bytesRead = is.read(buffer, 0, buffer.length)) > 0) {
                System.out.println(new String(buffer, 0, bytesRead));
            }
            //is.close();

            System.out.println("-------[END:" + list2[i].getName()
                    + "]---------------------");
        }

The count of files is shown as 3 and the first file in the 'mydirectory' is read correctly but when it tries to read the second file it says null pointer exception .... also, after i read the stream using retrieveFileStream method, if i try to print current working directory, it gives null value but says it is still connected....

Please let me know if i have some bugs in my code.... The only workaround for me was to connect to the ftp location for every input stream read which is not a good thing to do....

score 8 · Answer 1 · answered Mar 20 '15 at 05:06

  System.out.println(ftp.printWorkingDirectory());
        boolean status = ftp
                .changeWorkingDirectory("mydirectory");
        System.out.println("Status of Change Directory:" + status);
        System.out.println(ftp.printWorkingDirectory());
        InputStream is = null;

        System.out.println(ftp.printWorkingDirectory());
        System.out.println(ftp.isConnected());

        FTPFile[] list2 = ftp.listFiles();
        System.out.println("Number of files in this directory:"
                + list2.length);
        for (int i = 0; i < list2.length; i++) {
            System.out.println("-------[" + list2[i].getName()
                    + "]---------------------");
            is = ftp.retrieveFileStream(list2[i].getName());
            int bytesRead = 0;
            byte[] buffer = new byte[1024];
            while ((bytesRead = is.read(buffer, 0, buffer.length)) > 0) {
                System.out.println(new String(buffer, 0, bytesRead));
            }
            is.close();
            while(!ftpClient.completePendingCommand());
            System.out.println("-------[END:" + list2[i].getName()
                    + "]---------------------");
        }

while(!ftpClient.completePendingCommand()); To finalize the file transfer you must call completePendingCommand() and check its return value to verify success.

There are a few FTPClient methods that do not complete the entire sequence of FTP commands to complete a transaction. These commands require some action by the programmer after the reception of a positive intermediate command. After the programmer's code completes its actions, it must call this method to receive the completion reply from the server and verify the success of the entire transaction. This is what the commons API says, http://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/ftp/FTPClient.html#retrieveFileStream(java.lang.String)

score 0 · Answer 2 · answered Nov 12 '12 at 23:12

There are two optimizations that I would suggest. First, I would recommend using FTPClient.listNames. This will return an array of Strings. Secondly, use IOUtils.readFully(inputStream) that returns a String representation of contents in the file.

Now coming to the problem in hand, I assume you are executing these commands in locl passive mode. include this in your code above ftp.retreiveFileStream.

ftp.enterLocalPassiveMode();
ftp.setFileType(FTPClient.BINARY_FILE_TYPE);
ftp.setRemoteVerificationEnabled(false);

Commons FTPClient InputStream from multiple files in a single connection

2 Answers2