3

My problem is that I was assigned to modify and improve upon a program that does LZW compression. My program, as far as I know, runs fine, but relies on System.in and System.out redirection for the input file and output file. For my test program, I run through a folder full of files and for each file, run 4 different tests on the program. To emulate the command line input/output redirection, on the first iteration of the loop for each file I do the following:

FileOutputStream fos = new FileOutputStream(compressOut); // compressOut is compressed file
PrintStream ps = new PrintStream(fos);
System.setOut(ps);
FileInputStream fis = new FileInputStream(file); // file is file to be compressed
System.setIn(fis);
fis.mark(0);// mark not supported so this is actually left over from my test code -- wanted to show I tried to do this

The first time it runs the program I'm testing, it works brilliantly and returns a compression ratio and prepares to write it to the program output file.

However, upon the second program call (and, if I were to eliminate all of the extra calls, the second iteration of the for loop) it crashes and returns that the input stream is empty. I haven't gotten the chance yet to see if the output stream will do the same thing, but what I have attempted to do to reset the stream is as follows:

Before each call program call, I have the following blocks of code which I **thought would fix the problem, but to no avail:

//System.setIn(null); // Tried - didn't work
//System.setOut(null); // Tried - didn't work
fos.close();
ps.close();
fis.close();
fos = new FileOutputStream(compressOut);
ps = new PrintStream(fos);
System.setOut(ps);
fis = new FileInputStream(file);
System.setIn(fis);
//fis.reset(); // Tried - didn't work

I've tried all sorts of combinations of different ways to reset the input stream, but every solution still returns the same result, an error message indicating that it's reading from an empty input stream - an error which means that the stream is at the end of the file (or there is nothing in the file).

The only other error I can get is that mark() isn't supported when I call reset if I attempt the mark(0); and reset(); methods.

I've done reading and can't find any solid answers as to how to get this to work. The program turns the StdIn into a BufferedInputStream and StdOut to a BufferedOutputStream, so I need a method that will be compatible and provide performance similar to StdIn/StdOut redirection.

TL;DR: I need to reset my FileInputStream and can't do so using mark() reset() nor if I reinitialize the FileInputStream

Update: tried wrapping FIS into a BufferedInputStream as suggested on a similar post, where mark() is supported. However, the mark() and reset() methods are only good for the size of the BufferedInputStream (which I believe is 4096 bytes) which pretty much excludes the potential usefulness of the mark/reset feature in all of my files. They are all larger than 4kB :(

Update 2: I have decided to post the full snippet of code from where I begin the for-loop to where the empty stream error occurs, in case anyone can see something I can't. This is the code in question:

public static void main(String[] args) throws IOException, InterruptedException {
    File programOutputPath = new File("-- the compression file path --");
    File folderPath = new File("-- the relative folder path --");
    File compressOut = new File(folderPath + "compressed.lzw");
    File[] allFiles = folderPath.listFiles(); // get an array of all files
    FileWriter fw = new FileWriter(programOutputPath);
    for(File file : allFiles) {
        FileOutputStream fos = new FileOutputStream(compressOut);
        PrintStream ps = new PrintStream(fos);
        System.setOut(ps);
        BufferedInputStream bis = new BufferedInputStream(new FileInputStream(file));
        System.setIn(bis);
        String[] cArgs = new String[2];
        cArgs[0] = "-"; cArgs[1] = "-n";
        String[] dArgs = new String[1];
        dArgs[0] = "+";
        String[] LZWArgs = new String[1];
        LZWArgs[0] = "-";
        System.err.println("File: " + file.toString());
        if(!file.canWrite()) {
            System.err.println("SKIPPED FILE");
        }
        if(file.getName().equalsIgnoreCase(".gitignore") || file.getName().equalsIgnoreCase("compressed.lzw")) continue;
        MyLZW.main(cArgs); // runs fine
        if(decompress) {
            //MyLZW.main(dArgs); // not executing this
        }
        long sizeUnc = file.length();
        long sizeC = compressOut.length();
        double ratio = (double)sizeUnc/(double)sizeC; // compression ratio is correct
        System.err.println("java MyLZW - -r <" + file.getName() + "> " + " compressed.lzw compression ratio:" + ratio ); // works fine
        fw.write("java MyLZW - -n <" + file.getName() + "> " + " compressed.lzw compression ratio:" + ratio + "\n");
        cArgs[1] = "-r";
        bis.close(); // close BufferedInputStream
        bis = new BufferedInputStream(new FileInputStream(file)); // reinitialize BIS
        System.setIn(bis); // setIn to newly initialized BufferedInputStream
        MyLZW.main(cArgs); // crashes here b/c empty input stream
halfer
  • 19,824
  • 17
  • 99
  • 186
dddJewelsbbb
  • 626
  • 4
  • 17
  • “My program … relies on System.in and System.out …” Change it to rely on an InputStream and an OutputStream. It will be considerably more flexible that way, and you can pass new streams to each invocation. – VGR Oct 12 '16 at 16:43
  • @VGR Thanks for the suggestion, and if all else fails I will try this, but I'd prefer to not have to edit the dependencies the program runs on as I was given the dependencies as-is and told not to edit them. The only file I'm supposed to touch is the source file that controls the compression. This has become more of a curiosity of figuring out how to make my program work this way than an absolute necessity to make it work this way. In a nutshell, I'm trying to figure out primarily how to dynamically change StdOut and StdIn as required for future works. Thanks very much for the suggestion :) – dddJewelsbbb Oct 12 '16 at 16:55
  • If you want to re-start a FileInputStream just create a new one (or switch to an RandomAccessFile). Streams are by design meant to be read only once - the mark/reset system is just a crutch for situation where there are no alternatives or you just need to rewind a few bytes. – Robert Oct 12 '16 at 17:21
  • @Robert Thanks for the reply. I tried doing it with both a FileInputStream and BufferedInputStream. In one instance, I tried just reinitializing the already-declared FIS/BIS, and I also tried declaring a brand new one and switching the System.in to that. Both of them read an empty input stream upon the second run of the program, and I do reinit the BIS/FIS (I've tried both) and set the System.in to the appropriate reference. This has me scratching my head a lot. – dddJewelsbbb Oct 12 '16 at 17:33

1 Answers1

1

It sounds like the problem is internal to the program; it may be retaining a reference to the original value of System.in on subsequent calls.

Another possibility is that the program is clobbering the input file (replacing it with an empty file) as a side-effect.

There is no problem calling System.setIn() repeatedly, and creating a new FileInputStream as you have done will indeed read the file again from the beginning. Therefore, the problem lies in code that you have not posted.


Inside the program, the input is initialized as a static variable, like this:

private static BufferedInputStream in = new BufferedInputStream(System.in); 

That initialization happens only once, when the class is loaded. This is a bad design, and it should be fixed. Because System.in is read only once, the subsequent changes you make with System.setIn() are ignored.

If you are unable to fix it properly, you could reset that variable with reflection, but this problem could just be the tip of the iceberg of bad design. Alternatively, you could try isolating this program in its own class loader, and create a new class loader each time your master process runs the program.

erickson
  • 265,237
  • 58
  • 395
  • 493
  • Thanks, I'll seed through that and see if I can find anything. Is there anything on your mind that could cause this to retain a reference to the original System.in? That's what I thought as well, because it's definitely hitting the end of the file and not updating. I can also post my code, but I am totally aware that it is not anybody's job or obligation to seed through my code for mistakes. – dddJewelsbbb Oct 12 '16 at 17:08
  • p.s. will upvote when I get three more reputation so I can cast upvotes – dddJewelsbbb Oct 12 '16 at 17:09
  • 1
    @dddJewelsbbb It looks like `MyLZW` was designed to be run one time, in a process by itself, so it might use poor practices like stashing values in static variables. Again, this problem is inside `MyLZW`, and since you haven't posted that code, I can only guess. – erickson Oct 12 '16 at 17:35
  • You're right. I looked in the file that controls the Binary I/O and both of the Buffered(In/Out)putStreams that are initialized from System.in and System.out are declared as static variables in each of them. Is there a way to "flush" the program and have it reference its own static variables, or is it pretty much over the first time I call the program? Thanks a lot for all the help thus far. I think you hit the nail on the head. – dddJewelsbbb Oct 12 '16 at 17:39
  • 1
    @dddJewelsbbb It really depends on how those static variables are used. I am having a hard time imagining why, even though they are static, those variables wouldn't be reset each time `main` is invoked on `MyLZW`. Can you see why that is? How are those variables set initially? – erickson Oct 12 '16 at 18:15
  • MyLZW references the files BinaryStdOut.java and BinaryStdIn.java with static methods to read and write based on type. In both files, at the beginning of the class, they use Buffered[Input/Output]Streams and are initialized as such: – dddJewelsbbb Oct 12 '16 at 18:19
  • `private static BufferedInputStream in = new BufferedInputStream(System.in);` `private static BufferedOutputStream out = new BufferedOutputStream(System.out);` – dddJewelsbbb Oct 12 '16 at 18:19
  • My thoughts are the same as yours, that each time I invoke MyLZW, they should reinitialize the buffers based on the current System.in and System.out. I figured they would because I invoke the main program with different arguments. Is it probable that the static files remain the same without resetting upon re-invocation of the program? That's the only thing I can even think of that would cause this to happen. – dddJewelsbbb Oct 12 '16 at 18:22
  • 1
    @dddJewelsbbb Okay, the code makes it crystal clear: those variables are initialized only once, when the class is loaded. That's a really poor design, and I can think of only two solutions: rewrite the class so it functions properly, or use reflection to reset those variables before each run. The second option is ridiculous, and those two variables might just be the first of many symptoms to reveal themselves, all stemming from the bad design. – erickson Oct 12 '16 at 18:28
  • 1
    @dddJewelsbbb A third option is to load the program inside a separate class loader, and discard that loader after each run. That would be tricky too. – erickson Oct 12 '16 at 18:29
  • You have no clue how helpful you have been. Thanks so much for all of your advice and help. I'll get onto looking into these solutions. I'd like to be able to do this without editing the dependency files, so I might look into class loaders and reflection. If all else fails, I'll just write some batch script as a friend of mine suggested. I just feel like it's better to learn how to do something like this programatically rather than by hand because the goal is to run 14 files with 5 different compressions and gather compression ratios for each run. Thanks a lot! Infinite appreciation – dddJewelsbbb Oct 12 '16 at 18:43