110

Apache Commons IO has a nice convenience method IOUtils.toString() to read an InputStream to a String.

Since I am trying to move away from Apache Commons and to Guava: is there an equivalent in Guava? I looked at all classes in the com.google.common.io package and I couldn't find anything nearly as simple.

Edit: I understand and appreciate the issues with charsets. It just so happens that I know that all my sources are in ASCII (yes, ASCII, not ANSI etc.), so in this case, encoding is not an issue for me.

Jonik
  • 80,077
  • 70
  • 264
  • 372
Sean Patrick Floyd
  • 292,901
  • 67
  • 465
  • 588
  • 2
    About the charsets: It's still good for a library to require you to specify that you know what charset you're dealing with (e.g. `Charsets.US_ASCII`) rather than letting you say "eh, whatever charset I guess?" which all to many people seem happy to do. Especially since Java doesn't use a default that makes sense, like UTF-8. – ColinD Nov 15 '10 at 16:46
  • I know. That's why I am using UTF-8 as default version in my own answer. – Sean Patrick Floyd Nov 15 '10 at 16:54
  • See also the docs: https://code.google.com/p/guava-libraries/wiki/IOExplained – Vadzim Jul 09 '15 at 11:52
  • @Vadzim those docs didn't exist when this question was asked :-) – Sean Patrick Floyd Jul 10 '15 at 07:21

9 Answers9

87

You stated in your comment on Calum's answer that you were going to use

CharStreams.toString(new InputStreamReader(supplier.get(), Charsets.UTF_8))

This code is problematic because the overload CharStreams.toString(Readable) states:

Does not close the Readable.

This means that your InputStreamReader, and by extension the InputStream returned by supplier.get(), will not be closed after this code completes.

If, on the other hand, you take advantage of the fact that you appear to already have an InputSupplier<InputStream> and used the overload CharStreams.toString(InputSupplier<R extends Readable & Closeable>), the toString method will handle both the creation and closing of the Reader for you.

This is exactly what Jon Skeet suggested, except that there isn't actually any overload of CharStreams.newReaderSupplier that takes an InputStream as input... you have to give it an InputSupplier:

InputSupplier<? extends InputStream> supplier = ...
InputSupplier<InputStreamReader> readerSupplier = 
    CharStreams.newReaderSupplier(supplier, Charsets.UTF_8);

// InputStream and Reader are both created and closed in this single call
String text = CharStreams.toString(readerSupplier);

The point of InputSupplier is to make your life easier by allowing Guava to handle the parts that require an ugly try-finally block to ensure that resources are closed properly.

Edit: Personally, I find the following (which is how I'd actually write it, was just breaking down the steps in the code above)

String text = CharStreams.toString(
    CharStreams.newReaderSupplier(supplier, Charsets.UTF_8));

to be far less verbose than this:

String text;
InputStreamReader reader = new InputStreamReader(supplier.get(), 
    Charsets.UTF_8);
boolean threw = true;
try {
  text = CharStreams.toString(reader);
  threw = false;
}
finally {
  Closeables.close(reader, threw);
}

Which is more or less what you'd have to write to handle this properly yourself.


Edit: Feb. 2014

InputSupplier and OutputSupplier and the methods that use them have been deprecated in Guava 16.0. Their replacements are ByteSource, CharSource, ByteSink and CharSink. Given a ByteSource, you can now get its contents as a String like this:

ByteSource source = ...
String text = source.asCharSource(Charsets.UTF_8).read();
ColinD
  • 108,630
  • 30
  • 201
  • 202
  • Thanks for the great info (+1). But this is very verbose. I think that combining the accepted answer with Closeables.closeQuietly() is easier. – Sean Patrick Floyd Nov 15 '10 at 16:47
  • @CollinD: I have used your method in one of my answer.Please have a look at the [code](http://stackoverflow.com/questions/4154026/java-special-chars-replace/4154530#4154530) and tell me if this is the right way to use InputSupplier. – Emil Nov 16 '10 at 06:21
  • 1
    @ColinD, if the inputStream is coming from inside of a doPost servlet, is there any point in closing it? (or worrying about closing it) – Blankman Apr 22 '12 at 19:43
  • CharStreams.toString(InputSupplier) is now deprecated. I created a CharSource (from a ByteSource using asCharSource) then used its toString as the docs suggest. – John Lehmann Feb 04 '14 at 23:52
  • @JohnLehmann: I hope you used `read()` rather than `toString()` on the `CharSource`! `toString()` doesn't read the source, it just returns a string representation of the object itself. – ColinD Feb 05 '14 at 17:47
  • But where does one get a ByteSource if all you have is an InputStream? I've used `byte[] bytes = ByteStreams.toByteArray(inputStream);` and then `CharSource charSource = ByteSource.wrap(bytes).asCharSource(Charsets.UTF_8);`, but there's got to be a better way given how common this is. – Ted M. Young Mar 19 '14 at 19:58
  • 4
    @TedM.Young: If all you have is an `InputStream`, and you want to get it as a `String`, `CharStreams.toString(new InputStreamReader(inputStream, charset))` is the way to go. `ByteSource` and `CharSource` are specifically for cases where you have something that can act as a source of `InputStream`s or `Reader`s. – ColinD Mar 19 '14 at 20:11
  • Thanks for taking the time to update this answer for Guava 16! – durron597 Jun 18 '14 at 17:17
56

If you've got a Readable you can use CharStreams.toString(Readable). So you can probably do the following:

String string = CharStreams.toString( new InputStreamReader( inputStream, "UTF-8" ) );

Forces you to specify a character set, which I guess you should be doing anyway.

Calum
  • 5,846
  • 2
  • 26
  • 22
  • 4
    Actually, I'll use a combination of your and Jon Skeet's answers: ` CharStreams.toString(new InputStreamReader(supplier.get(), Charsets.UTF_8))` – Sean Patrick Floyd Nov 15 '10 at 15:25
  • Yep, lots of ways to combine the options! – Calum Nov 15 '10 at 15:43
  • 10
    @S.P.Floyd: If you have an `InputSupplier` I'd strongly recommend using `CharStreams.newReaderSupplier(supplier, Charsets.UTF_8)` rather than `new InputStreamReader`. The reason is that when given the `InputStreamReader`, `toString` will _not_ close that `Reader` (and thus not the underlying stream!). By using an `InputSupplier` for the `Reader`, the `toString` method will handle closing the `Reader` for you. – ColinD Nov 15 '10 at 16:10
17

UPDATE: Looking back, I don't like my old solution. Besides it is 2013 now and there are better alternatives available now for Java7. So here is what I use now:

InputStream fis = ...;
String text;
try (  InputStreamReader reader = new InputStreamReader(fis, Charsets.UTF_8)){
        text = CharStreams.toString(reader);
}

or if with InputSupplier

InputSupplier<InputStreamReader> spl = ...
try (  InputStreamReader reader = spl.getInput()){
        text = CharStreams.toString(reader);
    }
husayt
  • 14,553
  • 8
  • 53
  • 81
17

Nearly. You could use something like this:

InputSupplier<InputStreamReader> readerSupplier = CharStreams.newReaderSupplier
    (streamSupplier, Charsets.UTF_8);
String text = CharStreams.toString(readerSupplier);

Personally I don't think that IOUtils.toString(InputStream) is "nice" - because it always uses the default encoding of the platform, which is almost never what you want. There's an overload which takes the name of the encoding, but using names isn't a great idea IMO. That's why I like Charsets.*.

EDIT: Not that the above needs an InputSupplier<InputStream> as the streamSupplier. If you've already got the stream you can implement that easily enough though:

InputSupplier<InputStream> supplier = new InputSupplier<InputStream>() {
    @Override public InputStream getInput() {
        return stream;
    }
};
assylias
  • 321,522
  • 82
  • 660
  • 783
Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Jon, is stream via request.getInputStream? Also, will yours close the stream like ColinD mentioned in @Calum's answer? – Blankman Apr 22 '12 at 16:24
  • Oh, and it a servlet doPost environment, should I be closing the stream anyhow? – Blankman Apr 22 '12 at 16:27
  • @Blankman: Ah, so that's your context - it wasn't at all clear from your question. It doesn't matter too much whether you close a request stream, but I'd generally do so. I'll edit this answer though - there's no such overload, it seems. – Jon Skeet Apr 22 '12 at 16:29
  • wait, it wasn't my question, I just found this via G as I am reading a posted file from the stream. it seems you are doing the method that closes the underlying stream since you used the newReaderSupplier. – Blankman Apr 22 '12 at 16:34
  • 1
    I'm just doing this now: String payLoad = CharStreams.toString(new InputStreamReader(request.getInputStream(), "UTF-8")); – Blankman Apr 22 '12 at 16:35
  • @Blankman: You could do that, but that won't close the stream. – Jon Skeet Apr 22 '12 at 17:11
  • but doesn't it really matter? is it more efficient or something? – Blankman Apr 22 '12 at 19:43
  • @Blankman: It's more suitable for extracting to a general purpose method, for one thing. – Jon Skeet Apr 22 '12 at 19:46
  • Interestingly enough, `Charsets` is somewhat slower than using the `String` name of a charset, because of defensive copies that occur when using `Charset` objects. If a security manager is present, the overhead is worse. – BeeOnRope Jan 24 '13 at 09:51
  • @BeeOnRope: I'd usually prefer to use `Charsets` everywhere until I'd proved that a specific use of it was a significant performance problem. (It's somewhat crazy that the better practice (IMO) of using a rich data type performs worse than a string lookup, but that's a different matter.) – Jon Skeet Jan 24 '13 at 10:10
  • Yeah, this came from real world testing, where a parsing application got a 20% boost when switching to "UTF-8" from Charsets.UTF_8. This particular bit of code was highly sensitive to byte[] -> String speed as that was the bottleneck operation. I'd expect most typical applications wouldn't see much of an effect, but it's something to keep in mind. There aren't really many alternatives to those constructors if what you need is a String - doing it yourself gets slower for moderate or larger strings since you need to make defensive copies that the String internal ctors can avoid. – BeeOnRope Jan 24 '13 at 22:55
  • 1
    @BeeOnRope: I guess one intermediate approach is `Charsets.UTF_8.name()` - more typo-resistant. – Jon Skeet Jan 24 '13 at 23:01
12

Another option is to read bytes from Stream and create a String from them:

new String(ByteStreams.toByteArray(inputStream))
new String(ByteStreams.toByteArray(inputStream), Charsets.UTF_8)

It's not 'pure' Guava, but it's a little bit shorter.

ponomandr
  • 1,523
  • 13
  • 21
  • Unfortunately, `ByteStreams.toByteArray()` does not close the stream, according to the Javadoc. – The Alchemist Mar 28 '15 at 18:31
  • That's true. I haven't seen any Guava's function that closes stream. Well, except closeQuietly. – ponomandr Mar 28 '15 at 22:57
  • 1
    Typically, the stream is opened in try-with-resources statement and closed automatically, so it should't be responsibility of toByteArray() – ponomandr Mar 28 '15 at 23:04
4

There is much shorter autoclosing solution in case when input stream comes from classpath resource:

URL resource = classLoader.getResource(path);
byte[] bytes = Resources.toByteArray(resource);
String text = Resources.toString(resource, StandardCharsets.UTF_8);

Uses Guava Resources, inspired by IOExplained.

Vadzim
  • 24,954
  • 11
  • 143
  • 151
4

Based on the accepted answer, here is a utility method that mocks the behavior of IOUtils.toString() (and an overloaded version with a charset, as well). This version should be safe, right?

public static String toString(final InputStream is) throws IOException{
    return toString(is, Charsets.UTF_8);
}


public static String toString(final InputStream is, final Charset cs)
throws IOException{
    Closeable closeMe = is;
    try{
        final InputStreamReader isr = new InputStreamReader(is, cs);
        closeMe = isr;
        return CharStreams.toString(isr);
    } finally{
        Closeables.closeQuietly(closeMe);
    }
}
Sean Patrick Floyd
  • 292,901
  • 67
  • 465
  • 588
  • Looks pretty much fine to me. Guava's IO stuff does work best if you learn to think in terms of reusable input suppliers rather than 1-shot streams and readers (when possible), but I guess since you're converting existing IOUtils code that would be a big change. – ColinD Nov 15 '10 at 17:00
  • 2
    In my guava 14, the closeQuietly is already deprecated. The suggestion is to use the try-with-resources feature that exists in Java 7. More on this at https://code.google.com/p/guava-libraries/wiki/ClosingResourcesExplained – Bertie Jun 05 '13 at 11:01
  • 2
    @AlbertKam agreed. But remember: this answer is three years old. – Sean Patrick Floyd Jun 07 '13 at 08:23
  • @SeanPatrickFloyd: Thanks ! Actually i got to the newer solution starting from your answer. I was thinking of adding the comment for the others who might be using the newer version. :) – Bertie Jun 08 '13 at 09:34
2

EDIT (2015): Okio is the best abstraction and tools for I/O in Java/Android that I know of. I use it all the time.

FWIW here's what I use.

If I already have a stream in hand, then:

final InputStream stream; // this is received from somewhere
String s = CharStreams.toString(CharStreams.newReaderSupplier(new InputSupplier<InputStream>() {
    public InputStream getInput() throws IOException {
        return stream;
    }
}, Charsets.UTF_8));

If I'm creating a stream:

String s = CharStreams.toString(CharStreams.newReaderSupplier(new InputSupplier<InputStream>() {
    public InputStream getInput() throws IOException {
        return <expression creating the stream>;
    }
}, Charsets.UTF_8));

As a concrete example, I can read an Android text file asset like this:

final Context context = ...;
String s = CharStreams.toString(CharStreams.newReaderSupplier(new InputSupplier<InputStream>() {
    public InputStream getInput() throws IOException {
        return context.getAssets().open("my_asset.txt");
    }
}, Charsets.UTF_8));
orip
  • 73,323
  • 21
  • 116
  • 148
0

For a concrete example, here's how I can read an Android text file asset:

public static String getAssetContent(Context context, String file) {
    InputStreamReader reader = null;
    InputStream stream = null;
    String output = "";

    try {
        stream = context.getAssets().open(file);
        reader = new InputStreamReader(stream, Charsets.UTF_8);
        output = CharStreams.toString(reader);
    } catch (IOException e) {
        e.printStackTrace();
    } finally {
        if (stream != null) {
            try {
                stream.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        if (reader != null) {
            try {
                reader.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    }

    return output;
}
TruMan1
  • 33,665
  • 59
  • 184
  • 335