29

After reading https://www.airpair.com/java/posts/spring-streams-memory-efficiency, I am tempted to stream results out of a database, but as I discussed with a colleague (cfr. comment he added to that article), one needs to remember to use the try-with-resources construct to avoid any memory leaks.

  1. Why doesn't the Java 8 library take care of closing streams itself after each terminal operation (without having to wrap the stream instantiation in a try-with-resources)?
  2. If applicable, are there any plans for this functionality to be added to Java, or would it make sense to request it?
Sébastien Dubois
  • 869
  • 9
  • 19

3 Answers3

31

Because streams that require explicit resource release is actually a pretty unusual case. So we chose not to burden all stream execution with something that is only valuable for .01% of usages.

We made Stream Autocloseable so that you can release resources from the source if you want to, but this is where we stopped, and for a good reason.

Not only would doing this automagically burden the majority of users with extra work that they don't need, but this would also violate a general principle: he who allocates the resource is responsible for closing the resource. When you call 

BufferedReader reader = ...
reader.lines().op().op()...

you are the one opening the resource, not the stream library, and you should close it. In fact, since closing a stream resulting from calling an accessor method on some resource-holding object will sometimes close the underlying object, you probably don't want the stream closing the BufferedReader for you -- you might want it to stay open after the call.

If you want to close the resource, this is easy too:

try (BufferedReader reader = ...) {
    reader.lines().op()...
}

You're probably using streams in a particular way, so it probably seems "obvious" what streams should do -- but there are more use cases out there than yours. So rather than catering to specific use cases, we approached it from the general principle: if you opened the stream, and you want it closed, close it yourself, but if you didn't open it, it's not for you to close.

Brian Goetz
  • 90,105
  • 23
  • 150
  • 161
  • 1
    This could be a catch-22 issue, streams requiring a resource release could be rare because the current library does not help support their case. AFAIK, `Stream` was chiefly introduced to support parallelization, but from its API contract it is clear that parallelization is just one thing it should help doing. I'm sure many developers could wish to use the stream API for other purposes such as O(1) memory usage, avoiding stop-the-world GC events, and having a clear separation of iteration and processing concerns (I'm just reusing Marko Topolnik's wording here). – Sébastien Dubois Mar 02 '15 at 16:43
  • 3
    I now better understand why `Stream` is currently like this so I will accept this answer. But to me as an end-user and with my current understanding, having a(nother?) stream implementation which takes care of always closing underlying resources in a finally block still appears needed in the Java library. – Sébastien Dubois Mar 02 '15 at 16:45
  • 4
    Correction: stream was introduced to provide a more abstract way of expressing aggregate operations on arbitrary datasets; this is a necessary precondition for practical parallelism, but I think everyone would agree that Stream is pretty useful even without parallelism. The support for `close()` and `Autocloseable` represent stretching the design to provide minimal support for streams whose sources represent user-managed resources (as opposed to memory, which is managed by the VM). I think this is really "2% empty" vs "98% full", you are probably just swimming in the 2% right now. – Brian Goetz Mar 02 '15 at 17:38
  • Thanks for clarifying the historical reasons leading to the current `Stream`'s design. I also do see it as positively "full", this is a great addition to Java; and I do understand that resource-handling is an aspect which stretches the core `Stream` API contract. I would not venture myself into quantifying the current and future number of developers interested in resource-based-streams; however, I see that Marko Topolnik's suggestion would apply to the canonical REST-API-over-data-store model, which sounds like quite a few people. – Sébastien Dubois Mar 02 '15 at 20:09
  • 1
    Naïvely, something like `StreamSupport.stream(Spliterator spliterator, boolean parallel, boolean closeUponTerminationOrException)`... – Sébastien Dubois Mar 02 '15 at 20:11
  • 5
    `doing this automagically [would] burden the majority of users with extra work that they don't need` this I don't understand: what's the work the user is burdened with? An automatic call to `close`? For most streams it's a no-op anyway, I don't understand the burden. – Marko Topolnik Mar 03 '15 at 17:09
  • 1
    `he who allocates the resource is responsible for closing the resource`---this is a better argument not to close the stream automatically. However, in the context of OP's use case, there are other legitimate contracts, such as "Spring's view layer guarantees to close the stream you pass to it in the return value of the Controller". Again, no need to auto-close the stream in the terminal operation. – Marko Topolnik Mar 03 '15 at 17:11
  • 7
    Also, for the terminal operations `iterator` and `spliterator`, there's no way we could close the stream at that point, because if we did, the resulting iterator/spliterator wouldn't work. So sure, there are alternate policies we could have pursued, that also might not have been "wrong", but the one we chose was based on a clear and sensible principle. – Brian Goetz Mar 03 '15 at 18:27
  • @MarkoTopolnik, I think the "work" here is the call to `close`. The fact that it doesn't actually do anything makes it even worse in my opinion because not only are you calling it, you're calling it for no reason. A method call is pretty cheap, but I'm sure there are hundreds of places in Java you could potentially add no-op method calls, and in the aggregate they'd become a significant burden. – Samuel Edwin Ward Mar 10 '15 at 15:27
  • 1
    @SamuelEdwinWard I'm quite convinced that this is not what Brian had in mind. What he had in mind was burdening the client with the obligation to use a stream inside a *try-with-resources*, which is implied by it being `AutoCloseable`. Normally the IDE even emits a warning when you don't do that, but now they need a special case for `Stream`. It may have slipped Brian's mind that streams actually are `AutoCloseable`, so they do all the damage they can in that area. The compromise was to document that this is a special case and the Stream does not usually need to sit inside a try-with-resources – Marko Topolnik Mar 10 '15 at 15:37
  • 1
    @MarkoTopolnik The IDEs are trying to be helpful, but just because something _can_ be closed does not mean it is an error to not close it all cases. The spec of `AutoCloseable` was clarified in 8 to make this clear. (And no, it did not slip my mind; there were many hours of meetings on the subject, and many alternatives considered. The status quo is a practical compromise that stems from a sensible principle (he who opens should close); I realize that there were other reasonable choices too, but we only get to pick one, and we felt this one was best.) – Brian Goetz Mar 12 '15 at 16:25
  • 6
    @BrianGoetz I just vasted whole day to find out simple line like `boolean res = Files.list(dir).anyMatch(e -> true);` leaves unclosed resource. Note it is actually not me, who opens the resource, it is opened in Files class and it's behavior is described there as `.onClose(()->ds.close());`. As of your example, `bufferedReader.lines().op().op()...` there is no stream close handler added in lines() method, so even if the stream closed after all, user still needs to handle bufferedReader himself. – George Vinokhodov Dec 08 '17 at 17:25
  • 3
    @BrianGoetz terminalOperator should be truely terminal. What the matter of leaving stream open if it is unusable any more? – George Vinokhodov Dec 08 '17 at 17:37
  • 2
    This decision is very confusing because of methods like `Files.list()` which return a stream directly and are intended to be used in a fluent style without ever assigning the intermediate stream to a variable. It's entirely unintuitive that you would have to wrap that into a try-with-resources. The documentation for `Stream.onClose` should mention that they are never run unless someone explicitly saves the stream and closes it. – whaleberg Jun 12 '20 at 19:05
1

I think you're mixing java.io.InputStream with java.util.stream.Stream, which are two very very different concepts.

try-with-resources works on objects implementing the Autoclosable interface, such as InputStreams. InputStreams represent an abstract source of data related to IO.

java.util.stream.Stream<T> on the other hand, implements a concept from functional programming, which represents a kind of a dynamic collection which is not necessarily statically built, but can rather be generated, and consequently potentially infinite.

What Marko Topolnik (the author of the article you linked to) essentially does in the article, is suggest a way to wrap an IO source into a java.util.stream.Stream. This is quite a clever approach, but java.util.stream.Streams are not in general intended for this purpose.

Because they are not in general intended for use with IO, there is no reason for them to include closing after terminal operations.

EDIT:

After you've clarified that you hadn't in fact mixed up the two (sorry for assuming so), thanks to this answer, I found that your exact example is answered in the documentation of AutoCloseable (emphasis added by myself):

It is possible, and in fact common, for a base class to implement AutoCloseable even though not all of its subclasses or instances will hold releasable resources. For code that must operate in complete generality, or when it is known that the AutoCloseable instance requires resource release, it is recommended to use try-with-resources constructions. However, when using facilities such as Stream that support both I/O-based and non-I/O-based forms, try-with-resources blocks are in general unnecessary when using non-I/O-based forms.

Community
  • 1
  • 1
Zoltán
  • 21,321
  • 14
  • 93
  • 134
  • 3
    `java.util.stream.Stream` also implements `Autoclosable`, and that is precisely what this question is about. – davmac Mar 02 '15 at 15:55
0

Why doesn't the Java 8 library take care of closing streams itself after each terminal operation (without having to wrap the stream instantiation in a try-with-resources)?

Because an exception may occur during or before the terminal operation, and because you may not want the terminal operation to close the stream. You can use try-with-resource if you definitely want the stream to close.

If applicable, are there any plans for this functionality to be added to Java, or would it make sense to request it?

It would not make sense, see answer above.

davmac
  • 20,150
  • 1
  • 40
  • 68
  • do you have a reference for that? – Martin Serrano Mar 02 '15 at 15:52
  • @MartinSerrano a reference for what, exactly? It's trivial to open a stream and cause an exception before performing a terminal operation. – davmac Mar 02 '15 at 15:52
  • for your answer. is this just your opinion? seems like @Zoltan answer is more on target. – Martin Serrano Mar 02 '15 at 15:55
  • @MartinSerrano I suspect that neither Zoltan nor yourself really understand the question that is being asked. In so far as everything in language is open to some interpretation, then yes, this is a matter of opinion; but if I understand the question correctly, then no, my statements in the answer above are factual. – davmac Mar 02 '15 at 15:58
  • 1
    Take an `ArrayList`, throw a `RuntimeException` in `Stream.foreach()` and try to call a terminal operation afterwards on the `Stream`. Even if you don't throw any exception, the `Stream` cannot be reused after operated upon. I don't see any reason not to automatically close a `Stream` after a terminal operation. – benez May 11 '17 at 17:35
  • @benez see the answer from Brian Goetz above. – davmac May 12 '17 at 15:35