Unable to understand this Java Stream+Generics example

Question

Could someone help me to understand why this code behaves as described in the comments

// 1) compiles
List<Integer> l = Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);

/*
 *  2) does not compile
 *  
 *  Exception in thread "main" java.lang.Error: Unresolved compilation problems: 
 *      Type mismatch: cannot convert from Object to <unknown>
 *      The type ArrayList does not define add(Object, Integer) that is applicable here
 *      The type ArrayList does not define addAll(Object, Object) that is applicable here
 */
Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList::addAll);

// 3) compiles
Stream.of(1, 2, 3).collect(ArrayList<Integer>::new, ArrayList::add, ArrayList::addAll);

/*
 *  4) does not compile
 *  
 *  Exception in thread "main" java.lang.Error: Unresolved compilation problems: 
 *      Type mismatch: cannot convert from Object to <unknown>
 *      The type ArrayList does not define add(Object, Integer) that is applicable here
 *      The type ArrayList<Integer> does not define addAll(Object, Object) that is applicable here
 */
Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);  

It has clearly something to do with the definition of a type in generic methods, it's an information that must be somehow provided... but why is it mandatory? Where and how, syntactically, should I have figured it out from the signature of methods of() and collect()?

public static<T> Stream<T> of(T... values) {...}

<R> R collect(Supplier<R> supplier,
              BiConsumer<R, ? super T> accumulator,
              BiConsumer<R, R> combiner);

Answer 1

Although this is not an answer which analyzes the Lambda spec on http://download.oracle.com/otndocs/jcp/lambda-0_9_3-fr-eval-spec/index.html, I nevertheless tried to find out on what it depends.

Copying two methods from the Stream class:

static class Stream2<T> {

    @SafeVarargs
    @SuppressWarnings("varargs") // Creating a stream from an array is safe
    public static<T> Stream2<T> of(T... values) {
        return new Stream2<>();
    }

     public  <R> R collect(  Supplier<R> supplier,
             BiConsumer<R, ? super T> accumulator,
             BiConsumer<R, R> combiner){return null;}

}

This compiles:

Stream2.of(1,2,3).collect(ArrayList::new, ArrayList::add, ArrayList::addAll );

like OP's (2).

Now changing the collect method to by moving the first argument to the third place

     public  <R> R collect(BiConsumer<R, ? super T> accumulator,
             BiConsumer<R, R> combiner,
             Supplier<R> supplier
     ){return null;}

This still works (5):

 Stream2.of(1,2,3).collect(ArrayList::add, ArrayList::addAll,ArrayList::new );

Also this works (6):

 Stream2.of(1,2,3).collect(ArrayList::add, ArrayList::addAll,ArrayList<Integer>::new );

These don't work (7,8):

 Stream2.of(1,2,3).collect(ArrayList<Integer>::add, ArrayList::addAll,ArrayList::new );
 Stream2.of(1,2,3).collect(ArrayList<Integer>::add, ArrayList<Integer>::addAll,ArrayList::new );

But this works again (9):

 Stream2.of(1,2,3).collect(ArrayList<Integer>::add, ArrayList<Integer>::addAll,ArrayList<Integer>::new );

So i guess when a supplier is annotated with the explicit type argument, it seems to work. When only the consumers are, it does not. But maybe someone else knows why this makes a difference.

EDIT: Trying to use a TestList, it gets even stranger:

public class StreamTest2 {

    public static void main(String[] args) {

        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll);
        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll2);
        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll3);
        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll4);
        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll5);
        Stream.of(1, 2, 3).collect(TestList::new, TestList::add, TestList<Integer>::addAll6);

    }
}

class TestList<T> extends AbstractList<T> {

    @Override
    public T get(int index) {
        return null;
    }

    @Override
    public int size() {
        return 0;
    }

    @Override
    public boolean addAll(Collection<? extends T> c) {
        return true;
    }

    public boolean addAll2(TestList<? extends T> c) {
        return true;
    }
    public boolean addAll3(Collection<T> c) {
        return true;
    }

    public boolean addAll4(List<? extends T> c) {
        return true;
    }
    public boolean addAll5(AbstractList<? extends T> c) {
        return true;
    }

    public boolean addAll6(Collection<? extends T> c) {
        return true;
    }

    @Override
    public boolean add(T e) {
        return true;
    }
}

addAll does not work, but addAll2-6 do work. Even addAll6 works, which has the same signature as the original addAll.

Answer 2

Whenever you struggle about compiler errors, you should include, which compiler you have used and its version number. And if you have used a compiler other than the standard javac, you should give javac a try and compiler the results.

When you write

List<Integer> l = Stream.of(1, 2, 3)
    .collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);

the compiler will use the target type List<Integer> for inferring the type R (which matches exactly the target type here). Without a target type like in

Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList::addAll);

the compiler will infer the type R from the supplier and infer ArrayList<Object> instead. Since an ArrayList<Object> is capable of holding Integer instances and provides the necessary add and addAll methods, this construct compiles without problems when using the standard javac. I tried jdk1.8.0_05, jdk1.8.0_20, jdk1.8.0_40, jdk1.8.0_51, jdk1.8.0_60, jdk1.9.0b29, and jdk1.9.0b66 to be sure that there are no version specific bugs involved. I guess, you are using Eclipse, which is known for having problems with the Java 8 type inference.

Similarly, using

Stream.of(1, 2, 3).collect(ArrayList<Integer>::new, ArrayList::add, ArrayList::addAll);

works but now your hint forces the inferred type for R to be ArrayList<Integer>. In contrast

Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);

does not work as the compiler is inferring ArrayList<Object> for the return type of the supplier which is not compatible with the method ArrayList<Integer>::addAll. But the following would work:

Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Object>::addAll);

However, you don’t need any explicit type when using standard javac…

Answer 3

When confronted with this kind of situation, I feel the best way to understand the problem is with pure reasoning and logic. Type-inference is a beast that covers an entire chapter of the JLS. Let's forget about ECJ and javac for the moment, think through the 4 examples and determine whether a given compiler could or should be able to compile it according to the JLS.

So let's consider the signature of collect:

<R> R collect(Supplier<R> supplier,
              BiConsumer<R, ? super T> accumulator,
              BiConsumer<R, R> combiner);

The questions with this signature is: what is R and how will a compiler be able to determine what R is?

We can argue that a compiler will be able to infer a type for R with what we're giving as parameter to collect. As an example, the first is a Supplier<R> so if, e. g., we are to give as parameter () -> new StringBuilder(), a compiler should be able to infer R as StringBuilder.

---

Let's consider the following case:

List<Integer> l = Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList::addAll);

In this example, the 3 parameters of collect are 3 method-references and we're assigning the result to a List<Integer>. That's information a compiler could take: we are saying to it that the type used is Integer.

Okay, so should it compile? Let's consider the 3 arguments to collect separately:

• The first, in this case, is a Supplier<ArrayList<Integer>> and we are giving ArrayList::new. Can this method-reference refer to an existing method (constructor in this case)? Well yes, it can refer to the empty constructor of ArrayList (as a lambda - () -> new ArrayList<Integer>()) because ArrayList<Integer> can be bound to List<Integer>. So far so good.
• The second is BiConsumer<ArrayList<Integer>, ? super Integer>. Note that T = Integer here because the Stream is composed of integer literals which are of type int. We're giving ArrayList::add, which can refer to add(e) (as a lambda: (list, element) -> list.add(element)).
• The third is BiConsumer<List<Integer>, List<Integer>> and we're giving ArrayList::addAll. It can also refer to addAll(c): addAll takes as parameter a Collection<? extends Integer> and List<Integer> can be bound to this type.

So basically, with only reasoning, such an expression should compile.

Now, let's consider your 4 cases:

Case 1:

List<Integer> l = Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);

We're assigning the result of the expression as a List<Integer> so we're telling a compiler: R is List<Integer> here. The difference with the case above is that we're giving the method reference ArrayList<Integer>::addAll. Let's take a closer look at this. This method-reference is trying to refer to a method name addAll which would take as parameter a List<Integer> (our R) and should be applied to a ArrayList<Integer> (the type we're explicitely using in the method-reference). Well this is exactly the same as what we concluded in the reasoning above; it should work: R = List<Integer> can be bound to ArrayList<Integer>.

Case 2

Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList::addAll);

The difference with the case above is that we're not assigning the result of the expression. So a compiler is left to infer the type based on the supplier: ArrayList::new, so it should infer it as ArrayList<Object>.

• ArrayList::add can be bound to BiConsumer<ArrayList<Object>, ? super Integer> because the add method of a ArrayList<Object> can take an Integer as argument.
• ArrayList::addAll can be bound to BiConsumer<ArrayList<Object>, ArrayList<Object>>.

So a compiler should be able to compile that.

Case 3

Stream.of(1, 2, 3).collect(ArrayList<Integer>::new, ArrayList::add, ArrayList::addAll);

The difference with case 2 is that we're explicitely telling the compiler that the supplier supplies ArrayList<Integer> instances, not just ArrayList<Object>. Does it change anything? It should not, the reasoning made in case 2 still holds here. So it should compile just as well.

Case 4

Stream.of(1, 2, 3).collect(ArrayList::new, ArrayList::add, ArrayList<Integer>::addAll);

The difference with case 2 is that this time, we're giving ArrayList<Integer>::addAll. Based from case 2, we know that a compiler inferred R to be ArrayList<Object> because of the supplier (that does not has a specific type). This should cause a problem here: ArrayList<Integer>::addAll tries to reference the method addAll on a ArrayList<Integer> but we saw that, for a compiler, this was inferred as ArrayList<Object> and an ArrayList<Object> is not an ArrayList<Integer>. So this should not compile.

What could we do to make it compile?

• Change the supplier to have a specific Integer type.
• Remove the explicit <Integer> from the method-reference.
• Write the method reference as ArrayList::<Integer> addAll instead.

Conclusion

I tested the examples with Eclipse Mars 4.5.1 and javac 1.8.0_60. The result is that javac, behaves exactly as with our reasoning concluded: only case 4 is not compiled.

Bottom line, Eclipse has a small bug.