Using reflection to invoke Type parameterized methods

Question

The google cloud dataflow sdk has a class that registers Coders for different Avro types.

CoderRegistry cr = p.getCoderRegistry();
cr.registerStandardCoders();
cr.registerCoder(Row.class, AvroCoder.of(Row.class));
cr.registerCoder(Destination.class, AvroCoder.of(Destination.class));
cr.registerCoder(Device.class, AvroCoder.of(Device.class));
cr.registerCoder(Location.class, AvroCoder.of(Location.class));
cr.registerCoder(Source.class, AvroCoder.of(Source.class));
cr.registerCoder(DimensionalMetric.class, AvroCoder.of(DimensionalMetric.class));
cr.registerCoder(DimensionSet.class, AvroCoder.of(DimensionSet.class));
cr.registerCoder(MetricSet.class, AvroCoder.of(MetricSet.class));

But as you can imagine this gets pretty cumbersome and I would like to use java reflection API to automatically register all classes in the package com.brightcove.rna.model. I imagine this would look something like this:

void registerAllModels(Pipeline p) {
    CoderRegistry cr = p.getCoderRegistry();
    Reflections r = new Reflections("com.brightcove.rna.model");
    Set<Class<? extends IndexedRecord>> classes = r.getSubTypesOf(IndexedRecord.class);
    for (Class<? extends IndexedRecord> clazz : classes) {
        cr.registerCoder(clazz, AvroCoder.of(clazz));
    }
}

Unfortunately, this doesn't work. What am I doing wrong here?

Can you provide the error message or describe the erroneous behavior? — Kenn Knowles, Dec 29 '15 at 18:35

score 4 · Accepted Answer · edited May 23 '17 at 11:52

You have a compile error in the following line:

cr.registerCoder(clazz, AvroCoder.of(clazz));

The error is:

The method registerCoder(Class<?>, Class<?>) in the type CoderRegistry is not applicable for the arguments (Class<capture#1-of ? extends IndexedRecord>, AvroCoder<capture#2-of ? extends IndexedRecord>).

Basically, the issue is that Java compiler cannot infer that the wildcard types on both parameters are the same, and is reporting an error. This is a common Java issue, see this question, for example.

A fix is the following, which may require you to suppress compiler warnings about raw types and an unchecked conversion:

  cr.registerCoder(clazz, (Coder) AvroCoder.of(clazz.newInstance().getSchema()));

This fixes two problems:

Ensures registerCoder(Class<T>, Coder<T>) is used instead of the overloaded registerCoder(Class<?>, Class<?>).
AvroCoder.of(Class<T>) uses Avro's ReflectData.get().getSchema() to generate a schema, which may not work on all types, such as GenericRecord. An alternative is to construct the coder via AvroCoder.of(Schema) and get the schema from the automatically-generated class.

score 0 · Answer 2 · answered Dec 29 '15 at 19:51

It turned out that a little more reading up on the error message that I was getting helped me reach the root of the problem.

AnalyticsPipeline.java:110: error: no suitable method found for registerCoder(Class<CAP#1>,AvroCoder<CAP#2>)
            cr.registerCoder(clazz, AvroCoder.of(clazz));
              ^
    method CoderRegistry.registerCoder(Class<?>,Class<?>) is not applicable
      (argument mismatch; no instance(s) of type variable(s) T#1 exist so that AvroCoder<T#1> conforms to Class<?>)
    method CoderRegistry.registerCoder(Class<?>,CoderFactory) is not applicable
      (argument mismatch; no instance(s) of type variable(s) T#1 exist so that AvroCoder<T#1> conforms to CoderFactory)
    method CoderRegistry.<T#2>registerCoder(Class<T#2>,Coder<T#2>) is not applicable
      (inferred type does not conform to equality constraint(s)
        inferred: CAP#3
        equality constraints(s): CAP#3,CAP#1)
  where T#1,T#2 are type-variables:
    T#1 extends Object declared in method <T#1>of(Class<T#1>)
    T#2 extends Object declared in method <T#2>registerCoder(Class<T#2>,Coder<T#2>)
  where CAP#1,CAP#2,CAP#3 are fresh type-variables:
    CAP#1 extends IndexedRecord from capture of ? extends IndexedRecord
    CAP#2 extends IndexedRecord from capture of ? extends IndexedRecord
    CAP#3 extends IndexedRecord from capture of ? extends IndexedRecord

As stated above, javac thinks that the types of the arguments don't match (correctly so as I have provided no information to that fact). My solution turned out to be a fairly simple modification. By adding a helper function that fixes the type.

void registerCoders(Pipeline p) {
        CoderRegistry cr = p.getCoderRegistry();
        Reflections r = new Reflections("com.brightcove.rna.model");
        Set<Class<? extends IndexedRecord>> classes = r.getSubTypesOf(IndexedRecord.class);
        for (Class<? extends IndexedRecord> clazz : classes) {
            registerAvroType(cr, clazz);
        }
    }

    <T extends IndexedRecord> void registerAvroType(CoderRegistry cr, Class<T> clazz) {
        cr.registerCoder(clazz, AvroCoder.of(clazz));
    }

I was able to convey to the compiler that the types are indeed the same and the problem was resolved.

Using reflection to invoke Type parameterized methods

2 Answers2