KafkaBindingRebalanceListener Bean not autowired with Multi-binder

Question

There is another post about a similar issue. The accepted answer works well with a single top-level binder. But I don't have luck with multi-binder. Not sure I mess up something or this technique only supports a single top-level binder.

Working YAML (only one top-level binder):

spring:
  cloud:
    function:
      definition: consume;
    stream:
      function:
        bindings:
          consume-in-0: input
      bindings:
        input:
          destination: students
          group: groupA
      kafka:
        binder:
          brokers: 192.168.86.23:9092

Broken YAML (multi-binder structure):

spring:
  cloud:
    function:
      definition: consume;
    stream:
      function:
        bindings:
          consume-in-0: input
      bindings:
        input:
          destination: students
          group: groupA
          binder: kafkaBinder
      binders:
        kafkaBinder:
          type: kafka
          environment:
            spring:
              cloud:
                stream:
                  kafka:
                    binder:
                      brokers: 192.168.86.23:9092

SpringBootApplication:

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Bean
    KafkaBindingRebalanceListener rebal() {
        return new KafkaBindingRebalanceListener() {

            @Override
            public void onPartitionsAssigned(String bindingName, Consumer<?, ?> consumer,
                                             Collection<TopicPartition> partitions, boolean initial) {
                System.out.println(bindingName + " assignments: " + partitions + ", initial call :" + initial);
            }

        };
    }
}

Another post: KafkaBindingRebalanceListener Bean not autowired by KafkaMessageChannelBinder Bean

Gary Russell · Answer 1 · 2020-11-16T23:02:49.557

This is a known problem; there are several such issues; add your voice to this one so it gets more attention.

I added a temporary work-around to this answer this morning for another issue.

If you don't mind using reflection, you can use that technique here too...

@Bean
KafkaBindingRebalanceListener rebalanceListener() {
    return new KafkaBindingRebalanceListener() {

        @Override
        public void onPartitionsAssigned(String bindingName,
                org.apache.kafka.clients.consumer.Consumer<?, ?> consumer, Collection<TopicPartition> partitions,
                boolean initial) {

            if (initial) {
                System.out.println("SEEK");
                consumer.seekToBeginning(partitions);
            }
        }

    };
}

// BEGIN HACK TO REPLACE REBALANCE LISTENER IN BINDER

@Autowired(required = false)
private Collection<DefaultBinderFactory.Listener> binderFactoryListeners;

@Bean
public BinderFactory binderFactory(BinderTypeRegistry binderTypeRegistry,
        BindingServiceProperties bindingServiceProperties) {

    DefaultBinderFactory binderFactory = new DefaultBinderFactory(
            getBinderConfigurations(binderTypeRegistry, bindingServiceProperties),
            binderTypeRegistry) {

                @Override
                public synchronized <T> Binder<T, ?, ?> getBinder(String name,
                        Class<? extends T> bindingTargetType) {

                    Binder<T, ?, ?> binder = super.getBinder(name, bindingTargetType);
                    if (binder instanceof KafkaMessageChannelBinder) {
                        new DirectFieldAccessor(binder).setPropertyValue("rebalanceListener", rebalanceListener());
                    }
                    return binder;
                }


    };
    binderFactory.setDefaultBinder(bindingServiceProperties.getDefaultBinder());
    binderFactory.setListeners(this.binderFactoryListeners);
    return binderFactory;
}

// Had to copy this because it's private in BindingServiceConfiguration
private static Map<String, BinderConfiguration> getBinderConfigurations(
        BinderTypeRegistry binderTypeRegistry,
        BindingServiceProperties bindingServiceProperties) {

    Map<String, BinderConfiguration> binderConfigurations = new HashMap<>();
    Map<String, BinderProperties> declaredBinders = bindingServiceProperties
            .getBinders();
    boolean defaultCandidatesExist = false;
    Iterator<Map.Entry<String, BinderProperties>> binderPropertiesIterator = declaredBinders
            .entrySet().iterator();
    while (!defaultCandidatesExist && binderPropertiesIterator.hasNext()) {
        defaultCandidatesExist = binderPropertiesIterator.next().getValue()
                .isDefaultCandidate();
    }
    List<String> existingBinderConfigurations = new ArrayList<>();
    for (Map.Entry<String, BinderProperties> binderEntry : declaredBinders
            .entrySet()) {
        BinderProperties binderProperties = binderEntry.getValue();
        if (binderTypeRegistry.get(binderEntry.getKey()) != null) {
            binderConfigurations.put(binderEntry.getKey(),
                    new BinderConfiguration(binderEntry.getKey(),
                            binderProperties.getEnvironment(),
                            binderProperties.isInheritEnvironment(),
                            binderProperties.isDefaultCandidate()));
            existingBinderConfigurations.add(binderEntry.getKey());
        }
        else {
            Assert.hasText(binderProperties.getType(),
                    "No 'type' property present for custom binder "
                            + binderEntry.getKey());
            binderConfigurations.put(binderEntry.getKey(),
                    new BinderConfiguration(binderProperties.getType(),
                            binderProperties.getEnvironment(),
                            binderProperties.isInheritEnvironment(),
                            binderProperties.isDefaultCandidate()));
            existingBinderConfigurations.add(binderEntry.getKey());
        }
    }
    for (Map.Entry<String, BinderConfiguration> configurationEntry : binderConfigurations
            .entrySet()) {
        if (configurationEntry.getValue().isDefaultCandidate()) {
            defaultCandidatesExist = true;
        }
    }
    if (!defaultCandidatesExist) {
        for (Map.Entry<String, BinderType> binderEntry : binderTypeRegistry.getAll()
                .entrySet()) {
            if (!existingBinderConfigurations.contains(binderEntry.getKey())) {
                binderConfigurations.put(binderEntry.getKey(),
                        new BinderConfiguration(binderEntry.getKey(), new HashMap<>(),
                                true, "integration".equals(binderEntry.getKey()) ? false : true));
            }
        }
    }
    return binderConfigurations;
}

// END HACK

Actually, you can use that technique with a `DirectFieldAccessor` to replace the rebalance listener. I edited the answer. — Gary Russell, Nov 16 '20 at 23:03

KafkaBindingRebalanceListener Bean not autowired with Multi-binder

1 Answers1