Spring Autowired Shared Queue NullPointerException

Question

I'm using Spring for the first time and am trying to implement a shared queue wherein a Kafka listener puts messages on the shared queue, and a ThreadManager that will eventually do something multithreaded with the items it takes off the shared queue. Here is my current implementation:

The Listener:

@Component
public class Listener {
    @Autowired
    private QueueConfig queueConfig;
    private ExecutorService executorService;
    private List<Future> futuresThread1 = new ArrayList<>();
    public Listener() {
        Properties appProps = new AppProperties().get();
        this.executorService = Executors.newFixedThreadPool(Integer.parseInt(appProps.getProperty("listenerThreads")));
    }
    //TODO: how can I pass an approp into this annotation?
    @KafkaListener(id = "id0", topics = "bose.cdp.ingest.marge.boseaccount.normalized")
    public void listener(ConsumerRecord<?, ?> record) throws InterruptedException, ExecutionException
        {
            futuresThread1.add(executorService.submit(new Runnable() {
                    @Override public void run() {
                        try{
                            queueConfig.blockingQueue().put(record);
//                            System.out.println(queueConfig.blockingQueue().take());
                        } catch (Exception e){
                            System.out.print(e.toString());
                        }

                    }
            }));
        }
}

The Queue:

@Configuration
public class QueueConfig {
    private Properties appProps = new AppProperties().get();

    @Bean
    public BlockingQueue<ConsumerRecord> blockingQueue() {
        return new ArrayBlockingQueue<>(
                Integer.parseInt(appProps.getProperty("blockingQueueSize"))
        );
    }
}

The ThreadManager:

@Component
public class ThreadManager {
    @Autowired
    private QueueConfig queueConfig;
    private int threads;

    public ThreadManager() {
        Properties appProps = new AppProperties().get();
        this.threads = Integer.parseInt(appProps.getProperty("threadManagerThreads"));
    }


    public void run() throws InterruptedException {
        ExecutorService executorService = Executors.newFixedThreadPool(threads);
        try {
            while (true){
                queueConfig.blockingQueue().take();
            }
        } catch (Exception e){
            System.out.print(e.toString());
            executorService.shutdownNow();
            executorService.awaitTermination(1, TimeUnit.SECONDS);
        }
    }
}

Lastly, the main thread where everything is started from:

@SpringBootApplication
public class SourceAccountListenerApp {
    public static void main(String[] args) {
        SpringApplication.run(SourceAccountListenerApp.class, args);
        ThreadManager threadManager = new ThreadManager();
        try{
            threadManager.run();
        } catch (Exception e) {
            System.out.println(e.toString());
        }
    }
}

The problem

I can tell when running this in the debugger that the Listener is adding things to the queue. When the ThreadManager takes off the shared queue, it tells me the queue is null and I get an NPE. It seems like autowiring isn't working to connect the queue the listener is using to the ThreadManager. Any help appreciated.

I updated my post to show the main thread that serves as the app entry point. — A.A., Sep 18 '18 at 13:07
ThreadManager threadManager = new ThreadManager(); This is the problem. ThreadManager is not a bean, since you create the object manually. Is you app a webapp? command line? You need to have a entrypoint to enable the ThreadManager. — Deividi Cavarzan, Sep 18 '18 at 13:31
This is a microservice. First time doing that as well. So I need to make ThreadManager into a bean? — A.A., Sep 18 '18 at 13:49
I post an answer, I think that solves. You need to use the ThreadManager as a bean, not via manually instance creation. — Deividi Cavarzan, Sep 18 '18 at 13:53

Tommy Brettschneider · Answer 1 · 2018-09-17T21:47:04.203

You use Spring´s programatic, so called 'JavaConfig', way of setting up Spring beans (classes annotated with @Configuration with methods annotated with @Bean). Usually at application startup Spring will call those @Bean methods under the hood and register them in it's application context (if scope is singleton - the default - this will happen only once!). No need to call those @Bean methods anywhere in your code directly... you must not, otherwise you will get a separate, fresh instance that possibly is not fully configured!

Instead, you need to inject the BlockingQueue<ConsumerRecord> that you 'configured' in your QueueConfig.blockingQueue() method into your ThreadManager. Since the queue seems to be a mandatory dependency for the ThreadManager to work, I'd let Spring inject it via constructor:

@Component
public class ThreadManager {

    private int threads;

    // add instance var for queue...
    private BlockingQueue<ConsumerRecord> blockingQueue;

    // you could add @Autowired annotation to BlockingQueue param,
    // but I believe it's not mandatory... 
    public ThreadManager(BlockingQueue<ConsumerRecord> blockingQueue) {
        Properties appProps = new AppProperties().get();
        this.threads = Integer.parseInt(appProps.getProperty("threadManagerThreads"));
        this.blockingQueue = blockingQueue;
    }

    public void run() throws InterruptedException {
        ExecutorService executorService = Executors.newFixedThreadPool(threads);
        try {
            while (true){
                this.blockingQueue.take();
            }
        } catch (Exception e){
            System.out.print(e.toString());
            executorService.shutdownNow();
            executorService.awaitTermination(1, TimeUnit.SECONDS);
        }
    }
}

Just to clarify one more thing: by default the method name of a @Bean method is used by Spring to assign this bean a unique ID (method name == bean id). So your method is called blockingQueue, means your BlockingQueue<ConsumerRecord> instance will also be registered with id blockingQueue in application context. The new constructor parameter is also named blockingQueue and it's type matches BlockingQueue<ConsumerRecord>. Simplified, that's one way Spring looks up and injects/wires dependencies.

Thanks Tommy, this is a great explanation and I found it very helpful. Now that ThreadManager is taking the blockingQueue parameter, I need to pass it in where I call ThreadManager (I've added the main thread to my original post near the bottom). Or do I need to configure Spring to also execute ThreadManager somehow? — A.A., Sep 18 '18 at 13:12
I suppose the `@Bean CommandLineRunner runner(ThreadManager manager){}` method in class `SourceAccountListenerApp` (as suggested by Deividi Cavarzan) should be just fine. that way the fully Spring configured ThreadManager instance with it's injected BlockingQueue will be injected as a method parameter... — Tommy Brettschneider, Sep 18 '18 at 14:07
Thanks. Wish I could check your answer off as well as most helpful as you both helped alot. — A.A., Sep 18 '18 at 14:43

score 3 · Accepted Answer · answered Sep 18 '18 at 13:51

This is the problem:

ThreadManager threadManager = new ThreadManager();

Since you are creating the instance manually, you cannot use the DI provided by Spring.

One simple solution is implement a CommandLineRunner, that will be executed after the complete SourceAccountListenerApp initialization:

@SpringBootApplication
public class SourceAccountListenerApp {
    public static void main(String[] args) {
        SpringApplication.run(SourceAccountListenerApp.class, args);            
    }

    // Create the CommandLineRunner Bean and inject ThreadManager 
    @Bean
    CommandLineRunner runner(ThreadManager manager){
        return args -> {
            manager.run();
        };
    }

}

Spring Autowired Shared Queue NullPointerException

2 Answers2