Scala Netty is there any way to share a ReplayingDecoder

Question

I am looking to open up multiple connections using a netty client bootstrap in order to parse messages coming from multiple sources. The messages all have the same format, however, due to the amount of data that needs to be processed, I must run each connection on separate threads (This is assuming netty creates a thread per client channel, which I couldn't find a reference for - if that's not the case, how would this be achieved?).

This is the code that I use to connect to the data server:

var b = new Bootstrap()
        .group(group)
        .channel(classOf[NioSocketChannel])
        .handler(RawFeedChannelInitializer)


var ch1 = b.clone().connect(host, port).sync().channel();
var ch2 = b.clone().connect(host, port).sync().channel();

The initializer calls RawPacketDecoder, which extends ReplayingDecoder, and is defined here. The code works well without @Sharable when opening a single connection, but for the purpose of my application I must connect to the same server multiple times.

This results in the runtime error @Sharable annotation is not allowed pointing to my RawPacketDecoder class.

I am not entirely sure on how to get past this issue, short of reimplementing in scala an instantiable class of ReplayingDecoder as my decoder based directly on ByteToMessageDecoder.

Any help would be greatly appreciated.

Note: I am using netty 4.0.32 Final

score 1 · Accepted Answer · edited May 23 '17 at 12:01

I found the solution in this StockExchange answer.

My issue was that I was using an object based ChannelInitializer (singleton), and ReplayingDecoder as well as ByteToMessageDecoder are not sharable.

My initializer was created as a scala object, and therefore a single instance allowed. Changing the initializer to a scala class and instantiating for each bootstrap clone solved the problem. I modified the bootstrap code above as follows:

var b = new Bootstrap()
    .group(group)
    .channel(classOf[NioSocketChannel])
    //.handler(RawFeedChannelInitializer)

var ch1 = b.clone().handler(new RawFeedChannelInitializer()).connect(host, port).sync().channel();
var ch2 = b.clone().handler(new RawFeedChannelInitializer()).connect(host, port).sync().channel();

I am not sure whether this ensures multithreading as wanted but it does allow to split the data access into multiple connections to the feed server.

Edit Update: After performing additional research on the subject, I have determined that netty does in fact create a thread per channel; this was verified by printing to console after the creation of each channel:

println("No. of active threads: " + Thread.activeCount());

The output shows an incremental number as channels are created and associated with their respective threads.

By default NioEventLoopGroup uses 2*Num_CPU_cores threads as defined here:

DEFAULT_EVENT_LOOP_THREADS = Math.max(1, SystemPropertyUtil.getInt(
              "io.netty.eventLoopThreads",
               Runtime.getRuntime().availableProcessors() * 2));

This value can be overriden to something else by setting

val group = new NioEventLoopGroup(16)

and then using the group to create/setup the bootstrap.

Netty doesn't make a thread per channel, it shares all incoming channels over its own threads, and 1 thread can have multiple channels if there are more channels than threads — Ferrybig, Jan 17 '17 at 12:26

Scala Netty is there any way to share a ReplayingDecoder

1 Answers1