-3

I want to bring the data from solace to hadoop using flume, can some one let me know how to write interceptor to convert protobuf to avro ?

thunderhemu
  • 492
  • 1
  • 4
  • 8

1 Answers1

0

There's a very detailed integration guide available describing how to use the JMS Flume Source to receive messages from a Solace message bus.

Is this the interface you are using?

If so the blog post by Ken Barr (https://solace.com/blog/devops/solace-as-flume-channel-technical-look) gives an implementation of both Flume Source and Sink. The complete source code is at http://dev.solace.com/wp-content/uploads/solace-flume-channel.tgz

The FlumeEventToSolaceMessageConverter.solaceToFlume() method is the one you'd need to modify to support your protobuf to avro use case. OOTB it just assumes that the body of the JMS message is an avro message.

On GitHub we found a protobuf to avro converter (vpon/protobuf-to-avro) which generates a POJO converter using a .proto schema file.

Mic Hussey
  • 41
  • 2
  • The question is I have enabled JNDI Conection factory at solace end, and I am thinking solace as JMS source I am trying to bring the data from solace using flume, what should I need to configure at sink end in flume. – thunderhemu Apr 05 '17 at 10:22