Questions tagged [flume-ng]

Apache Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. The Flume-NG is refactoring of the first generation Flume to solve certain known issues and limitations of the original design.

This tag should be used with questions about Flume-NG API and specific features of new-generation versions (e.g. Flume HDFS Sink was introduced only in NG version and cannot be used in previous releases).

397 questions
-1
votes
1 answer

Streaming Kafka Messages to MySQL Database

I want to write Kafka messages to MySQL database. There is an example in this link. In that example, apache flume used for consume messages and writing it to MySQL. I'm using same code and when i run the flume-ng agent and event always becomes…
fena coder
  • 217
  • 4
  • 17
-1
votes
1 answer

Copy files from my local filesystem to HDFS using Flume

Using a file generator in java, I will have a stream of directories and files in my local filesystem that I need to move in HDFS. I search in internet and i saw that i can use Flume, however I didn't find any resources that explains to me how to do…
Yassine
  • 33
  • 1
  • 6
-1
votes
1 answer

Flume plugin sources from RabbitMQ Queue and Sink to Cassandra

Is there any flume plugin, Which sources from RabbitMQ Queue and Sink to Cassandra. RabbitMQ <== Flume Source ==> Sink ==> Cassandra. Or any other way to consume RabbitMQ Queue and store in Cassandra.
uttam palkar
  • 1,585
  • 5
  • 14
  • 17
-1
votes
1 answer

Pubmed DataSet - Connecting FTP with Flume

To retrieve the datas from Pubmed DataSet [ncbi], I used FireFTP Addon (in firefox) to retrieve the xml, pdf, txt contents. [http://www.ncbi.nlm.nih.gov/pmc/tools/ftp/] I have successfully installed Apache Flume. The main objective is -- I need to…
Soundarya Thiagarajan
  • 574
  • 2
  • 13
  • 31
-1
votes
1 answer

Running flume agent with custom source

I am trying to configure an flume agent with custom source from here, I tried to run flume agent using command flume-ng agent --conf conf --conf-file conf/twitter1.conf --name TwitterAgent But I am unable to start my Flume-agent, it shows error…
riyasbase
  • 3
  • 3
-1
votes
2 answers

Flume twitter config error

I am trying to extract twitter data using flume. but i am getting the following error 15/04/08 23:16:36 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error java.lang.NoSuchMethodError:…
Hussain Shaik
  • 117
  • 1
  • 4
  • 11
-3
votes
1 answer

Writing flume interceptors to bring the data from solace

I want to bring the data from solace to hadoop using flume, can some one let me know how to write interceptor to convert protobuf to avro ?
thunderhemu
  • 492
  • 1
  • 4
  • 8
1 2 3
26
27