0

I download Apache Flume 1.11.0, Hadoop 3.3.1 and Java JDK 8u341. I configured everything correctly and tried to retrieve data from Twitter. My Hadoop and java does work well. Also created a directory in localhost: directory

The following is the conf file:

    TwitterAgent.sources = Twitter 
    TwitterAgent.channels = MemChannel 
    TwitterAgent.sinks = HDFS 
    TwitterAgent.sources.Twitter.type = org.apache.flume.source.twitter.TwitterSource
    TwitterAgent.sources.Twitter.consumerKey = 
    TwitterAgent.sources.Twitter.consumerSecret = 
    TwitterAgent.sources.Twitter.accessToken = 
    TwitterAgent.sources.Twitter.accessTokenSecret = 
    TwitterAgent.sources.Twitter.keywords = tutorials point,java, bigdata, mapreduce, mahout, hbase, nosql
    TwitterAgent.sinks.HDFS.type = hdfs
    TwitterAgent.sinks.HDFS.hdfs.path = 
    TwitterAgent.sinks.HDFS.hdfs.fileType = DataStream 
    TwitterAgent.sinks.HDFS.hdfs.writeFormat = Text 
    TwitterAgent.sinks.HDFS.hdfs.batchSize = 1000
    TwitterAgent.sinks.HDFS.hdfs.rollSize = 0 
    TwitterAgent.sinks.HDFS.hdfs.rollCount = 10000 
    TwitterAgent.channels.MemChannel.type = memory 
    TwitterAgent.channels.MemChannel.capacity = 10000 
    TwitterAgent.channels.MemChannel.transactionCapacity = 100
    TwitterAgent.sources.Twitter.channels = MemChannel
    TwitterAgent.sinks.HDFS.channel = MemChannel 


But while using Flume to retrieve data, it gives the following message: cmd_message

Even after a hour, this stays like this. It doesn't show errors and no data is collected... Should I provide any other info? What should be done?

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245

0 Answers0