Stack Overflow
Stack Overflow

Questions tagged [flume-twitter]

Apache Flume that connects to Twitter firehose.

Requires the consumer and access tokens and secrets of a Twitter developer account to access.

Flume-Twitter

79 questions
2
votes
0 answers

How to load a CSV (comma seperated) file into HBase table using Flume?

I want to load a CSV (just comma separated) file into my Hbase table. I already tried it with help of some googled articles, now just I am able to load entire row (or line) as value into Hbase, i.e. all values in single row are getting stored as…
prasad
  • 339
  • 8
  • 23
2
votes
1 answer

How to pass parameters in flume

I have a twitter flume source defined like this in the flume configuration file TwitterAgent.sources.Twitter.type = com.cloudera.flume.source.TwitterSource TwitterAgent.sources.Twitter.channels = MemChannel TwitterAgent.sources.Twitter.consumerKey =…
Praveen Kumar
  • 107
  • 2
  • 6
1
vote
1 answer

Flume twitter - Please use V2 filtered and sample volume stream as alternatives

I am trying to ingest some twitter data though apache flume and so, sink to kafka. This is only for study purpose and I am stuck in this erro: root@engsrv:/de/apache-flume-1.10.0-bin/bin# ./flume-ng agent --conf-file…
Potter
  • 434
  • 8
  • 21
1
vote
0 answers

getting error : accessing '/1.1/statuses/sample.json?stall_warnings=true'. Reason: Please use V2 filtered and sample volume stream as alternatives

Problem accessing '/1.1/statuses/sample.json?stall_warnings=true'. Reason: Please use V2 filtered and sample volume stream as alternatives. can anyone help me to get out from these? what will be the changes in configuration?
anonymus
  • 11
  • 1
1
vote
1 answer

Running flume agent to get Twitter data

I have been trying to run a flume agent on my windows system to get twitter data. I am following this blog https://acadgild.com/blog/streaming-twitter-data-using-flume But, whenever i try to run the flume agent I get the follwing error - flume-ng…
Shwetabh Dixit
  • 45
  • 8
1
vote
0 answers

Insert statement on Hive using Oozie never ends

im new with cloudera...., Im already working with oozie, flume, hive and impala...., I have a process that manually works as I want. I've wanted to improve the functionality using oozie for to optimize processes. Flume, is not there cause Flume…
Cristhian Plaza
  • 11
  • 3
1
vote
1 answer

Using FLUME to store data in Hadoop

I have followed all the steps for hadoop installation and Flume from tutorials. I am a naive in Big Data tools. I am getting the following errors. I dont understand, where the problem is? I have also read a lot of post on installation, but still I…
Shivam
  • 113
  • 1
  • 10
1
vote
0 answers

Error while creating an External Table in hive and transfer twitter data in avro format to this table

I used Flume to fetch Twitter data. This data is stored as avsc file in hdfs. I created avro schema file "tweeter.avsc" and saved it in hdfs. But when I am trying to Create External Table using command below I get error. Command: CREATE EXTERNAL…
Vishal Sharma
  • 175
  • 1
  • 3
  • 8
1
vote
3 answers

What should be flume.conf parametres for save tweets to single FlumeData file per hour?

We are saving tweets in a directory order like /user/flume/2016/06/28/13/FlumeData... .But each hour it creates more than 100 FlumeData file.I have changed TwitterAgent.sinks.HDFS.hdfs.rollSize = 52428800 (50 mb) same thing happened again.After that…
mgurcan
  • 170
  • 1
  • 12
1
vote
3 answers

Hadoop: java.net.ConnectException: Connection refused

Hello I have been trying to follow this tutorial: http://www.tutorialspoint.com/apache_flume/fetching_twitter_data.htm for a very long time now and I am absolutely stuck at Step 3: Create a Directory in HDFS. I have ran start-dfs.sh and…
Troy Zuroske
  • 762
  • 1
  • 12
  • 31
1
vote
1 answer

Reading gz.parquet file

Hello I need to read the data from gz.parquet files but dont know how to?? Tried with impala but i get the same result as parquet-tools cat without the table structure. P.S: any suggestions to improve the spark code are most welcome. I have the…
Mouzzam Hussain
  • 449
  • 3
  • 8
  • 20
1
vote
1 answer

Unhandled error java.lang.NoSuchMethodError while running flume for downloading twitter data

When i run this command hduser@ubuntu:/usr/local/flume$ bin/flume-ng agent --conf ./conf/ -f conf/twitterflume-agent.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent the flume is getting started but after some time it throws me the exception…
vimal Raj kumar
  • 11
  • 2
1
vote
1 answer

Exception follows-org.apache.flume.FlumeException: Unable to load source type: com.cloudera.flume

I am trying to do twitter analysis using Flume. For getting the tweets from the twitter i have set all the required params (consumerKey, consumerSecret, accessToken and accessTokenSecret) in the flume.conf files . TwitterAgent.sources =…
Nick La
  • 11
  • 6
1
vote
0 answers

Flume channel Capacity is full giving exception

I have started flume Twitteragent to fetch the twitter data into the hdfs after few mins the data was not able to write to the hdfs it was poping a message in the terminal as mentioned below. Error org.apache.flume.ChannelException: Take list for…
Ashish Singh
  • 51
  • 2
  • 8
1
vote
4 answers

Exception follows. org.apache.flume.FlumeException: Unable to load source type in flume twitter analysis

I am trying to do twitter analysis using Flume and Hive . For getting the tweets from the twitter i have set all the required params (consumerKey, consumerSecret, accessToken and accessTokenSecret) in the flume.conf files . TwitterAgent.sources =…
user3752667
1
2
3 4 5 6