-1

I need to use Twitter Search API and get twitter data into my HDFS. The Search API uses REST calls and I am trying to understand how I would use that with Apache Flume. I have required keys for authorization to use Twitter's APIs but I just don't understand how I could implement the Search API using flume. Need Help! Thank you!

SaurabJ21
  • 105
  • 3
  • 8

2 Answers2

0
  1. create a developer account in twitter and generate your own secret keys.
  2. Add a flume-source-1.0-SNAPSHOT.jar file into lib folder.
  3. using secret keys create a flume config file to get tweets from twitter and store it into hdfs.
  4. start you flume using below command

command : bin/flume-ng agent --conf ./conf/ -f conf/flume.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent

For more details refer the below link :

http://www.thecloudavenue.com/2013/03/analyse-tweets-using-flume-hadoop-and.html

Suresh Ram
  • 1,034
  • 3
  • 16
  • 40
0

First you need to create application in twitter. Go to https://apps.twitter.com/ and create application then remember the generated four keys- Consumer key, Consumer Secret key, Access Token and Access Token Secret.

Download apache-flume-1.x.0-bin.tar.gz.here x means latest version.After untar the file add following command into ~/.bashrc file:

export FLUME_HOME=location of untar flume file
export PATH=$PATH:$FLUME_HOME/bin

after saving the above file run source ~/.bashrc in terminal.After this follow below site command:

https://acadgild.com/blog/streaming-twitter-data-using-flume/

Hetvi
  • 1
  • 1