Apache Flume that connects to Twitter firehose.
Requires the consumer and access tokens and secrets of a Twitter developer account to access.
Questions tagged [flume-twitter]
79 questions
0
votes
1 answer
Unknown files format of tweets from Flume
I'm trying to get tweets using Flume. I am working with cloudera
I use the twitter source provided here
Below is my configuration file:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks =…
user971961
- 65
- 1
- 4
0
votes
1 answer
Flume not accepting keywords for Twitter stream
a Hadoop neophyte here, using this tutorial: https://acadgild.com/blog/streaming-twitter-data-using-flume/ to capture tweets. Here is my flume.conf file:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks =…
JLA
- 100
- 2
- 10
0
votes
1 answer
How to filter multiple source data using Apache Flume?
I am using flume to handle multiple sources data and stored in HDFS but I could not understand how to filter data before storing in HDFS.
paul
- 519
- 6
- 13
0
votes
1 answer
How to manage the concurrency between flume agent.
I'm working on Big Data project. We are using flume to download files from sftp to HDFS.
Then, we configured 3 agents. They read from the same source. As consequence, we get 3 duplicated files into HDFS, which is not good. Whereas, we must have…
Chayma Sakouhi
- 79
- 2
- 10
0
votes
1 answer
Twitter Flume data not in readable format
My twitter data is not in readable format. I fetched twitter data using flume, which i am trying to read using hive tables. But i can't see proper data in hdfs. Below are the details:
flume-env.sh -
export…
Hellboy
- 29
- 6
0
votes
0 answers
SLF4j multiple bindings exception
error screenshot here
I am trying to stream real time twitter data into HDFS using Apache flume. And when i run the command ./flume-ng agent -c /usr/local/apache-flume-1.4.0-bin/conf/ -f /usr/local/apache-flume-1.4.0-bin/conf/flume.conf -n…
Vivek
- 1
0
votes
1 answer
Flume Twitter Streaming Issue
I'm using Flume 1.6.0-cdh5.9.1 to stream Tweets using Twitter source.
The configuration file is below:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks = HDFS
TwitterAgent.sources.Twitter.type =…
Saptarshi Dey
- 31
- 6
0
votes
1 answer
Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Cannot validate serde: org.apache.hadoop.hive.serde2.avro.AvroSerde
I am creating a hive table using avro serde to store twitter data.
Here's the code-
create table tweets
row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerde'
stored as inputformat
…
vinay
- 167
- 1
- 4
- 12
0
votes
1 answer
ERROR node.PollingPropertiesFileConfigurationProvider: Failed to load configuration data. Exception follows. org.apache.flume.FlumeException:
here moved snapshot jar in lib foldergetting this error after running flume command,
i.e
user@ubuntu:~/hadoop_store/apache-flume-1.4.0-bin/bin$ ./flume-ng agent -n TwitterAgent -c conf -f ../conf/twitter1.conf
also tried…
Abhijit
- 73
- 11
0
votes
1 answer
getting error in twitter streaming through flume
**While running the flume command i am getting the following error,i tried changing the envi variables in .bashrc along with classpath in flume.env.sh ,still no use
Picked up JAVA_TOOL_OPTIONS: -javaagent:/usr/share/java/jayatanaag.jar
16/12/08…
Abhijit
- 73
- 11
0
votes
0 answers
Taking data from twitter and load it to hdfs using Flume
I'm getting error when running following command in hadoop
bin/flume-ng agent -c /usr/local/hadoop/flume/conf -f usr/local/hadoop/flume/conf/flume-twitter.conf -n TwitterAgent - flume.root.logger=INFO,console
shows the below error when executing…
praveen
- 319
- 3
- 6
- 17
0
votes
1 answer
Issue while getting Twitter data in HDFS using Flume
I am trying to fetch the twitter data in HDFS but getting issue.
Here is my flume.conf file
TwitterAgent.sources= Twitter
TwitterAgent.channels=…
ashutosh
- 1
0
votes
1 answer
could not load Twitter source data into Flume
I am getting the following error while running import twitter source data
could not find or load main class flume.node.Application error
can anyone help me resolve this issue?
user2631600
- 759
- 1
- 11
- 18
0
votes
0 answers
Flume --PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException:
I have installed java,hadoop and flume in Ubuntu.
When I run flume job to extract data from twitter I get error like below;
Not sue the issue behind this.
sun.security.validator.ValidatorException: PKIX path building failed:…
Alka
- 267
- 1
- 9
0
votes
1 answer
hive hadoop: selecting data from table getting error
after I created an external table in Hive I wanted to know to the number of tweets so I wrote the following query but I got this error,please how to solve this problem and this is the configuration of mapred-site.xml
…
javac
- 2,819
- 1
- 20
- 22