Apache Flume that connects to Twitter firehose.
Requires the consumer and access tokens and secrets of a Twitter developer account to access.
Questions tagged [flume-twitter]
79 questions
1
vote
1 answer
Flume-ng hdfs sink .tmp file refresh rate control proprty
I am trying to refresh the .tmp file with additional events in every 5 minutes, my source is slow and it takes 30 min to get 128MB file in my hdfs sink.
Is there any property in flume hdfs sink where I can control the refresh rate of .tmp file…
RAJESH
- 404
- 6
- 18
1
vote
2 answers
Flume - TwitterSource language filter
I would like to ask your help in the following case.
I'm currently using Cloudera CDH 5.1.2 and I tried to collect Twitter data using Flume as it is described in the following porsts…
Peter Ridzi
- 21
- 5
1
vote
2 answers
Apache Flume 1.5 not giving expected results in Hadoop 2/Automatic fail-over cluster configuration
I have configured Apache Hadoop 2 cluster in HA/Automatic fail-over configuration on CentOS 6.5//64-bit. I have installed Flume 1.5 (apache-flume-1.5.0-bin.tar.gz).
I want to analyse twitter data using flume/Hive with some key words filtering. See…
I Bajwa PHD
- 1,708
- 1
- 20
- 42
0
votes
0 answers
I am unable to connect twitter source data
`Hello Team,
I hope you are good. Actually, I am in need of an assistance regarding Apache flume.
To give you the context, I am newly learning Bigdata and Flume, while loading twitter data into flume I am unable to run fully as I got the below error…
0
votes
0 answers
Is there a way to use Twitter 2.0 as a Flume Source?
TwitterSource from Apache Flume only supports v1 endpoints. I need a work around to using Twitter as the source for Apache Flume.
This is the error which pops up when I use the Flume, firehose class
2023-03-18 00:38:03,290 ERROR…
0
votes
0 answers
Streaming Twitter Data with Flume - Twitter4J error
I am streaming twitter data using Flume and but getting an error regarding the twitter4j Jar File.
01:42:03.075 [Twitter Stream consumer / [1][Establishing connection]] ERROR org.apache.flume.source.twitter.TwitterSource - Exception while streaming…
netdevmike
- 1
- 2
0
votes
1 answer
trying to collect twitter data using flume and send that to kafka but getting an ERROR regarding batch size
I believe this has something to do with my flume.conf file.
Flume_project.conf
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional…
netdevmike
- 1
- 2
0
votes
0 answers
Why is Apache Flume not retrieving Twitter data?
I download Apache Flume 1.11.0, Hadoop 3.3.1 and Java JDK 8u341. I configured everything correctly and tried to retrieve data from Twitter. My Hadoop and java does work well.
Also created a directory in localhost:
directory
The following is the conf…
0
votes
0 answers
Fetching twitter data using flumes 1.9.0
i am getting below error while trying to fetch the live tweets using flumes 1.9.0
*
content="text/html; charset=utf-8"/>\nError 403
Please use V2 filtered and sample volume stream as alternatives
…
Saurabh Singh
- 1
- 1
0
votes
1 answer
Twitter streaming data using apache flume getting error
I am working on a small task related to Hadoop and flume. I am trying to fetch the Twitter data using flume but getting the below error
ERROR node.AbstractConfigurationProvider: Sink HDFS has been removed
due to an error during configuration…
Piyush Jiwane
- 179
- 3
- 13
0
votes
2 answers
Unable to retrieve Twitter streaming data using Flume
I am trying to stream and retrieve Twitter data using Flume but unable to do so because of some sort of error.
When I try executing it using the command:
flume-ng agent -n TwitterAgent -c conf -f /home/hadoop/Flume/conf/twitter.conf
I get the…
Pratheek Menon
- 21
- 7
0
votes
0 answers
Execution error, return code 1 when trying to create an external table in Hive
I have searched and tried all solutions online and I seem to be getting the same error when trying to create an external table in hive:
FAILED: Execution Error, return code 1…
Abiodun
- 959
- 6
- 17
- 38
0
votes
1 answer
Create Form in Cloudera
I am working in Cloudera ecosystem and as a beginner, I started by implementing flume twitter example and completed it successfully. Now, whenever I want to stream data from Twitter for different keywords I have to go into the configuration file of…
zain ul abidin
- 197
- 2
- 13
0
votes
1 answer
Flume twitter stream
I am trying to execute flume to get data from twitter stream but received this error while executing the flume.
[ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start…
zain ul abidin
- 197
- 2
- 13
0
votes
1 answer
Flume agent does not contain any valid channels
I am new to Flume. Im trying to pull data from Twitter, but I am not being successful. (I am using Cloudera Quickstart)
My conf file looks like this:
TwitterAgent.sources = Twitter
TwitterAgent.channels = MemChannel
TwitterAgent.sinks =…
DevPerson
- 137
- 1
- 4
- 12