Flume twitter config error

Question

I am trying to extract twitter data using flume. but i am getting the following error

15/04/08 23:16:36 ERROR node.PollingPropertiesFileConfigurationProvider: Unhandled error
 java.lang.NoSuchMethodError: twitter4j.conf.Configuration.isStallWarningsEnabled()Z
at twitter4j.TwitterStreamImpl.<init>(TwitterStreamImpl.java:60)
at twitter4j.TwitterStreamFactory.<clinit>(TwitterStreamFactory.java:40)
at com.cloudera.flume.source.TwitterSource.<init>(TwitterSource.java:64)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
at java.lang.Class.newInstance(Class.java:433)
at org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:42)
at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:327)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:102)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)

I have used the flume-sources-1.0-SNAPSHOT.jar from cloudera.The TwitterAgent runs with the above mentioned error. Is there any work around for it?

Thanks in advance.

Add more content to your question. What did you do to provoke this error? — Peanut, Apr 08 '15 at 12:26

score 0 · Accepted Answer · answered Apr 09 '15 at 09:08

0

This is obviously a dependency error. the flume-sources library expects a version of Twitter4j that isn't present, hence the NoSuchMethod error. I would suggest that you pull the right versions, whihc would be

1.6.0-SNAPSHOT for twitter source and 3.0.3 for twitter4j. You should consult flume's pom.xml, which has all the version info you need.

It should be noted that you should use the most current version as possible, as old implementations will not work. Twitter broke their old APIs in the meantime.

Hope this helps.

answered Apr 09 '15 at 09:08

Erik Schmiegelow

2,739
1
18
22

Where can we download 1.6.0-SNAPSHOT jar? – MChirukuri Jun 22 '15 at 17:09
You'll need to check out the flume sources for that and build with maven. – Erik Schmiegelow Jun 22 '15 at 22:22

score 0 · Answer 2 · answered Jul 06 '16 at 06:38

This is an issue with fully qualified name of the Class in your Agent.conf file. In the older versions, class name is: com.cloudera.flume.source.TwitterSource In the latest version of the flume TwitterSource is already shipped and no need to download separately. The class name is changed to org.apache.flume.source.twitter.TwitterSource

Please carefully change the class name, defitely It will work for you

Flume twitter config error

2 Answers2