0

I am configuring flume to stream weather data, i have written flume agent as given below,

WeatherAgent.sources= Weather

WeatherAgent.sinks = hdfs-write

WeatherAgent.channels= memory-channel

WeatherAgent.sources.Weather.type = weather

WeatherAgent.sources.Weather.bind = api.openweathermap.org/data/2.5/forecast/city?id=524901&APPID=*********

WeatherAgent.sources.Weather.port = 11111

WeatherAgent.sinks.hdfs-write.type = hdfs

WeatherAgent.sinks.hdfs-write.hdfs.path = hdfs://localhost:8020/user/hadoop/flume

WeatherAgent.sinks.hdfs-write.rollInterval = 30

WeatherAgent.sinks.hdfs-write.hdfs.writeFormat=Text

WeatherAgent.sinks.hdfs-write.hdfs.fileType=DataStream

WeatherAgent.channels.memoryChannel.type = memory

WeatherAgent.channels.memoryChannel.capacity=10000

WeatherAgent.sources.Weather.channels=memoryChannel

WeatherAgent.sinks.hdfs-write.channel=memoryChannel

i am getting error as

weather agent does not contain any valid channels making it as invalid.

i am new to flume,i dont know which all parameters to be addded in agent config file,i even dont know written agent is correct or wrong,please help me to solve this. if any Help links are there to write custom or new agents please post.

thanks in advance

Suyog

Suyog
  • 3
  • 6

1 Answers1

0
 it is type mismatching. In your configuration file you mentioned  

    WeatherAgent.channels= memory-channel

  but you are using  memoryChannel instead of memory-channel

    WeatherAgent.channels.memoryChannel.type = memory

    WeatherAgent.channels.memoryChannel.capacity=10000

    WeatherAgent.sources.Weather.channels=memoryChannel
Sravan K Reddy
  • 1,082
  • 1
  • 10
  • 19
  • thank you sravan for your help .. channel issue is resolve but i am not able to get data its giving warning as no configuration found for this host. – Suyog Mar 17 '15 at 07:23
  • thank you sravan for your help .. channel issue is resolve but i am not able to get data its giving warning as no configuration found for this host.it is stopping by saying starting new configuration,please help me on this – Suyog Mar 17 '15 at 07:31
  • hi sravan with your help i am able to solve above errors now i am getting one more error "UNABLE TO START EVENTDRIVENSOURCE RUNNER " please help me on this – Suyog Mar 17 '15 at 10:26
  • can you post detailed error log – Sravan K Reddy Mar 17 '15 at 10:39
  • hi sorry to ask this but can please tell me where can i find error log of flume becoz log4j properties file does not any information realated to this agent. and please tell me where can put all text of error coz here comment opton has charector limit i cant copy entire exception here. please help i am new on this. – Suyog Mar 17 '15 at 10:55
  • Hi sravan i can copy only this much in comment please help on this ERROR lifecycle.LifecycleSupervisor: Unable to start EventDriv enSourceRunner: { source:org.apache.flume.source.NetcatSource{name:Weather,state :IDLE} } - Exception follows. org.apache.flume.FlumeException: java.net.SocketException: Unresolved address at org.apache.flume.source.NetcatSource.start(NetcatSource.java:168) at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSour ceRunner.java:44) – Suyog Mar 18 '15 at 05:31