0

I install hadoop in c:\hadoop2.5.2 and then unzip flume binary file in it under c:\hadop2.5.2\apache-flume-1.6.0

I am getting the error as shown below any idea ?

C:\hadoop2.5.2\apache-flume-1.6.0\bin>flume-ng agent -conf -f conf/flume-twitter.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent

C:\hadoop2.5.2\apache-flume-1.6.0\bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File C:\hadoop2.5.2\apache-
flume-1.6.0\bin\flume-ng.ps1 agent -conf -f conf/flume-twitter.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent
C:\hadoop2.5.2\apache-flume-1.6.0\bin\flume-ng.ps1 : A parameter cannot be found that matches parameter name 'Dflume.root.logger=DEBUG,cons
ole'.
At line:0 char:1
+  <<<<
    + CategoryInfo          : InvalidArgument: (:) [flume-ng.ps1], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : NamedParameterNotFound,flume-ng.ps1

C:\hadoop2.5.2\apache-flume-1.6.0\bin>

I have set parameters in flume.sh 

export JAVA_HOME=C:\Java\jdk1.7.0_79\

# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote"

# Note that the Flume conf directory is always included in the classpath.
FLUME_CLASSPATH="C:\hadoop2.5.2\apache-flume-1.6.0\lib\flume-sources-1.0-SNAPSHOT.jar"

Please guys any idea how to solve this ?

Community
  • 1
  • 1
srk
  • 599
  • 1
  • 6
  • 17
  • I think you may need to change `-Dflume.root.logger=DEBUG,console` to `-Dflume.root.logger="DEBUG,console"`.. did you try this already? – vmachan Jan 28 '16 at 17:55
  • @vmachan its not working – srk Jan 28 '16 at 18:35
  • does it still give you the same error? – vmachan Jan 28 '16 at 18:36
  • @vmachan i figured it out but now it gives another error "Problem accessing '/1/statuses/filter.json'. Reason: e> Unauthorized" but my credentials are correct. – srk Jan 28 '16 at 19:37
  • Could you post the updated command and error log?.. If you are running this on Windows.. the forward slashes "might" be causing the issue.. also the path `/1/statuses/filter.json' does look problematic, where does this get configured? – vmachan Jan 28 '16 at 19:41
  • @vmachan i solved that issue too, its time zone. Now there is an another issue. Its not writting the log file says "HIVE_HOME and HBASE_HOME" not found + "java.io.IOException: Incomplete HDFS URI, no host:" – srk Jan 28 '16 at 19:58

1 Answers1

2

I've gotten the same issue. It seems the document on the Flume website are outdated. Here is the example I have tried and it works for me.

bin\flume-ng agent -conf conf -conf-file conf/flume-conf.properties --name agent1 -property "flume.root.logger=INFO,console"

Replace -D by -property Let reference source code of Apache Flume on the Github. See from the line 20.

So, you should try:

bin\flume-ng agent -conf -f conf/flume-twitter.conf -property "flume.root.logger=INFO,console" -n TwitterAgent
NangSaigon
  • 1,253
  • 12
  • 14
  • Yes, looks like documentation doesn't seem to be up to date. Easier way to check is to check your flume-ng.ps1 in bin folder. In that, `-property ""name1=val;name2=val; .."" sets a JDK system property value` is what referring to JVM run time arguments. – sbm Sep 15 '16 at 04:32