Job Tracker is not running

Question

I've installed single node hadoop in my local machine. All processess are running except job tracker. I want to run my job from Ozzie, so for the same I need Job Tracker running.

JPS Output

12545 Jps
12371 SecondaryNameNode
12180 DataNode
12046 NameNode

start-dfs.sh Output

19/09/15 18:47:24 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [localhost]
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-hduser-namenode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-hduser-datanode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-hduser-secondarynamenode-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
19/09/15 18:47:41 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

start-yarn.sh Output

starting yarn daemons
starting resourcemanager, logging to /usr/local/hadoop/logs/yarn-hduser-resourcemanager-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out
localhost: starting nodemanager, logging to /usr/local/hadoop/logs/yarn-hduser-nodemanager-ec2-18-222-170-204.us-east-2.compute.amazonaws.com.out

cat mapred-site.xml

<configuration>
 <property>
  <name>mapred.job.tracker</name>
  <value>localhost:54311</value>
  <description>The host and port that the MapReduce job tracker runs
  at.  If "local", then jobs are run in-process as a single map
  and reduce task.
  </description>
 </property>
    <property>
            <name>mapreduce.framework.name</name>
            <value>yarn</value>
    </property>
    <property>
            <name>yarn.app.mapreduce.am.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
    <property>
            <name>mapreduce.map.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
    <property>
            <name>mapreduce.reduce.env</name>
            <value>HADOOP_MAPRED_HOME=$HADOOP_INSTALL</value>
    </property>
</configuration>

cat yarn-site.xml

<configuration>
    <property>
            <name>yarn.acl.enable</name>
            <value>0</value>
    </property>

    <property>
            <name>yarn.resourcemanager.hostname</name>
            <value>103.101.71.26</value>
    </property>

    <property>
            <name>yarn.nodemanager.aux-services</name>
            <value>mapreduce_shuffle</value>
    </property>
</configuration>

As per the mapred-site.xml file job tracker should run on localhost:54311, but its not running as service when I do JPS not even in web browser.

May I know please what can be done to run it ..?

Thanks

Related - https://stackoverflow.com/questions/30863615/why-we-are-configuring-mapred-job-tracker-in-yarn — OneCricketeer, Sep 15 '19 at 17:10

OneCricketeer · Answer 1 · 2019-09-15T17:11:01.040

1

JobTracker is a Hadoop 1 concept, and was replaced by YARN ResourceManager. Oozie uses YARN, too

It's not clear what you followed to setup your configurations, but this property isn't mentioned in official installation guides.

https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/SingleCluster.html#YARN_on_a_Single_Node

In fact, majority of the property starting with mapred.* are deprecated and you can find valid mapred-site XML properties here - https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

edited Sep 15 '19 at 17:11

answered Sep 15 '19 at 16:56

OneCricketeer

179,855
19
132
245

But am getting this error "org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.net.BindException: Problem binding to [103.101.71.26:8031] java.net.BindException: Cannot assign requ ested address; For more details see: http://wiki.apache.org/hadoop/BindException" – Jenny Rose Sep 15 '19 at 16:59
I've followed this configuration https://www.bogotobogo.com/Hadoop/BigData_hadoop_Install_on_ubuntu_single_node_cluster.php – Jenny Rose Sep 15 '19 at 17:00
Well, I would suggest following the official Apache guide... Or if you're in AWS anyway (based on your logs), why not just use EMR? That error you're getting is for YARN. Either that ip is wrong, or something is already running on that port – OneCricketeer Sep 15 '19 at 17:07
DO you have any refernce which I could use to schedule job using ozzie ..? I'm using this https://www.edureka.co/blog/apache-oozie-tutorial/, but it uses jobtracker and you said its depriciated. – Jenny Rose Sep 15 '19 at 17:18
That's still the resource manager address. https://stackoverflow.com/questions/37693868/jobtracker-property-in-job-properties-of-oozie – OneCricketeer Sep 15 '19 at 17:28

Job Tracker is not running

1 Answers1