Unable to find 'resource-types.xml'. This shows while I try to run wordcount job on hadoop on win 10

Question

The job would start but just get stuck and wont do anything. The files and job info are as follows: 1. Job Information on cmd

PS C:\hadoop-env\hadoop-3.2.1> hadoop jar C:\hadoop-env\hadoop-3.2.1\share\hadoop\mapreduce\hadoop-mapreduce-examples-3.2.1.jar wordcount /input/senbad.txt /output
2023-04-15 14:03:47,011 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
2023-04-15 14:03:47,856 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/shekh/.staging/job_1681547354686_0002
2023-04-15 14:03:47,960 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,095 INFO input.FileInputFormat: Total input files to process : 1
2023-04-15 14:03:48,150 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,176 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,189 INFO mapreduce.JobSubmitter: number of splits:1
2023-04-15 14:03:48,286 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
2023-04-15 14:03:48,302 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1681547354686_0002
2023-04-15 14:03:48,303 INFO mapreduce.JobSubmitter: Executing with tokens: []
2023-04-15 14:03:48,498 INFO conf.Configuration: resource-types.xml not found
2023-04-15 14:03:48,499 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.
2023-04-15 14:03:48,969 INFO impl.YarnClientImpl: Submitted application application_1681547354686_0002
2023-04-15 14:03:49,014 INFO mapreduce.Job: The url to track the job: http://LAPTOP-SEF3N6DS:8088/proxy/application_1681547354686_0002/
2023-04-15 14:03:49,015 INFO mapreduce.Job: Running job: job_1681547354686_0002

2. core site config

<configuration>
   <property>
       <name>fs.defaultFS</name>
       <value>hdfs://localhost:9000</value>
   </property>
</configuration>

3. hdfs site config

<configuration>
<property>
       <name>dfs.replication</name>
       <value>1</value>
   </property>
   <property>
       <name>dfs.namenode.name.dir</name>
       <value>C:\hadoop-env\hadoop-3.2.1\data\dfs\namenode</value>
   </property>
   <property>
       <name>dfs.datanode.data.dir</name>
       <value>C:\hadoop-env\hadoop-3.2.1\data\dfs\datanode</value>
   </property>

</configuration>

4. mapred site config

<configuration>
   <property>
       <name>mapreduce.framework.name</name>
       <value>yarn</value>
   </property>
</configuration>

5. yarn site config

<configuration>
   <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
   </property>
   <property>
          <name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>  
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
   </property>
</configuration>

I created a resource-types.xml file in the path where all xml files are located and i reffered to the answer in this link

How do I fix a "resouce-types.xml" error?

resource-type config

<configuration>
  <resources>
    <resourceType name="GPU" units="NONE">
      <schedulerInclude>true</schedulerInclude>
      <yarnInclude>true</yarnInclude>
    </resourceType>
    <resourceType name="FPGA" units="NONE">
      <schedulerInclude>true</schedulerInclude>
      <yarnInclude>true</yarnInclude>
    </resourceType>
  </resources>
</configuration>

My friends which dont have this resource-type file were able to run the job fine but I am having this issue.

That line isn't an error.... If your job fails, open `The url to track the job` link. For example, if jobs are stuck, maybe YARN processes aren't actually running — OneCricketeer, Apr 15 '23 at 20:38
@OneCricketeer when i run jps command on cmd all 5 operations are running — Shekhar Gangal, Apr 16 '23 at 17:20
Okay, and when you open YARN UI? You see your job? But how many healthy nodes? How much cluster memory is available? — OneCricketeer, Apr 17 '23 at 12:29

score 0 · Answer 1 · answered Aug 16 '23 at 02:07

Had the same error, This helped me solve my error. In my case the health report from http://localhost:8088/cluster/nodes/unhealthy indicated a directory that had permission issues. Saved me a lot of trouble.

This was the origin solution https://stackoverflow.com/a/64079458/14028754

Unable to find 'resource-types.xml'. This shows while I try to run wordcount job on hadoop on win 10

1 Answers1