Hive進行數據統計時報錯：org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster

報錯詳情：

2020-04-09 22:56:58,827 ERROR [Listener at 0.0.0.0/45871] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:979)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1293)
    at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$6.run(MRAppMaster.java:1761)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1757)
    at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1691)
Caused by: java.lang.NullPointerException
    at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
    at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
    ... 14 more
2020-04-09 22:56:58,818 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted
2020-04-09 22:56:58,819 INFO [Ping Checker] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted

閱讀源代碼，會發現這是由於org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.java這個類中的一段代碼引起的，涉及的代碼如下：

if (rmIds != null) {
      List<String> urls = new ArrayList<>();
      for (String rmId : rmIds) {
        String url = getUrlByRmId(yarnConf, rmId);
        // urls.add(url); // 注釋掉這端代碼，修改為下面的if語句判斷是否為null
        if (url != null) {
          urls.add(url);
        }        
      }
      if (!urls.isEmpty()) {
        params.put(RM_HA_URLS, StringUtils.join(",", urls));
      }
    }

這與yarn-site.xml配置HA的兼容性有關，取決於 yarn.resourcemanager.webapp.address 和 yarn.resourcemanager.webapp.https.address 是否為空。

修改源碼的方法：

1、首先將hadoop-yarn-server-web-proxy-3.2.1.jar從服務器上拿下來。

 

2、使用JD-GUI打開，復制org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.java類中代碼；

3、在eclipse新建一個工程，然后新建一個類插入復制的代碼，並進行修改。

4、有報錯的話就build class，把本地的hadoop相關的包導入

5、將該工程導出為jar包。

6、分別打開hadoop-yarn-server-web-proxy-3.2.1.jar 和剛導出的jar包用解壓軟件打開，替換原來的org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.class文件。

修改完后替換原來的jar包，重啟hdfs和yarn.job運行成功！