1.org.apache.hadoop.yarn.exceptions.InvalidAuxServiceException: The auxService:mapreduce_shuffle does not exist
問題分析:The auxService:mapreduce_shuffle does not exist;yarn-site.xml配置文件有問題
解決方法:
1)vim /usr/local/hadoop/etc/hadoop/yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
2)重新啟動yarn:
/usr/local/hadoop/sbin/stop-yarn.sh
/usr/local/hadoop/sbin/start-yarn.sh
2.ERROR tool.ImportTool: Import failed: org.apache.hadoop.mapred.FileAlreadyExistsException: Output directory hdfs://192.168.15.115:9000/hive/root already exists
解決方法: hadoop fs -rm -r /hive/root
3.ERROR exec.DDLTask: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.ObjectMapper.readerFor(Ljava/lang/Class;)Lcom/fasterxml/jackson/databind/ObjectReader;
造成的原因:sqoop與hive使用的jackson包的版本沖突導致
解決方法:將sqoop的lib目錄下jackson*.jar包備份,hive的lib目錄下的jackson相關jar包拷貝到sqoop的lib目錄下
4. ERROR hive.HiveConfig: Could not load org.apache.hadoop.hive.conf.HiveConf. Make sure HIVE_CONF_DIR is set correctly.
解決方法:
1)/etc/profile最后加入 export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$HIVE_HOME/lib/*
然后刷新配置,source /etc/profile
2)復制hive庫文件中hive-exec*到sqoop的庫文件中
5. ERROR Could not register mbeans java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
解決方法:
vim /usr/local/java/jre/lib/security/java.policy
新增語句:
grant {
// JMX Java Management eXtensions
permission javax.management.MBeanTrustPermission "register";
};
6.創建job的時候報錯:
Exception in thread "main" java.lang.NoClassDefFoundError: org/json/JSONObject
造成的原因:這是因為sqoop缺少java-json.jar包.
解決方法:
java-json.jar下載地址:http://www.java2s.com/Code/Jar/j/Downloadjavajsonjar.htm
將java-json.jar放到/usr/local/sqoop/lib
鏈接:https://www.jianshu.com/p/2e88edf689c9
來源:簡書
著作權歸作者所有。商業轉載請聯系作者獲得授權,非商業轉載請注明出處。