Sqoop連接數據庫MySQL報錯


1、問題描述

(1)問題示例:

[Hadoop@master TestDir]$ sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******
2021-11-05 16:18:52,289 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2021-11-05 16:18:52,316 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2021-11-05 16:18:52,379 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/StringUtils
at org.apache.sqoop.manager.MySQLManager.initOptionDefaults(MySQLManager.java:73)
at org.apache.sqoop.manager.SqlManager.<init>(SqlManager.java:89)
at com.cloudera.sqoop.manager.SqlManager.<init>(SqlManager.java:33)
at org.apache.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:51)
at com.cloudera.sqoop.manager.GenericJdbcManager.<init>(GenericJdbcManager.java:30)
at org.apache.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:46)
at com.cloudera.sqoop.manager.CatalogQueryManager.<init>(CatalogQueryManager.java:31)
at org.apache.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:38)
at com.cloudera.sqoop.manager.InformationSchemaManager.<init>(InformationSchemaManager.java:31)
at org.apache.sqoop.manager.MySQLManager.<init>(MySQLManager.java:65)
at org.apache.sqoop.manager.DefaultManagerFactory.accept(DefaultManagerFactory.java:67)
at org.apache.sqoop.ConnFactory.getManager(ConnFactory.java:184)
at org.apache.sqoop.tool.BaseSqoopTool.init(BaseSqoopTool.java:272)
at org.apache.sqoop.tool.ListDatabasesTool.run(ListDatabasesTool.java:44)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.StringUtils
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 20 more

(2)問題綜述:

命令:sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******

上述命令,是用於查看MySQL已有的數據庫。不能成功連接,不能顯示出MySQL中已有數據庫

2、解決方案

參考:https://www.cndba.cn/dave/article/3303

(1)根據參考,下載驅動包:commons-lang-2.6.jar

下載地址:https://mirrors.tuna.tsinghua.edu.cn/apache//commons/lang/binaries/commons-lang-2.6-bin.zip

下載地址可能存在失效,只需要找到commons-lang-2.6.jar

(2)上述下載壓縮包:commons-lang-2.6-bin.zip,須解壓,然后將其中的“commons-lang-2.6.jar拷貝至Sqoop中lib中。

(3)再次測試:

[Hadoop@master TestDir]$ sqoop list-databases --connect jdbc:mysql://master:3306/ --username Hive --password ******
2021-11-05 17:07:59,566 INFO sqoop.Sqoop: Running Sqoop version: 1.4.7
2021-11-05 17:07:59,584 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
2021-11-05 17:07:59,645 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
mysql
information_schema
performance_schema
sys
Test
source
hive

上述測試,成功查看MySQL中數據庫,說明問題已解決。

3、問題剖析

原因是:筆者所安裝的Sqoop版本為sqoop-1.4.7,缺少驅動包:commons-lang-2.6.jar,已存在commons-lang3-3.4.jar不合適。

因此需要另外下載驅動包:commons-lang-2.6.jar。

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM