官網文檔:http://sqoop.apache.org/docs/1.99.7/user/CommandLineClient.html#delete-link-function
一、了解sqoop數據導入的幾個重要概念
(1)connector:sqoop2中預定了各種鏈接,這些鏈接是一些配置模板。 #查看預定制的連接器 sqoop:000> show connector +------------------------+---------+------------------------------------------------------------+----------------------+ | Name | Version | Class | Supported Directions | +------------------------+---------+------------------------------------------------------------+----------------------+ | oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO | | sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO | | kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO | | kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO | | ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO | | hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO | | generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO | +------------------------+---------+------------------------------------------------------------+----------------------+ 其中,最基本的是generic-jdbc-connector,是mysql等關系型數據庫的連接器。支持數據的從關系型數據庫的導入導出。 除此之外,支持導入導出的還有:hdfs-connector、kite-connector和oracle-jdbc-connector。 僅支持數據導入的連接器有: sftp-connector 、 ftp-connector 以及kafka-connector 。 通過這些模板,可以創建出對應數據源的link,比如我們鏈接mysql,就是使用JDBC的方式進行鏈接,這時候就從這個generic-jdbc-connector模板繼承出一個link。那如果我們需要連接hdfs,則需要使用hdfs-connector模版。 (2)link:從connector繼承出的對象,用於指定的對數據源的鏈接。 (3)job:指定一個導入導出作業,必須指定數據源和數據目的地,並配置各項job參數,用於提交給mapreduce。
二、幾個常用的重要命令
#設置 set [server|option|truststore] #查看 show [server|version|connector|driver|link|job|submission|option|role|principal|privilege] #創建 create [link|job|role] #刪除 delete [link|job|role] #更新 update [link|job] #克隆 clone [link|job] #執行任務 start [job] #結束任務 stop [job] #查看任務狀態 status [job] #啟用 enable [link|job] #禁用 disable [link|job] #授權 grant [role|privilege] #回收授權 revoke [role|privilege]
set option --name verbose --value true set option --name poll-timeout --value 20000
三、查看
查看服務器
sqoop:000> show server -all Server host: localhost Server port: 12000 Server webapp: sqoop
查看版本
sqoop:000> show version -all client version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016 server version: Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016 API versions: [v1]
查看內置連接器
sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
| Name | Version | Class | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| oracle-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO |
| sftp-connector | 1.99.7 | org.apache.sqoop.connector.sftp.SftpConnector | TO |
| kafka-connector | 1.99.7 | org.apache.sqoop.connector.kafka.KafkaConnector | TO |
| kite-connector | 1.99.7 | org.apache.sqoop.connector.kite.KiteConnector | FROM/TO |
| ftp-connector | 1.99.7 | org.apache.sqoop.connector.ftp.FtpConnector | TO |
| hdfs-connector | 1.99.7 | org.apache.sqoop.connector.hdfs.HdfsConnector | FROM/TO |
| generic-jdbc-connector | 1.99.7 | org.apache.sqoop.connector.jdbc.GenericJdbcConnector | FROM/TO |
+------------------------+---------+------------------------------------------------------------+----------------------+
sqoop:000> show link 0 [main] WARN org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable +------+----------------+---------+ | Name | Connector Name | Enabled | +------+----------------+---------+ +------+----------------+---------+
其他查看命令
show server --all show option --name verbose show version --all show connector --all or show connector show driver show link --all show link --name linkName show job --all show job --name jobName show submission show submission -j jobName show submission --job jobName --detail
四、創建
創建link
create link --connector connectorName
create link -c connectorName
例如:
create link -connector generic-jdbc-connector
create link -connector hdfs-connector
創建job
create job --from fromLinkName --to toLinkName
create job -f fromLinkName -t toLinkName
五、更新
update link --name linkName
update job --name jobName
六、刪除
delete link --name linkName
delete job --name jobName
七、克隆
clone link --name linkName
clone job --name jobName
八、啟動
start job --name jobName
start job --name jobName --synchronous
九、停止
stop job --name jobName
十、查看任務執行狀態
status job --name jobName