[sqoop1.99.7] sqoop命令


 

官網文檔:http://sqoop.apache.org/docs/1.99.7/user/CommandLineClient.html#delete-link-function

 

一、了解sqoop數據導入的幾個重要概念

(1)connector:sqoop2中預定了各種鏈接,這些鏈接是一些配置模板。

#查看預定制的連接器

sqoop:000> show connector
+------------------------+---------+------------------------------------------------------------+----------------------+
|          Name          | Version |                           Class                            | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              |
| sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   |
| kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   |
| kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              |
| ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   |
| hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              |
| generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |
+------------------------+---------+------------------------------------------------------------+----------------------+


其中,最基本的是generic-jdbc-connector,是mysql等關系型數據庫的連接器。支持數據的從關系型數據庫的導入導出。

除此之外,支持導入導出的還有:hdfs-connector、kite-connector和oracle-jdbc-connector。

僅支持數據導入的連接器有: sftp-connector 、 ftp-connector 以及kafka-connector 。


通過這些模板,可以創建出對應數據源的link,比如我們鏈接mysql,就是使用JDBC的方式進行鏈接,這時候就從這個generic-jdbc-connector模板繼承出一個link。那如果我們需要連接hdfs,則需要使用hdfs-connector模版。


(2)link:從connector繼承出的對象,用於指定的對數據源的鏈接。

(3)job:指定一個導入導出作業,必須指定數據源和數據目的地,並配置各項job參數,用於提交給mapreduce。

 

二、幾個常用的重要命令

#設置
set [server|option|truststore]

#查看
show [server|version|connector|driver|link|job|submission|option|role|principal|privilege]

#創建
create [link|job|role]

#刪除
delete [link|job|role]

#更新
update [link|job]

#克隆
clone [link|job]

#執行任務
start [job]

#結束任務
stop [job]

#查看任務狀態
status [job]

#啟用
enable [link|job]

#禁用
disable [link|job]

#授權
grant [role|privilege]

#回收授權
revoke [role|privilege]

 

set option --name verbose --value true
set option --name poll-timeout --value 20000

 

三、查看

 

查看服務器

sqoop:000> show server -all
Server host: localhost
Server port: 12000
Server webapp: sqoop

 

查看版本

sqoop:000> show version -all
client version:
  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb 
  Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
server version:
  Sqoop 1.99.7 source revision 435d5e61b922a32d7bce567fe5fb1a9c0d9b1bbb 
  Compiled by abefine on Tue Jul 19 16:08:27 PDT 2016
API versions:
  [v1]

 

查看內置連接器

sqoop:000> show connector 
+------------------------+---------+------------------------------------------------------------+----------------------+
|          Name          | Version |                           Class                            | Supported Directions |
+------------------------+---------+------------------------------------------------------------+----------------------+
| oracle-jdbc-connector  | 1.99.7  | org.apache.sqoop.connector.jdbc.oracle.OracleJdbcConnector | FROM/TO              |
| sftp-connector         | 1.99.7  | org.apache.sqoop.connector.sftp.SftpConnector              | TO                   |
| kafka-connector        | 1.99.7  | org.apache.sqoop.connector.kafka.KafkaConnector            | TO                   |
| kite-connector         | 1.99.7  | org.apache.sqoop.connector.kite.KiteConnector              | FROM/TO              |
| ftp-connector          | 1.99.7  | org.apache.sqoop.connector.ftp.FtpConnector                | TO                   |
| hdfs-connector         | 1.99.7  | org.apache.sqoop.connector.hdfs.HdfsConnector              | FROM/TO              |
| generic-jdbc-connector | 1.99.7  | org.apache.sqoop.connector.jdbc.GenericJdbcConnector       | FROM/TO              |
+------------------------+---------+------------------------------------------------------------+----------------------+

 

sqoop:000> show link
0    [main] WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+------+----------------+---------+
| Name | Connector Name | Enabled |
+------+----------------+---------+
+------+----------------+---------+

 

其他查看命令

show server --all

show option --name verbose

show version --all

show connector --all or show connector

show driver

show link --all
show link --name linkName

show job --all 
show job --name jobName

show submission
show submission -j jobName
show submission --job jobName --detail

 

四、創建

創建link

create link --connector connectorName
create link -c connectorName
例如:
create link -connector generic-jdbc-connector
create link -connector hdfs-connector

 

創建job

create job --from fromLinkName --to toLinkName 
create job -f fromLinkName -t toLinkName

 

五、更新

update link --name linkName
update job --name jobName

 

六、刪除

delete link --name linkName
delete job --name jobName

 

七、克隆

clone link --name linkName
clone job --name jobName

 

八、啟動

start job --name jobName
start job --name jobName --synchronous

 

九、停止

stop job --name jobName

 

十、查看任務執行狀態

status job --name jobName

 


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM