sqoop安裝部署(筆記)


sqoop是一個把關系型數據庫數據抽向hadoop的工具。同時,也支持將hive、pig等查詢的結果導入關系型數據庫中存儲。
由於,筆者部署的hadoop版本是2.2.0,所以sqoop的版本是:sqoop-1.99.3-bin-hadoop200
1、下載sqoop  wget  http://mirrors.cnnic.cn/apache/sqoop/1.99.3/sqoop-1.99.3-bin-hadoop200.tar.gz
2、解壓sqoop安裝文件  tar -zxvf sqoop-1.99.3-bin-hadoop200.tar.gz
3、配置sqoop環境變量
  export SQOOP_HOME=/home/likehua/sqoop/sqoop
  export PATH = $SQOOP_HOME/bin:$PATH
  修改sqoop——server關於hadoop共享jar的引用
  vi server/conf/catalina.properties
  找到common.loader行,把/usr/lib/hadoop/lib/*.jar改成你的hadoop jar 包目錄
例如:/home/hadoop/hadoop/share/hadoop/yarn/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/yarn/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/*.jar,
/home/hadoop/hadoop/share/hadoop/hdfs/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/*.jar,
/home/hadoop/hadoop/share/hadoop/mapreduce/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/lib/*.jar,
/home/hadoop/hadoop/share/hadoop/common/*.jar
  vi server/conf/sqoop.properties
找到:mapreduce.configuration.directory行,修改值為你的hadoop配置文件目錄
如:/home/likehua/hadoop/hadoop/etc/hadoop
並且替換@LOGDIR@ 和@BASEDIR@ :
0,$ s/@LOGDIR@/logs/g
0,$ s/@BASEDIR@/base/g

然后找到你的數據庫jdbc驅動復制到sqoop/lib目錄下,如果不存在則創建.


 vi /etc/profile

增加以下內容:

export SQOOP_HOME=/home/hadoop/sqoop

export PATH=$PATH:$SQOOP_HOME/bin

export CATALINA_BASE=$SQOOP_HOME/server

export LOGDIR=$SQOOP_HOME/logs/
 

啟動 [root@db12c sqoop]# ./bin/sqoop.sh server start
Sqoop home directory: /home/likehua/sqoop/sqoop
Setting SQOOP_HTTP_PORT:     12000
Setting SQOOP_ADMIN_PORT:     12001
Using   CATALINA_OPTS:
Adding to CATALINA_OPTS:    -Dsqoop.http.port=12000 -Dsqoop.admin.port=12001
Using CATALINA_BASE:   /home/likehua/sqoop/sqoop/server
Using CATALINA_HOME:   /home/likehua/sqoop/sqoop/server
Using CATALINA_TMPDIR: /home/likehua/sqoop/sqoop/server/temp
Using JRE_HOME:        /usr/local/jdk1.7.0
Using CLASSPATH:       /home/likehua/sqoop/sqoop/server/bin/bootstrap.jar
(sqoop服務端是一個跑在tomcat上的服務程序)
關閉 sqoop server :./bin/sqoop.sh server stop
使用sqoop客戶端:
[root@db12c sqoop]# bin/sqoop.sh client
Sqoop home directory: /home/likehua/sqoop/sqoop
Sqoop Shell: Type 'help' or '\h' for help.

sqoop:000> show version --all#顯示版本:show version --all顯示連接器:show connector --all創建連接:create connection --cid 1

client version:
  Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b
  Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
server version:
  Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b
  Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
Protocol version:
  [1]
sqoop:000>


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM