Azkaban是什么?(一)
Hadoop工作流引擎之Azkaban與Oozie對比(四)
不多說,直接上干貨!
http://www.cnblogs.com/zlslch/category/938837.html
准備這四個
Azkaban 2.5安裝
-
系統時區設置
-
安裝和配置mysql
-
Azkaban Web Server 安裝
-
Azkaban Executor Server安裝
-
啟動驗證
-
Azkaban web server插件安裝
-
Azkaban Executor Server 插件安裝
一、系統時區配置
1.查看時區
[hadoop@master azkaban-web-2.5.0]$ more /etc/sysconfig/clock ZONE="Asia/Shanghai" UTC=true ARC=false
[hadoop@master ~]$ more /etc/sysconfig/clock # The time zone of the system is defined by the contents of /etc/localtime. # This file is only for evaluation by system-config-date, do not rely on its # contents elsewhere. ZONE="Asia/Shanghai" [hadoop@master ~]$ su root Password: [root@master hadoop]# vim /etc/sysconfig/clock [root@master hadoop]# more /etc/sysconfig/clock # The time zone of the system is defined by the contents of /etc/localtime. # This file is only for evaluation by system-config-date, do not rely on its # contents elsewhere. ZONE="Asia/Shanghai" UTC=true ARC=false [root@master hadoop]#
2.修改時區
時區文件都在/usr/share/zoneinfo,復制相應時區覆蓋/etc/localtime即可
[hadoop@master ~]$ ll /usr/share/zoneinfo
cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
修改web server 和excutor server 都需要配置時區,必須和系統時區一致(注意,這一步可以到最后才來修改)
default.timezone.id=Asia/Shanghai
安裝mysql
這個很簡單。不多贅述,大家可以去看我的博客
Hive環境的安裝部署(完美安裝)(集群內或集群外都適用)(含卸載自帶mysql安裝指定版本)
[hadoop@master app]$ pwd /home/hadoop/app [hadoop@master app]$ ll total 48 drwxrwxr-x 7 hadoop hadoop 4096 Apr 21 15:43 elasticsearch-2.4.0 drwxrwxr-x 6 hadoop hadoop 4096 Apr 21 12:12 elasticsearch-2.4.3 lrwxrwxrwx 1 hadoop hadoop 20 Apr 21 15:00 es -> elasticsearch-2.4.0/ lrwxrwxrwx 1 hadoop hadoop 11 Apr 20 12:19 flume -> flume-1.6.0 drwxrwxr-x 7 hadoop hadoop 4096 Apr 20 12:17 flume-1.6.0 drwxrwxr-x 7 hadoop hadoop 4096 Apr 20 12:00 flume-1.7.0 lrwxrwxrwx. 1 hadoop hadoop 12 Apr 12 11:27 hadoop -> hadoop-2.6.0 drwxr-xr-x. 10 hadoop hadoop 4096 Apr 12 16:33 hadoop-2.6.0 lrwxrwxrwx. 1 hadoop hadoop 13 Apr 12 11:28 hbase -> hbase-0.98.19 drwxrwxr-x. 8 hadoop hadoop 4096 Apr 12 17:27 hbase-0.98.19 lrwxrwxrwx. 1 hadoop hadoop 10 Apr 12 11:28 hive -> hive-1.0.0 drwxrwxr-x. 8 hadoop hadoop 4096 Apr 20 11:59 hive-1.0.0 lrwxrwxrwx. 1 hadoop hadoop 11 Apr 12 10:18 jdk -> jdk1.7.0_79 drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79 drwxr-xr-x. 8 hadoop hadoop 4096 Aug 5 2015 jdk1.8.0_60 lrwxrwxrwx 1 hadoop hadoop 26 Apr 21 22:18 kibana -> kibana-4.6.3-linux-x86_64/ drwxrwxr-x 11 hadoop hadoop 4096 Nov 4 23:24 kibana-4.6.3-linux-x86_64 lrwxrwxrwx. 1 hadoop hadoop 11 Apr 12 11:28 sqoop -> sqoop-1.4.6 drwxr-xr-x. 9 hadoop hadoop 4096 Apr 27 2015 sqoop-1.4.6 lrwxrwxrwx. 1 hadoop hadoop 15 Apr 12 11:28 zookeeper -> zookeeper-3.4.6 drwxr-xr-x. 10 hadoop hadoop 4096 Apr 12 17:13 zookeeper-3.4.6 [hadoop@master app]$ mkdir azkaban [hadoop@master app]$ ll total 52 drwxrwxr-x 2 hadoop hadoop 4096 May 1 11:57 azkaban drwxrwxr-x 7 hadoop hadoop 4096 Apr 21 15:43 elasticsearch-2.4.0 drwxrwxr-x 6 hadoop hadoop 4096 Apr 21 12:12 elasticsearch-2.4.3 lrwxrwxrwx 1 hadoop hadoop 20 Apr 21 15:00 es -> elasticsearch-2.4.0/ lrwxrwxrwx 1 hadoop hadoop 11 Apr 20 12:19 flume -> flume-1.6.0 drwxrwxr-x 7 hadoop hadoop 4096 Apr 20 12:17 flume-1.6.0 drwxrwxr-x 7 hadoop hadoop 4096 Apr 20 12:00 flume-1.7.0 lrwxrwxrwx. 1 hadoop hadoop 12 Apr 12 11:27 hadoop -> hadoop-2.6.0 drwxr-xr-x. 10 hadoop hadoop 4096 Apr 12 16:33 hadoop-2.6.0 lrwxrwxrwx. 1 hadoop hadoop 13 Apr 12 11:28 hbase -> hbase-0.98.19 drwxrwxr-x. 8 hadoop hadoop 4096 Apr 12 17:27 hbase-0.98.19 lrwxrwxrwx. 1 hadoop hadoop 10 Apr 12 11:28 hive -> hive-1.0.0 drwxrwxr-x. 8 hadoop hadoop 4096 Apr 20 11:59 hive-1.0.0 lrwxrwxrwx. 1 hadoop hadoop 11 Apr 12 10:18 jdk -> jdk1.7.0_79 drwxr-xr-x. 8 hadoop hadoop 4096 Apr 11 2015 jdk1.7.0_79 drwxr-xr-x. 8 hadoop hadoop 4096 Aug 5 2015 jdk1.8.0_60 lrwxrwxrwx 1 hadoop hadoop 26 Apr 21 22:18 kibana -> kibana-4.6.3-linux-x86_64/ drwxrwxr-x 11 hadoop hadoop 4096 Nov 4 23:24 kibana-4.6.3-linux-x86_64 lrwxrwxrwx. 1 hadoop hadoop 11 Apr 12 11:28 sqoop -> sqoop-1.4.6 drwxr-xr-x. 9 hadoop hadoop 4096 Apr 27 2015 sqoop-1.4.6 lrwxrwxrwx. 1 hadoop hadoop 15 Apr 12 11:28 zookeeper -> zookeeper-3.4.6 drwxr-xr-x. 10 hadoop hadoop 4096 Apr 12 17:13 zookeeper-3.4.6 [hadoop@master app]$
二、配置mysql
下載mysql數據腳本
wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-sql-script-2.5.0.tar.gz
tar -zxvf azkaban-sql-script-2.5.0.tar.gz –C ~/app/azkaban/
我下載的是離線,待會上傳。
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 0 [hadoop@master azkaban]$ rz [hadoop@master azkaban]$ ll total 4 -rw-r--r-- 1 hadoop hadoop 1928 May 1 11:43 azkaban-sql-script-2.5.0.tar.gz [hadoop@master azkaban]$
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 4 -rw-r--r-- 1 hadoop hadoop 1928 May 1 11:43 azkaban-sql-script-2.5.0.tar.gz [hadoop@master azkaban]$ tar -zxvf azkaban-sql-script-2.5.0.tar.gz
[hadoop@master azkaban]$ ll total 8 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 -rw-r--r-- 1 hadoop hadoop 1928 May 1 11:43 azkaban-sql-script-2.5.0.tar.gz [hadoop@master azkaban]$ rm azkaban-sql-script-2.5.0.tar.gz [hadoop@master azkaban]$ ll total 4 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 [hadoop@master azkaban]$
創建用戶、分配權限並執行腳本
mysql –uroot –pxxxx 創建數據庫: CREATE DATABASE azkaban; 創建用戶名:CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban' 分配權限:GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON `azkaban`.* TO 'azkaban'@'%' WITH GRANT OPTION; flush privileges; quit; 通過azkaban賬號登錄mysql:mysql –uazkaban –pazkaban Use azkaban; source ~/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
先開啟mysql服務進程
[root@master azkaban]# service mysqld start
Starting mysqld: [ OK ]
[root@master azkaban]# su hadoop
[hadoop@master azkaban]$
[hadoop@master azkaban]$ mysql -uroot -prootroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 2 Server version: 5.1.73 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> CREATE DATABASE azkaban; Query OK, 1 row affected (0.00 sec) mysql> CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban'; Query OK, 0 rows affected (0.00 sec) mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON 'azkaban'.* TO 'azkaban'@'%' WITH GRANT OPTION; ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''azkaban'.* TO 'azkaban'@'%' WITH GRANT OPTION' at line 1 mysql> GRANT SELECT,INSERT,UPDATE,DELETE,CREATE,INDEX ON `azkaban`.* TO 'azkaban'@'%' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> quit; Bye [hadoop@master azkaban]$
通過azkaban賬號登錄mysql
mysql –uazkaban –pazkaban use azkaban; source ~/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
則,再次執行
[hadoop@master azkaban]$ mysql -uazkaban -pazkaban ERROR 1045 (28000): Access denied for user 'azkaban'@'localhost' (using password: YES) [hadoop@master azkaban]$ mysql -uroot -prootroot Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.1.73 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'%' IDENTIFIED BY 'azkaban' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'master' IDENTIFIED BY 'azkaban' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL PRIVILEGES ON *.* to 'azkaban'@'localhost' IDENTIFIED BY 'azkaban' WITH GRANT OPTION; Query OK, 0 rows affected (0.00 sec) mysql> flush privileges; Query OK, 0 rows affected (0.00 sec) mysql> select user,host,password from mysql.user; +---------+-----------+-------------------------------------------+ | user | host | password | +---------+-----------+-------------------------------------------+ | root | localhost | *6C362347EBEAA7DF44F6D34884615A35095E80EB | | root | master | | | root | 127.0.0.1 | | | | localhost | | | | master | | | hive | % | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC | | hive | master | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC | | hive | localhost | *4DF1D66463C18D44E3B001A8FB1BBFBEA13E27FC | | azkaban | % | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 | | azkaban | master | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 | | azkaban | localhost | *8E02B7AE57F9A19E165EB45CD3F705BF66985B85 | +---------+-----------+-------------------------------------------+ 11 rows in set (0.01 sec) mysql> exit; Bye [hadoop@master azkaban]$
然后,此刻就可以通過azkaban賬號登錄mysql了。
mysql –uazkaban –pazkaban use azkaban; source ~/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql
因為,
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 4 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 [hadoop@master azkaban]$ cd azkaban-2.5.0/ [hadoop@master azkaban-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-2.5.0 [hadoop@master azkaban-2.5.0]$ ll total 88 -rwxr-xr-x 1 hadoop hadoop 129 Apr 22 2014 create.active_executing_flows.sql -rwxr-xr-x 1 hadoop hadoop 216 Apr 22 2014 create.active_sla.sql -rwxr-xr-x 1 hadoop hadoop 4694 Apr 22 2014 create-all-sql-2.5.0.sql -rwxr-xr-x 1 hadoop hadoop 610 Apr 22 2014 create.execution_flows.sql -rwxr-xr-x 1 hadoop hadoop 519 Apr 22 2014 create.execution_jobs.sql -rwxr-xr-x 1 hadoop hadoop 358 Apr 22 2014 create.execution_logs.sql -rwxr-xr-x 1 hadoop hadoop 224 Apr 22 2014 create.project_events.sql -rwxr-xr-x 1 hadoop hadoop 227 Apr 22 2014 create.project_files.sql -rwxr-xr-x 1 hadoop hadoop 280 Apr 22 2014 create.project_flows.sql -rwxr-xr-x 1 hadoop hadoop 285 Apr 22 2014 create.project_permissions.sql -rwxr-xr-x 1 hadoop hadoop 294 Apr 22 2014 create.project_properties.sql -rwxr-xr-x 1 hadoop hadoop 380 Apr 22 2014 create.projects.sql -rwxr-xr-x 1 hadoop hadoop 325 Apr 22 2014 create.project_versions.sql -rwxr-xr-x 1 hadoop hadoop 155 Apr 22 2014 create.properties.sql -rwxr-xr-x 1 hadoop hadoop 498 Apr 22 2014 create.schedules.sql -rwxr-xr-x 1 hadoop hadoop 189 Apr 22 2014 create.triggers.sql -rwxr-xr-x 1 hadoop hadoop 22 Apr 22 2014 database.properties -rwxr-xr-x 1 hadoop hadoop 671 Apr 22 2014 update-all-sql-2.1.sql -rwxr-xr-x 1 hadoop hadoop 156 Apr 22 2014 update-all-sql-2.2.sql -rwxr-xr-x 1 hadoop hadoop 395 Apr 22 2014 update.execution_logs.2.1.sql -rwxr-xr-x 1 hadoop hadoop 59 Apr 22 2014 update.project_properties.2.1.sql [hadoop@master azkaban-2.5.0]$
所以,才執行如下
[hadoop@master azkaban]$ mysql -uazkaban -pazkaban Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 5 Server version: 5.1.73 Source distribution Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use azkaban; Database changed mysql> source /home/hadoop/app/azkaban/azkaban-2.5.0/create-all-sql-2.5.0.sql Query OK, 0 rows affected (0.03 sec) Query OK, 0 rows affected (0.03 sec) Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.04 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.03 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.00 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.01 sec) Query OK, 0 rows affected (0.01 sec) Records: 0 Duplicates: 0 Warnings: 0 Query OK, 0 rows affected (0.02 sec)
mysql> exit;
Bye
[hadoop@master azkaban]$
三、Azkaban Web Server 安裝
1.下載安裝包
wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-web-server-2.5.0.tar.gz tar –zxvf azkaban-web-server-2.5.0.tar.gz –C ~/app/azkaban/
我這里,選擇離線上傳的方式,之前已經下載好了。
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 4 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 [hadoop@master azkaban]$ rz [hadoop@master azkaban]$ ll total 11716 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 -rw-r--r-- 1 hadoop hadoop 11989669 May 1 11:44 azkaban-web-server-2.5.0.tar.gz [hadoop@master azkaban]$
解壓縮
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 11716 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 -rw-r--r-- 1 hadoop hadoop 11989669 May 1 11:44 azkaban-web-server-2.5.0.tar.gz [hadoop@master azkaban]$ tar -zxvf azkaban-web-server-2.5.0.tar.gz
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 11720 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:29 azkaban-web-2.5.0 -rw-r--r-- 1 hadoop hadoop 11989669 May 1 11:44 azkaban-web-server-2.5.0.tar.gz [hadoop@master azkaban]$ rm azkaban-web-server-2.5.0.tar.gz [hadoop@master azkaban]$ ll total 8 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:29 azkaban-web-2.5.0 [hadoop@master azkaban]$
2.上傳依賴包
上傳mysql驅動包至 ~/app/azkaban/azkaban-web-2.5.0/extlib/
[hadoop@master extlib]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib [hadoop@master extlib]$ ll total 0 [hadoop@master extlib]$ rz [hadoop@master extlib]$ ll total 812 -rw-r--r-- 1 hadoop hadoop 827942 Nov 3 09:37 mysql-connector-java-5.1.21.jar [hadoop@master extlib]$
3.生成jetty ssl所需密鑰文件
keytool -keystore keystore -alias jetty -genkey -keyalg RSA Enter keystore password: mysteel What is your first and last name? [Unknown]: mysteel What is the name of your organizational unit? [Unknown]: mysteel What is the name of your organization? [Unknown]: mysteel What is the name of your City or Locality? [Unknown]:SH What is the name of your State or Province? [Unknown]:SH What is the two-letter country code for this unit? [Unknown]:CN Is CN=mysteel, OU= mysteel, O= mysteel, L=SH, ST=SH, C=CN correct? [no]: yes Enter key password for < mysteel > (RETURN if same as keystore password): mysteel
[hadoop@master extlib]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib [hadoop@master extlib]$ keytool -keystore keystore -alias jetty -genkey -keyalg RSA Enter keystore password:(密碼是mysteel) Re-enter new password: (密碼是mysteel) What is your first and last name? [Unknown]: mysteel What is the name of your organizational unit? [Unknown]: mysteel What is the name of your organization? [Unknown]: mysteel What is the name of your City or Locality? [Unknown]: SH What is the name of your State or Province? [Unknown]: SH What is the two-letter country code for this unit? [Unknown]: CN Is CN=mysteel, OU=mysteel, O=mysteel, L=SH, ST=SH, C=CN correct? [no]: yes Enter key password for <jetty> (RETURN if same as keystore password): Re-enter new password: (密碼是mysteel) [hadoop@master extlib]$
注意:這里keystore的密碼為mysteel,key的密碼為mysteel可自行修改 最后將生成的keystore文件放到 Azkaban安裝目錄中~/app/azkaban/azkaban-web-2.5.0/ (也可通過修改配置文件改變其默認存放地址)。
[hadoop@master extlib]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0/extlib [hadoop@master extlib]$ ll total 816 -rw-rw-r-- 1 hadoop hadoop 2211 May 1 14:39 keystore -rw-r--r-- 1 hadoop hadoop 827942 Nov 3 09:37 mysql-connector-java-5.1.21.jar [hadoop@master extlib]$ mv keystore /home/hadoop/app/azkaban/azkaban-web-2.5.0/ [hadoop@master extlib]$ cd /home/hadoop/app/azkaban/azkaban-web-2.5.0/ [hadoop@master azkaban-web-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0 [hadoop@master azkaban-web-2.5.0]$ ll total 32 -rw-r--r-- 1 hadoop hadoop 105 Apr 22 2014 azkaban.version drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 bin drwxr-xr-x 2 hadoop hadoop 4096 May 1 14:29 conf drwxr-xr-x 2 hadoop hadoop 4096 May 1 14:43 extlib -rw-rw-r-- 1 hadoop hadoop 2211 May 1 14:39 keystore drwxr-xr-x 2 hadoop hadoop 4096 May 1 14:29 lib drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 plugins drwxr-xr-x 6 hadoop hadoop 4096 May 1 14:29 web [hadoop@master azkaban-web-2.5.0]$
4.修改配置文件 (conf/azkaban.propreties)
修改首頁樣式及時區信息
azkaban.name=Job Tasks azkaban.label=mysteel workflow azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=Asia/Shanghai
#Azkaban Personalization Settings azkaban.name=Test azkaban.label=My Local Azkaban azkaban.color=#FF3601 azkaban.default.servlet.path=/index web.resource.dir=web/ default.timezone.id=America/Los_Angeles 改為
#Azkaban Personalization Settings
azkaban.name=Job Tasks
azkaban.label=mysteel workflow
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
修改mysql連接,可自行修改
database.type=mysql mysql.port=3306 mysql.host=192.168.80.145 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100
database.type=mysql mysql.port=3306 mysql.host=localhost mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 改為 database.type=mysql mysql.port=3306 mysql.host=192.168.80.145 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100
修改 Jetty 服務器屬性,包括 keystore 的相關配置
jetty.maxThreads=25 jetty.ssl.port=8443 jetty.port=8081 jetty.keystore=keystore jetty.password=mysteel jetty.keypassword= mysteel jetty.truststore=keystore jetty.trustpassword= mysteel
# Azkaban Jetty server properties. jetty.maxThreads=25 jetty.ssl.port=8443 jetty.port=8081 jetty.keystore=keystore jetty.password=password jetty.keypassword=password jetty.truststore=keystore jetty.trustpassword=password 改成 # Azkaban Jetty server properties. jetty.maxThreads=25 jetty.ssl.port=8443 jetty.port=8071 jetty.keystore=keystore jetty.password=mysteel jetty.keypassword=mysteel jetty.truststore=keystore jetty.trustpassword=mysteel
修改郵件設置(可選)
mail.sender=xxxxxxx@mysteel.com mail.host=smtp.126.com mail.user= xxxxxxx@mysteel.com mail.password=xxx (注:發送郵箱的密碼)
5.用戶及權限配置
進入 azkaban web 服務器 conf 目錄,修改 azkaban-users.xml ,增加管理員用戶:
< azkaban-users> < user username="azkaban" password="azkaban" roles="admin" groups="azkaban" /> < user username="metrics" password="metrics" roles="metrics"/> < user username="admin" password="admin" roles="admin,metrics" /> < role name="admin" permissions="ADMIN" /> < role name="metrics" permissions="METRICS"/> < /azkaban-users>
<azkaban-users> <user username="azkaban" password="azkaban" roles="admin" groups="azkaban" /> <user username="metrics" password="metrics" roles="metrics"/> <role name="admin" permissions="ADMIN" /> <role name="metrics" permissions="METRICS"/> </azkaban-users> 改為 <azkaban-users> <user username="azkaban" password="azkaban" roles="admin" groups="azkaban" /> <user username="metrics" password="metrics" roles="metrics"/> <user username="admin" password="admin" roles="admin,metrics"/> <role name="admin" permissions="ADMIN" /> <role name="metrics" permissions="METRICS"/> </azkaban-users>
四、Azkaban Executor Server安裝
1、下載安裝包
wget https://s3.amazonaws.com/azkaban2/azkaban2/2.5.0/azkaban-executor-server-2.5.0.tar.gz tar –zxvf azkaban-executor-server-2.5.0.tar.gz –C ~/app/azkaban/
我這里,選擇的是離線,再上傳的方式
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 8 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:43 azkaban-web-2.5.0 [hadoop@master azkaban]$ rz [hadoop@master azkaban]$ ll total 10904 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 -rw-r--r-- 1 hadoop hadoop 11157302 May 1 11:44 azkaban-executor-server-2.5.0.tar.gz drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:43 azkaban-web-2.5.0 [hadoop@master azkaban]$ clear [hadoop@master azkaban]$
解壓縮
[hadoop@master azkaban]$ pwd /home/hadoop/app/azkaban [hadoop@master azkaban]$ ll total 10904 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 -rw-r--r-- 1 hadoop hadoop 11157302 May 1 11:44 azkaban-executor-server-2.5.0.tar.gz drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:43 azkaban-web-2.5.0 [hadoop@master azkaban]$ tar -zxvf azkaban-executor-server-2.5.0.tar.gz
[hadoop@master azkaban]$ ll total 10908 drwxrwxr-x 2 hadoop hadoop 4096 May 1 12:04 azkaban-2.5.0 drwxrwxr-x 7 hadoop hadoop 4096 May 1 15:21 azkaban-executor-2.5.0 -rw-r--r-- 1 hadoop hadoop 11157302 May 1 11:44 azkaban-executor-server-2.5.0.tar.gz drwxrwxr-x 8 hadoop hadoop 4096 May 1 14:43 azkaban-web-2.5.0 [hadoop@master azkaban]$ rm azkaban-executor-server-2.5.0.tar.gz [hadoop@master azkaban]$
2.修改配置文件 (conf/azkaban.propreties)
修改時區信息
default.timezone.id=Asia/Shanghai
[hadoop@master conf]$ pwd /home/hadoop/app/azkaban/azkaban-executor-2.5.0/conf [hadoop@master conf]$ ll total 8 -rw-r--r-- 1 hadoop hadoop 55 Apr 22 2014 azkaban.private.properties -rw-r--r-- 1 hadoop hadoop 469 Apr 22 2014 azkaban.properties -rw-r--r-- 1 hadoop hadoop 0 Apr 22 2014 global.properties [hadoop@master conf]$ vim azkaban.properties
#Azkaban default.timezone.id=America/Los_Angeles 改為 #Azkaban default.timezone.id=Asia/Shanghai
修改mysql連接,可自行修改
database.type=mysql mysql.port=3306 mysql.host=192.168.80.145 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100
database.type=mysql mysql.port=3306 mysql.host=localhost mysql.database=azkaban2 mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100 改為 database.type=mysql mysql.port=3306 mysql.host=192.168.80.145 mysql.database=azkaban mysql.user=azkaban mysql.password=azkaban mysql.numconnections=100
配置端口,默認就好
executor.maxThreads=50 executor.port=12321 executor.flow.threads=30
在web server 的azkaban.properties配置executor的連接,特別注意,如果兩個server不是在一台主機上,web server 需要配置executor.host
# Azkaban Executor settings executor.port=12321 executor.host=ip/hostname
我這里,僅僅是都安裝在單節點master即同一台機器上,所以上述這一小步驟省略。
五、啟動驗證
啟動web server
~/app/azkaban/azkaban-web-2.5.0/bin/azkaban-web-start.sh(azkaban-web-shutdown.sh停止) 訪問 https://master:8443 出現界面安裝成功
或
訪問 https://192.168.80.145:8443 出現界面安裝成功
[hadoop@master azkaban-web-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0 [hadoop@master azkaban-web-2.5.0]$ ll total 32 -rw-r--r-- 1 hadoop hadoop 105 Apr 22 2014 azkaban.version drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 bin drwxr-xr-x 2 hadoop hadoop 4096 May 1 15:27 conf drwxr-xr-x 2 hadoop hadoop 4096 May 1 14:43 extlib -rw-rw-r-- 1 hadoop hadoop 2211 May 1 14:39 keystore drwxr-xr-x 2 hadoop hadoop 4096 May 1 14:29 lib drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 plugins drwxr-xr-x 6 hadoop hadoop 4096 May 1 14:29 web [hadoop@master azkaban-web-2.5.0]$ bin/azkaban-web-start.sh Using Hadoop from /home/hadoop/app/hadoop-2.6.0 Using Hive from /home/hadoop/app/hive bin/.. /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:bin/../lib/azkaban-2.5.0.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-configuration-1.8.jar:bin/../lib/commons-dbcp-1.4.jar:bin/../lib/commons-dbutils-1.5.jar:bin/../lib/commons-email-1.2.jar:bin/../lib/commons-fileupload-1.2.1.jar:bin/../lib/commons-io-2.4.jar:bin/../lib/commons-jexl-2.1.1.jar:bin/../lib/commons-lang-2.6.jar:bin/../lib/commons-logging-1.1.1.jar:bin/../lib/commons-pool-1.6.jar:bin/../lib/guava-13.0.1.jar:bin/../lib/h2-1.3.170.jar:bin/../lib/httpclient-4.2.1.jar:bin/../lib/httpcore-4.2.1.jar:bin/../lib/jackson-core-asl-1.9.5.jar:bin/../lib/jackson-mapper-asl-1.9.5.jar:bin/../lib/jetty-6.1.26.jar:bin/../lib/jetty-util-6.1.26.jar:bin/../lib/joda-time-2.0.jar:bin/../lib/jopt-simple-4.3.jar:bin/../lib/junit-4.8.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mail-1.4.5.jar:bin/../lib/mysql-connector-java-5.1.28.jar:bin/../lib/servlet-api-2.5.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.4.jar:bin/../lib/velocity-1.7.jar:bin/../lib/velocity-tools-2.0.jar:bin/../extlib/mysql-connector-java-5.1.21.jar:bin/../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/* [hadoop@master azkaban-web-2.5.0]$ 2017/05/01 15:35:30.167 +0800 ERROR [AzkabanWebServer] [Azkaban] Starting Jetty Azkaban Executor... 2017/05/01 15:35:30.278 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf 2017/05/01 15:35:30.280 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file 2017/05/01 15:35:30.338 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up Jetty Https Server with port:8443 and numThreads:25 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-web-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2017/05/01 15:35:30.532 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2017/05/01 15:35:30.897 +0800 INFO [AzkabanWebServer] [Azkaban] Loading user manager class azkaban.user.XmlUserManager
2017/05/01 15:35:31.352 +0800 INFO [XmlUserManager] [Azkaban] Loading user azkaban
2017/05/01 15:35:31.359 +0800 INFO [XmlUserManager] [Azkaban] Loading user metrics
2017/05/01 15:35:31.360 +0800 INFO [XmlUserManager] [Azkaban] Loading user admin
2017/05/01 15:35:33.429 +0800 INFO [AzkabanWebServer] [Azkaban] Loading JDBC for project management
2017/05/01 15:35:33.450 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old logs from execution_logs
2017/05/01 15:35:33.480 +0800 INFO [ProjectManager] [Azkaban] Project version retention is set to 3
2017/05/01 15:35:34.233 +0800 INFO [ExecutorManager] [Azkaban] Cleaning old log files before 2017-02-06T15:35:33.601+08:00
2017/05/01 15:35:34.283 +0800 INFO [ExecutorManager] [Azkaban] Cleaned up 0 log entries.
2017/05/01 15:35:34.483 +0800 INFO [TriggerManager] [Azkaban] TriggerManager loaded.
2017/05/01 15:35:34.484 +0800 INFO [AzkabanWebServer] [Azkaban] Loading built-in checker and action types
2017/05/01 15:35:34.533 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker BasicTimeChecker
2017/05/01 15:35:34.533 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker SlaChecker
2017/05/01 15:35:34.534 +0800 INFO [CheckerTypeLoader] [Azkaban] Registering checker ExecutionChecker
2017/05/01 15:35:34.534 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action ExecuteFlowAction
2017/05/01 15:35:34.535 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action KillExecutionAction
2017/05/01 15:35:34.550 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action AlertAction
2017/05/01 15:35:34.566 +0800 INFO [ActionTypeLoader] [Azkaban] Registering action CreateTriggerAction
2017/05/01 15:35:34.582 +0800 INFO [AzkabanWebServer] [Azkaban] Loading trigger based scheduler
2017/05/01 15:35:34.620 +0800 INFO [AzkabanWebServer] [Azkaban] Loading plug-in checker and action types
2017/05/01 15:35:34.621 +0800 ERROR [AzkabanWebServer] [Azkaban] plugin path plugins/triggers doesn't exist!
2017/05/01 15:35:34.623 +0800 INFO [AzkabanWebServer] [Azkaban] Setting timezone to Asia/Shanghai
2017/05/01 15:35:34.627 +0800 INFO [AzkabanWebServer] [Azkaban] Registering MBeans...
2017/05/01 15:35:34.747 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered.
2017/05/01 15:35:34.755 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxTriggerManager registered.
2017/05/01 15:35:34.765 +0800 INFO [AzkabanWebServer] [Azkaban] Bean azkaban.jmx.JmxExecutorManager registered.
2017/05/01 15:35:34.770 +0800 INFO [AzkabanWebServer] [Azkaban] Setting up web resource dir web/
2017/05/01 15:35:34.952 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loading all triggers from db.
2017/05/01 15:35:35.047 +0800 INFO [JdbcTriggerLoader] [Azkaban] Loaded 0 triggers.
2017/05/01 15:35:35.048 +0800 INFO [log] [Azkaban] jetty-6.1.26
2017/05/01 15:35:36.445 +0800 INFO [log] [Azkaban] Started SslSocketConnector@0.0.0.0:8443
2017/05/01 15:35:36.446 +0800 INFO [AzkabanWebServer] [Azkaban] Server running on ssl port 8443.
這里的用戶名好密碼都是我們在配置文件里,設置的azkaban
啟動 executor server
~/app/azkaban/azkaban-executor-2.5.0/bin/azkaban-executor-start.sh(azkaban-executor-shutdown.sh 停止
[hadoop@master azkaban-executor-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-executor-2.5.0 [hadoop@master azkaban-executor-2.5.0]$ ll total 24 -rw-r--r-- 1 hadoop hadoop 105 Apr 22 2014 azkaban.version drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 bin drwxr-xr-x 2 hadoop hadoop 4096 May 1 15:34 conf drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 extlib drwxr-xr-x 2 hadoop hadoop 4096 May 1 15:21 lib drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 plugins [hadoop@master azkaban-executor-2.5.0]$ bin/azkaban-executor-start.sh Using Hadoop from /home/hadoop/app/hadoop-2.6.0 Using Hive from /home/hadoop/app/hive bin/.. /home/hadoop/app/jdk/lib:.:/home/hadoop/app/jdk/lib:/home/hadoop/app/jdk/jre/lib:bin/../lib/azkaban-2.5.0.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-configuration-1.8.jar:bin/../lib/commons-dbcp-1.4.jar:bin/../lib/commons-dbutils-1.5.jar:bin/../lib/commons-email-1.2.jar:bin/../lib/commons-fileupload-1.2.1.jar:bin/../lib/commons-io-2.4.jar:bin/../lib/commons-jexl-2.1.1.jar:bin/../lib/commons-lang-2.6.jar:bin/../lib/commons-logging-1.1.1.jar:bin/../lib/commons-pool-1.6.jar:bin/../lib/guava-13.0.1.jar:bin/../lib/h2-1.3.170.jar:bin/../lib/httpclient-4.2.1.jar:bin/../lib/httpcore-4.2.1.jar:bin/../lib/jackson-core-asl-1.9.5.jar:bin/../lib/jackson-mapper-asl-1.9.5.jar:bin/../lib/jetty-6.1.26.jar:bin/../lib/jetty-util-6.1.26.jar:bin/../lib/joda-time-2.0.jar:bin/../lib/jopt-simple-4.3.jar:bin/../lib/junit-4.8.jar:bin/../lib/log4j-1.2.16.jar:bin/../lib/mail-1.4.5.jar:bin/../lib/mysql-connector-java-5.1.28.jar:bin/../lib/servlet-api-2.5.jar:bin/../lib/slf4j-api-1.6.1.jar:bin/../lib/slf4j-log4j12-1.6.4.jar:bin/../lib/velocity-1.7.jar:bin/../lib/velocity-tools-2.0.jar:bin/../extlib/*.jar:bin/../plugins/*/*.jar:/home/hadoop/app/hadoop-2.6.0/conf:/home/hadoop/app/hadoop-2.6.0/*:/home/hadoop/app/hive/conf:/home/hadoop/app/hive/lib/* Starting AzkabanExecutorServer on port 12321 ... [hadoop@master azkaban-executor-2.5.0]$ 2017/05/01 15:43:52.101 +0800 ERROR [AzkabanExecutorServer] [Azkaban] Starting Jetty Azkaban Executor... 2017/05/01 15:43:52.180 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban settings file from bin/../conf 2017/05/01 15:43:52.182 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban private properties file 2017/05/01 15:43:52.187 +0800 INFO [AzkabanServer] [Azkaban] Loading azkaban properties file 2017/05/01 15:43:52.285 +0800 INFO [AzkabanExecutorServer] [Azkaban] Setting timezone to Asia/Shanghai SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/hadoop/app/azkaban/azkaban-executor-2.5.0/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-1.0.0/lib/hive-jdbc-1.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 2017/05/01 15:43:52.430 +0800 INFO [log] [Azkaban] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2017/05/01 15:43:53.334 +0800 INFO [FlowRunnerManager] [Azkaban] Execution dir retention set to 86400000 ms 2017/05/01 15:43:53.388 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning recently finished 2017/05/01 15:43:53.389 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old projects 2017/05/01 15:43:53.390 +0800 INFO [FlowRunnerManager] [Azkaban] Cleaning old execution dirs 2017/05/01 15:43:53.431 +0800 INFO [AzkabanExecutorServer] [Azkaban] Registering MBeans... 2017/05/01 15:43:53.610 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxJettyServer registered. 2017/05/01 15:43:53.639 +0800 INFO [AzkabanExecutorServer] [Azkaban] Bean azkaban.jmx.JmxFlowRunnerManager registered. 2017/05/01 15:43:53.640 +0800 INFO [log] [Azkaban] jetty-6.1.26 2017/05/01 15:43:53.861 +0800 INFO [log] [Azkaban] Started SocketConnector@0.0.0.0:12321 2017/05/01 15:43:53.862 +0800 INFO [AzkabanExecutorServer] [Azkaban] Azkaban Executor Server started on port 12321
如何停止
[hadoop@master azkaban-web-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0 [hadoop@master azkaban-web-2.5.0]$ jps 5676 AzkabanWebServer 5866 AzkabanExecutorServer 6026 Jps
[hadoop@master azkaban-executor-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-executor-2.5.0 [hadoop@master azkaban-executor-2.5.0]$ ll total 44 -rw-rw-r-- 1 hadoop hadoop 0 May 1 15:43 azkaban-access.log -rw-r--r-- 1 hadoop hadoop 105 Apr 22 2014 azkaban.version -rw-rw-r-- 1 hadoop hadoop 302 May 1 15:43 azkaban-webserver.log drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 bin drwxr-xr-x 2 hadoop hadoop 4096 May 1 15:34 conf -rw-rw-r-- 1 hadoop hadoop 5 May 1 15:43 currentpid drwxrwxr-x 2 hadoop hadoop 4096 May 1 15:43 executions drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 extlib drwxr-xr-x 2 hadoop hadoop 4096 May 1 15:21 lib drwxr-xr-x 2 hadoop hadoop 4096 Apr 22 2014 plugins drwxrwxr-x 2 hadoop hadoop 4096 May 1 15:43 projects drwxrwxr-x 2 hadoop hadoop 4096 May 1 15:43 temp [hadoop@master azkaban-executor-2.5.0]$ bin/azkaban-executor-shutdown.sh [hadoop@master azkaban-executor-2.5.0]$ jps 5676 AzkabanWebServer 6111 Jps [hadoop@master azkaban-executor-2.5.0]$
[hadoop@master azkaban-web-2.5.0]$ pwd /home/hadoop/app/azkaban/azkaban-web-2.5.0 [hadoop@master azkaban-web-2.5.0]$ jps 5676 AzkabanWebServer 6087 Jps [hadoop@master azkaban-web-2.5.0]$ bin/azkaban-web-shutdown.sh
或kill掉
[hadoop@master azkaban-web-2.5.0]$ jps 6128 Jps 5676 AzkabanWebServer [hadoop@master azkaban-web-2.5.0]$ kill -9 5676 [hadoop@master azkaban-web-2.5.0]$ jps 6138 Jps [hadoop@master azkaban-web-2.5.0]$
這里不多贅述了。
六、Azkaban web server插件安裝(暫時沒用上,不先寫了)
為了后邊插件的安裝,需要在${AZKABAN_WEB_SERVER}/plugins/目錄下創建viewer目錄,用來安裝各種viewer插件。當安裝好一個插件后需要重啟服務,讓插件生效。
安裝HDFS Viewer插件
1.解壓插件
解壓插件 在viewer目錄中解壓azkaban-hdfs-viewer-2.5.0.tar.gz,得到azkaban-hdfs-viewer-2.5.0目錄,將其更名為hdfs。最終,這個插件的目錄路徑是:${AZKABAN_WEB_SERVER}/plugins/viewer/hdfs。
安裝Job Summary插件
安裝Reportal插件
七、Azkaban Executor Server 插件安裝(暫時沒用上,不先寫了)
進入${AZKABAN_EXECUTOR_SERVER}/plugins目錄,在該目錄下解壓azkaban-jobtype-2.5.0.tar.gz得到目錄:azkaban-jobtype-2.5.0。將該目錄更名為jobtypes。這個目錄用來存放之后安裝的所有插件,與Web Server中的viewer目錄類似。每次插件的安裝配置需要重啟Executor Server。
安裝Reportalhive插件
參考
http://www.cnblogs.com/qiaoyihang/p/6237810.html
同時,大家可以關注我的個人博客:
http://www.cnblogs.com/zlslch/ 和 http://www.cnblogs.com/lchzls/ http://www.cnblogs.com/sunnyDream/
詳情請見:http://www.cnblogs.com/zlslch/p/7473861.html
人生苦短,我願分享。本公眾號將秉持活到老學到老學習無休止的交流分享開源精神,匯聚於互聯網和個人學習工作的精華干貨知識,一切來於互聯網,反饋回互聯網。
目前研究領域:大數據、機器學習、深度學習、人工智能、數據挖掘、數據分析。 語言涉及:Java、Scala、Python、Shell、Linux等 。同時還涉及平常所使用的手機、電腦和互聯網上的使用技巧、問題和實用軟件。 只要你一直關注和呆在群里,每天必須有收獲
對應本平台的討論和答疑QQ群:大數據和人工智能躺過的坑(總群)(161156071)