azkaban的安裝部署


一、所需環境

1,JDK

2,HADOOP

4,mysql

3,HIVE

二、安裝azkaban

 

1,安裝git命令: yum install git

2,下載azkaban源碼:git clone https://github.com/azkaban/azkaban.git

3,安裝到 azkaban-tar目錄下

4,更改配置文件:

vi 安裝目錄/azkaban-tar/azkaban/git clone https://github.com/azkaban/azkaban.git

將final boolean isExecuteAsUser = this.sysProps.getBoolean(EXECUTE_AS_USER, true);
true修改false 

5,在azkaban目錄下執行:./gradlew distTar  

此步驟會生成5個壓縮包,分別放在下圖目錄中:

azkaban-db-3.37.0-10-gdd67b02.tar.gz
azkaban-exec-server-3.37.0-10-gdd67b02.tar.gz
azkaban-hadoop-security-plugin-3.37.0-10-gdd67b02.tar.gz
azkaban-solo-server-3.37.0-10-gdd67b02.tar.gz
azkaban-web-server-3.37.0-10-gdd67b02.tar.gz 

 

6,在當前azkaban文件夾下,將該五個壓縮包copy出來,放到 安裝目錄/azkaban-tar文件夾中。

cp  azkaban-*/build/distributions/*.tar.gz  安裝目錄/azkaban-tar

 

7,將這5個壓縮包解壓,然后重命令,將這5個壓縮包放在另外的 安裝目錄/azkaban下

8,登錄mysql mysql –u root –p

9,數據庫中執行:create database azkaban;

10,數據庫中執行:use azkaban;

11, 將azkaban sql表結構導入mysql,數據庫中執行:source 安裝目錄/azkaban/azkaban-db-3.46.0-2-gbb24a72/create-all-sql-3.46.0-2-gbb24a72.sql  

如果執行失敗,則將sql語句復制出來執行

 

12,部署azkaban-web-server

將 安裝目錄/azkaban/azkaban-solo-server/ 下的conf目錄拷貝到  安裝目錄/azkaban/ azkaban-web-server/conf

命令:cp -r  /安裝目錄/azkaban/azkaban-solo-server/conf   /安裝目錄/azkaban/azkaban-web-server/conf 

配置:azkaban.properties

 

如下(標紅的需要更改,其他的可以對比一下):

 

# Azkaban Personalization Settings

azkaban.name=Test

azkaban.label=My Local Azkaban

azkaban.color=#FF3601

azkaban.default.servlet.path=/index

web.resource.dir=/usr/local/azkaban/azkaban-web-server/web

default.timezone.id=Asia/Shanghai

# Azkaban UserManager class

user.manager.class=azkaban.user.XmlUserManager

user.manager.xml.file=/usr/local/azkaban/azkaban-web-server/conf/azkaban-users.xml

# Loader for projects

executor.global.properties=/usr/local/azkaban/azkaban-web-server/conf/global.properties

azkaban.project.dir=projects

database.type=mysql

mysql.port=3306

mysql.host=localhost

mysql.database=azkaban

mysql.user=root

mysql.password=root

mysql.numconnections=100

 

# Velocity dev mode

velocity.dev.mode=false

# Azkaban Jetty server properties.

jetty.use.ssl=false

jetty.maxThreads=25

jetty.port=8081

# Azkaban Executor settings

executor.maxThreads=50

executor.port=12321

executor.flow.threads=30

# mail settings

mail.sender=

mail.host=

# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

# enduser -> myazkabanhost:443 -> proxy -> localhost:8081

# when this parameters set then these parameters are used to generate email links.

# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

# azkaban.webserver.external_hostname=myazkabanhost.com

# azkaban.webserver.external_ssl_port=443

# azkaban.webserver.external_port=8081

job.failure.email=

job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats

jetty.connector.stats=true

executor.connector.stats=true

 

azkaban.native.lib=false

# Azkaban plugin settings

azkaban.jobtype.plugin.dir=plugins/jobtypes

 

13,在conf目錄下新增配置文件log4j.properties,內容如下

 

log4j.rootLogger=INFO,C

log4j.appender.C=org.apache.log4j.ConsoleAppender

log4j.appender.C.Target=System.err

log4j.appender.C.layout=org.apache.log4j.PatternLayout

log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

 

14,部署 azkaban-exec-server

將 /安裝目錄/azkaban/ azkaban-solo-server下的conf文件拷貝到 /安裝文件/azkaban/ azkaban-exec-server

命令:cp –r /安裝目錄/azkaban/azkaban-solo-server/conf  /安裝目錄/azkaban/azkaban-exec-server/conf

 

配置azkaban.properties:(標紅的需要更改,其他的可以對比一下)

 

# Azkaban Personalization Settings

azkaban.name=Test

azkaban.label=My Local Azkaban

azkaban.color=#FF3601

azkaban.default.servlet.path=/index

web.resource.dir=web/

default.timezone.id=Asia/Shanghai

# Azkaban UserManager class

user.manager.class=azkaban.user.XmlUserManager

user.manager.xml.file=conf/azkaban-users.xml

# Loader for projects

executor.global.properties=/usr/local/azkaban/azkaban-exec-server/conf/global.properties

azkaban.project.dir=projects

database.type=mysql

mysql.port=3306

mysql.host=localhost

mysql.database=azkaban

mysql.user=root

mysql.password=root

mysql.numconnections=100

# Velocity dev mode

velocity.dev.mode=false

# Azkaban Jetty server properties.

jetty.use.ssl=false

jetty.maxThreads=25

jetty.port=8081

# Azkaban Executor settings

executor.maxThreads=50

executor.port=12321

executor.flow.threads=30

# mail settings

mail.sender=

mail.host=

# User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users.

# enduser -> myazkabanhost:443 -> proxy -> localhost:8081

# when this parameters set then these parameters are used to generate email links.

# if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used.

# azkaban.webserver.external_hostname=myazkabanhost.com

# azkaban.webserver.external_ssl_port=443

# azkaban.webserver.external_port=8081

job.failure.email=

job.success.email=

lockdown.create.projects=false

cache.directory=cache

# JMX stats

jetty.connector.stats=true

executor.connector.stats=true

# Azkaban plugin settings

azkaban.jobtype.plugin.dir=plugins/jobtypes

 

azkaban.native.lib=false

 

在conf目錄下,新建配置log4j.properties文件,內容如下:

 

log4j.rootLogger=INFO,C

log4j.appender.C=org.apache.log4j.ConsoleAppender

log4j.appender.C.Target=System.err

log4j.appender.C.layout=org.apache.log4j.PatternLayout

log4j.appender.C.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n

 

15,在hadoop中的/share/Hadoop/common 路徑下找到hadoop-common-2.6.4.jar

   在jdk中的/jdk1.8.0_91/db/lib/路徑下找到derby.jar

將這倆個jar包拷貝到azkaban-exec-server和azkaban-web-server
安裝目錄下的extlib目錄,沒有則創建一個,找不到就下載

16,啟動

進入azkaban-exec-server目錄,執行start-exec.sh

進入azkaban-web-server目錄,執行start-web.sh

 

查看進程:jps

 

成功執行

通過http://localhost:8081可訪問到web-ui

16,此時的web界面:

 

需要更改配置文件解決:

 

將上圖中的web目錄的路徑加到azkaban.properties配置文件中,如下圖

 

 

再進入web目錄可以看到界面變得美觀

 

 

17, 用戶名密碼在 azkaban-web-server/conf/azkaban-users.xml 中配置


免責聲明!

本站轉載的文章為個人學習借鑒使用,本站對版權不負任何法律責任。如果侵犯了您的隱私權益,請聯系本站郵箱yoyou2525@163.com刪除。



 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM