Azkaban集群模式部署

1、安装所需软件

• Azkaban Web服务安装包：azkaban-web-server-0.1.0-SNAPSHOT.tar.gz
• Azkaban执行服务安装包：azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz
• sql脚本

节点规划 

2、数据库准备

linux123

进入mysql的客户端执行以下命令

mysql -uroot -p

执行以下命令：

SET GLOBAL validate_password_length=5;
SET GLOBAL validate_password_policy=0;
CREATE USER 'azkaban'@'%' IDENTIFIED BY 'azkaban';
GRANT all privileges ON azkaban.* to 'azkaban'@'%' identified by 'azkaban' WITH GRANT OPTION;
 
CREATE DATABASE azkaban;
use azkaban;
#解压数据库脚本
tar -zxvf azkaban-db-0.1.0-SNAPSHOT.tar.gz -C /opt/lagou/servers/azkaban
 
#加载初始化sql创建表
source /opt/lagou/servers/azkaban/azkaban-db-0.1.0-SNAPSHOT/create-all-sql0.1.0-SNAPSHOT.sql; 

3、配置Azkaban-web-server

进入linux122节点

解压azkaban-web-server

mkdir /opt/lagou/servers/azkaban
 
tar -zxvf azkaban-web-server-0.1.0-SNAPSHOT.tar.gz –C /opt/lagou/servers/azkaban/

进入azkaban-web-server根目录下

cd /opt/lagou/servers/azkaban/azkaban-web-server-0.1.0-SNAPSHOT
#⽣成ssl证书：
keytool -keystore keystore -alias jetty -genkey -keyalg RSA

注意：运行此命令后,会提示输入当前生成keystore的密码及相应信息,输入的密码请记住(所有密码统一以azkaban输入) 

修改 azkaban-web-server的配置文件 

cd /opt/lagou/servers/azkaban-web-server-3.51.0/conf
 
vim azkaban.properties
 
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai # 时区注意后⾯不要有空格
 
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
 
# Azkaban Jetty server properties. 开启使⽤ssl 并且知道端⼝
jetty.use.ssl=true
jetty.port=8443
jetty.maxThreads=25
 
# KeyStore for SSL ssl相关配置 注意密码和证书路径
jetty.keystore=keystore
jetty.password=azkaban
jetty.keypassword=azkaban
jetty.truststore=keystore
jetty.trustpassword=azkaban
 
# Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=linux123
mysql.database=azkaban
mysql.user=root
mysql.password=12345678
mysql.numconnections=100
 
#Multiple Executor 设置为false
azkaban.use.multiple.executors=true
#azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
azkaban.executorselector.comparator.Memory=1
azkaban.executorselector.comparator.LastDispatched=1
azkaban.executorselector.comparator.CpuUsage=1

添加属性

mkdir -p plugins/jobtypes
cd plugins/jobtypes/
vim commonprivate.properties
 
azkaban.native.lib=false
execute.as.user=false
memCheck.enabled=false

4、配置Azkaban-exec-server

linux123节点，上传exec安装包到/opt/lagou/software

tar -zxvf azkaban-exec-server-0.1.0-SNAPSHOT.tar.gz –C /opt/lagou/servers/azkaban/

修改azkaban-exec-server的配置文件 

cd /opt/lagou/servers/azkaban-exec-server-3.51.0/conf
 
vim azkaban.properties
 
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
default.timezone.id=Asia/Shanghai
 
# Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml
 
# Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects
 
# Where the Azkaban web server is located
azkaban.webserver.url=https://linux122:8443
 
# Azkaban mysql settings by default. Users should configure their own username and password.
database.type=mysql
mysql.port=3306
mysql.host=linux123
mysql.database=azkaban
mysql.user=root
mysql.password=12345678
mysql.numconnections=100
 
# Azkaban Executor settings
executor.maxThreads=50
executor.port=12321
executor.flow.threads=30

分发exec-server到linux121节点

cd /opt/lagou/servers
scp -r azkaban linux121:$PWD

5、启动服务

• 先启动exec-server
• 再启动web-server

#启动exec-server
bin/start-exec.sh
#启动web-server
bin/start-web.sh

激活我们的exec-server

启动webServer之后进程失败消失，可通过安装包根目录下对应启动日志进行排查。

需要手动激活executor

cd /opt/lagou/servers/azkaban/azkaban-exec-server-0.1.0-SNAPSHOT
curl -G "linux123:$(<./executor.port)/executor?action=activate" && echo

访问地址：https://linux122:8443