ubuntn azkaban4.0.0链接mysql(5.1/8.0)数据库配置

本文会介绍azkaban4.0.0安装部署，分为部署连接mysql5.*系列和mysql8.0系列。以下有说明，敬请留意

流程

下载
配置构建
打包
拷贝
配置运行
登陆

一、下载

azkaban官网4.0.0下载地址（建议复制链接，直接wget 再解压）

cd /data/ ###你想要得地方，没有可以创建，可以随意替换
wget https://codeload.github.com/azkaban/azkaban/tar.gz/refs/tags/4.0.0
tar -xvzf azkaban-4.0.0.tar.gz
apt install git #安装git 
apt install openjdk-8-jdk #安装 jdk8
1
2
3
4
5

二、配置构建

*如果是mysql5.数据库

cd  /data/azkaban-4.0.0
./gradlew build installDist -x test 
1
2

*如果是mysql8.0.数据库

cd  /data/azkaban-4.0.0
vim build.gradle
##原来
mysqlConnector       : 'mysql:mysql-connector-java:5.1.28',
##替换
mysqlConnector       : 'mysql:mysql-connector-java:8.0.22',
保存退出
vim  azkaban-db/src/main/java/azkaban/db/MySQLDataSource.java 
##原来
setDriverClassName("com.mysql.jdbc.Driver");
##改为
setDriverClassName("com.mysql.cj.jdbc.Driver");
保存退出
./gradlew build installDist -x test 
1
2
3
4
5
6
7
8
9
10
11
12
13
14

*如果是mysql8.0.数据库，但是没改配置已经构建过了，才发现是mysql8.0得

cd  /data/azkaban-4.0.0
./gradlew clean    ###清除构建
vim build.gradle
##原来
mysqlConnector       : 'mysql:mysql-connector-java:5.1.28',
##替换
mysqlConnector       : 'mysql:mysql-connector-java:8.0.22',
保存退出
vim  azkaban-db/src/main/java/azkaban/db/MySQLDataSource.java 
##原来
setDriverClassName("com.mysql.jdbc.Driver");
##改为
setDriverClassName("com.mysql.cj.jdbc.Driver");
保存退出
./gradlew build installDist -x test 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

三、打包

分别打包
cd /data/azkaban-4.0.0/azkaban-exec-server/build/install/azkaban-exec-server
tar -cf azkaban-exec-server.tar.gz azkaban-exec-server
cd /data/azkaban-4.0.0/azkaban-web-server/build/install/azkaban-web-server
tar -cf azkaban-web-server.tar.gz azkaban-web-server
cd /data/azkaban-4.0.0/azkaban-db/build/install/azkaban-db
tar -cf azkaban-db.tar.gz azkaban-db
1
2
3
4
5
6
7

说明
azkaban-exec-server.tar.gz 执行器
azkaban-web-server.tar.gz 客户端
azkaban-db.tar.gz 数据库配置

Azkaban Executor Server 处理工作流和作业的实际执行。您可以从 master 分支构建最新版本。有关从源代码构建的说明，请参见此处。
Azkaban Executor Server handles the actual execution of the workflow and jobs. You can build the latest version from the master branch. See here for instructions on Building from Source.
Extract the package (executor distribution tar.gz from build folder) into a directory after gradle build. There should be the following directories.
Folder Description bin The scripts to start/stop Azkaban solo server
conf The configuration files for Azkaban solo server lib The jar
dependencies for Azkaban extlib Additional jars that are added to
extlib will be added to Azkaban’s classpath plugins the directory
where plugins can be installed

Azkaban Web Server handles project management, authentication, scheduling and trigger of executions. You can build the latest version
from the master branch. See here for instructions on Building from
Source.

Extract the package (executor distribution tar.gz from build folder)
into a directory after gradle build. There should be the following
directories.

Folder Description bin The scripts to start/stop Azkaban solo server
conf The configuration files for Azkaban solo server lib The jar
dependencies for Azkaban web The web (css, javascript, image) files
for Azkaban web server

azkaban-db主要需要它

/azkaban-db/build/distributions/azkaban-db-<version>.sql文件
azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql#数据库表
1
2

四、拷贝

把刚刚生成的包放到合适的服务器上，因为我是统一一台部署，所以就放一个文件夹下(按照需求来)

cd /opt
mv /data/azkaban-4.0.0/azkaban-exec-server/build/install/azkaban-exec-server/azkaban-exec-server.tar.gz /opt
mv /data/azkaban-4.0.0/azkaban-web-server/build/install/azkaban-web-server/azkaban-web-server.tar.gz  /opt
mv  /data/azkaban-4.0.0/azkaban-db/build/install/azkaban-db/azkaban-db.tar.gz /opt
tar -xf azkaban-exec-server.tar.gz
tar -xf azkaban-web-server.tar.gz
tar -xf azkaban-db.tar.gz 
1
2
3
4
5
6
7

五、配置运行

配置数据库

安装数据库
apt install mysql ###总之是安装mysql,自己选5.7还是8.0
# Example database creation command, although the db name doesn't need to be 'azkaban'
mysql> CREATE DATABASE azkaban;
# Example database creation command. The user name doesn't need to be 'azkaban'
mysql> CREATE USER 'username'@'%' IDENTIFIED BY 'password';
# give the user INSERT, SELECT, UPDATE, DELETE permission on all tables in the Azkaban db.
mysql> GRANT SELECT,INSERT,UPDATE,DELETE ON azkaban.* to ''@'%' WITH GRANT OPTION;




Mysql Packet Size may need to be re-configured. MySQL may have, by default, a ridiculously low allowable packet size. To increase it, you’ll need to have the property max_allowed_packet set to a higher number, say 1024M. To configure this in linux, open /etc/my.cnf. Somewhere after mysqld, add the following:

[mysqld]
...
max_allowed_packet=1024M
To restart MySQL, you can run:

$ sudo /sbin/service mysqld restart

创建阿兹卡班表
mysql> use azkaban;
mysql> source /opt/azkaban-db/create-all-sql-0.1.0-SNAPSHOT.sql;

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

配置azkaban-exec-server

cd /opt/azkaban-exec-server
vim  conf/azkaban.properties
default.timezone.id= #这里是时区
database.type=mysql
mysql.port=3306   #这里是mysql端口
mysql.host=127.0.0.1   #这里是mysql地址
mysql.database=azkaban #这里是mysql数据库名称
mysql.user=azkaban    #这里是mysql用户名
mysql.password=azkaban@123#这里是mysql密码
保存退出
然后运行：

cd /opt/azkaban-exec-server
./bin/start-exec.sh
之后，记得通过调用来激活执行器：

cd /opt/azkaban-exec-server
curl -G "localhost:$(<./executor.port)/executor?action=activate" && echo


查看日志
cat execServerLog***.out |grep -i error##根据自己运行时间来
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22