本次测试采用3台虚拟机,操作系统版本为centos7.6。
Hadoop版本为3.3.4,其中Namenode采用HA高可用架构,Zookeeper版本为3.8.0
kerberos采用默认YUM源安装,版本为:1.15.1-55
操作系统用户:hadoop 操作系统用户组:hadoop
| IP地址 | 主机名 | ZK | HDFS | YARN | KDC |
| 192.168.121.101 | node101.cc.local | server.1 | NameNode DataNode JournalNode | ResourceManager NodeManager JobHistory | KDC master |
| 192.168.121.102 | node102.cc.local | server.2 | NameNode DataNode JournalNode | ResourceManager NodeManager | KDC slaver 1级 |
| 192.168.121.103 | node103.cc.local | server.3 | DataNode JournalNode | NodeManager | KDC slaver 2级 |
本次测试使用统一hadoop服务主体(Principal),不再为各服务创建独立主体(Principal)
| 服务 | 所在主机 | 主体(Principal) |
| NameNode DataNode JournalNode ResourceManager NodeManager JobHistory Web UI | node101.cc.local | hadoop/node101.cc.local |
| NameNode DataNode JournalNode ResourceManager NodeManager Web UI | node102.cc.local | hadoop/node102.cc.local |
| DataNode JournalNode NodeManager Web UI | node103.cc.local | hadoop/node103.cc.local |
### 在每个节点均执行操作 ###
echo "####mkdir for keytab####"
mkdir /etc/security/keytab/
chown -R root:hadoop /etc/security/keytab/
chmod 770 /etc/security/keytab/
使用Kerberos管理员用户登录Kerberos数据库客户端kadmin
#####在node101节点上执行#########
# kadmin -p kws/admin@CC.LOCAL
Authenticating as principal kws/admin@CC.LOCAL with password.
Password for kws/admin@CC.LOCAL: kws!101
kadmin: addprinc -randkey hadoop/node101.cc.local
kadmin: ktadd -k /etc/security/keytab/hadoop.keytab hadoop/node101.cc.local
#####在node102节点上执行##########
kadmin -p kws/admin@CC.LOCAL
Authenticating as principal kws/admin@CC.LOCAL with password.
Password for kws/admin@CC.LOCAL: kws!101
kadmin: kadmin: addprinc -randkey hadoop/node102.cc.local
kadmin: xst -k /etc/security/keytab/hadoop.keytab hadoop/node102.cc.local
#####在node103节点上执行##########
kadmin -p kws/admin@CC.LOCAL
Authenticating as principal kws/admin@CC.LOCAL with password.
Password for kws/admin@CC.LOCAL: kws!101
kadmin: addprinc -randkey hadoop/node103.cc.local
kadmin: xst -k /etc/security/keytab/hadoop.keytab hadoop/node103.cc.local说明:
addprinc test/test:作用是新建主体addprinc:增加主体
-randkey:密码随机,因hadoop各服务均通过keytab文件认证,故密码可随机生成
xxx/xxx:新增的主体
xst -k /etc/security/keytab/test.keytab test/test:作用是将主体的密钥写入keytab文件,生成keytab文件,使用ktadd或xst命令均可
xst:将主体的密钥写入keytab文件
-k /etc/security/keytab/test.keytab:指明keytab文件路径和文件名
xxx/xxx:主体
#####在node101节点上执行#########
# kadmin -p kws/admin -w kws\!101 -q"addprinc -randkey hadoop/node101.cc.local"
# kadmin -pkws/admin -wkws\!101 -q"xst -k /etc/security/keytab/hadoop.keytab hadoop/node101.cc.local"#####在node102节点上执行#########
# kadmin -pkws/admin -wkws\!101 -q"addprinc -randkey hadoop/node102.cc.local"
# kadmin -pkws/admin -wkws\!101 -q"xst -k /etc/security/keytab/hadoop.keytab hadoop/node102.cc.local"#####在node103节点上执行#########
# kadmin -pkws/admin -wkws\!101 -q"addprinc -randkey hadoop/node103.cc.local"
# kadmin -pkws/admin -wkws\!101 -q"xst -k /etc/security/keytab/hadoop.keytab hadoop/node103.cc.local"说明:
-p:主体
-w:密码,密码中有!符号时需要使用\转义Bash命令
-q:执行语句
chown -R root:hadoop /etc/security/keytab/
chmod 660 /etc/security/keytab/*
编辑hadoop-3.3.4/etc/hadoop/core-site.xml
### 在每个节点均执行操作 ###
###提示:通过vi手工添加
中的内容 ###
hadoop.security.auth_to_local
RULE:[2:$1/$2·@$0](hadoop/.*@CC.LOCAL)s/.*/hadoop/
DEFAULT
hadoop.security.authentication
kerberos
hadoop.security.authorization
true
hadoop.rpc.protection
authentication
说明:
Hadoop使用被hadoop.security.auth_to_local指定的规则来映射kerberos principals到操作系统账号
RULE:[
RULE:[2:$1/$2@$0](hadoop/.*@.*CC.LOCAL)s/.*/hadoop/
[
2表示@前包含两个component
在格式化串中,$0 表示realm,$1表示第一个component,$2表示第二个component
(
(hadoop/.*@.*CC.LOCAL)为接收过滤器是一个标准的正则表达式,用于匹配第一部分——principal translation,输出的短名称,只有当成功匹配时,才会将该短名称传递到第三部分——short name substitution。当不匹配时,则跳过该rule,进行下一条rule的匹配,否则进行下一步。
可以理解为linux中sed替换命令 (s/.../.../g) ,其输入是principal translation提取出的短名称。这部分是可选的
DEFAULT 是默认规则,默认将principal的第一个component作为短名称输出
验证匹配规则
#hadoop org.apache.hadoop.security.HadoopKerberosName hadoop/node101.cc.local@CC.LOCAL
Name: hadoop/node101.cc.local@CC.LOCAL to hadoop
编辑hadoop-3.3.4/etc/hadoop/hdfs-site.xml
### 在每个节点均执行操作 ###
###提示:通过vi手工添加
中的内容 ###
dfs.block.access.token.enable
true
dfs.namenode.kerberos.principal
hadoop/_HOST@CC.LOCAL
dfs.namenode.keytab.file
/etc/security/keytab/hadoop.keytab
dfs.datanode.kerberos.principal
hadoop/_HOST@CC.LOCAL
dfs.datanode.keytab.file
/etc/security/keytab/hadoop.keytab
dfs.journalnode.kerberos.principal
hadoop/_HOST@CC.LOCAL
dfs.journalnode.keytab.file
/etc/security/keytab/hadoop.keytab
dfs.namenode.kerberos.internal.spnego.principal
hadoop/_HOST@CC.LOCAL
dfs.web.authentication.kerberos.principal
hadoop/_HOST@CC.LOCAL
dfs.web.authentication.kerberos.keytab
/etc/security/keytab/hadoop.keytab
dfs.http.policy
HTTPS_ONLY
dfs.data.transfer.protection
authentication
编辑hadoop-3.3.4/etc/hadoop/yarn-site.xml
### 在每个节点均执行操作 ###
###提示:通过vi手工添加
中的内容 ###
yarn.resourcemanager.principal
hadoop/_HOST@CC.LOCAL
yarn.resourcemanager.keytab
/etc/security/keytab/hadoop.keytab
yarn.nodemanager.principal
hadoop/_HOST@CC.LOCAL
yarn.nodemanager.keytab
/etc/security/keytab/hadoop.keytab
编辑hadoop-3.3.4/etc/hadoop/mapred-site.xml
### 在每个节点均执行操作 ###
###提示:通过vi手工添加
中的内容 ###
mapreduce.jobhistory.principal
hadoop/_HOST@CC.LOCAL
mapreduce.jobhistory.keytab
/etc/security/keytab/hadoop.keytab
# openssl req -new -x509 -keyout /etc/security/keytab/hdfs_ca_key -out /etc/security/keytab/hdfs_ca_cert -days 36500 -subj '/C=CC/ST=CC/L=CC/O=CC/OU=CC/CN=CC'
Generating a 2048 bit RSA private key
.....................................................................................................+++
.....................+++
writing new private key to '/etc/security/keytab/hdfs_ca_key'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
以上命令使用 OpenSSL 工具生成一个自签名的 X.509 证书,执行完成后,会在目录下生成私钥文件hdfs_ca_key和证书文件hdfs_ca_cert
scp -rp /etc/security/keytab/hdfs_ca_* node102:/etc/security/keytab/
scp -rp /etc/security/keytab/hdfs_ca_* node103:/etc/security/keytab/
keystore文件存储了SSL握手所涉及的私钥以及证书链信息
#####node101####
# keytool -keystore /etc/security/keytab/keystore -alias node101 -genkey -keyalg RSA -dname "CN=node101.cc.local, OU=CC, O=CC, L=CC, ST=CC, C=CC"#####node102####
# keytool -keystore /etc/security/keytab/keystore -alias node102 -genkey -keyalg RSA -dname "CN=node102.cc.local, OU=CC, O=CC, L=CC, ST=CC, C=CC"#####node103####
# keytool -keystore /etc/security/keytab/keystore -alias node103 -genkey -keyalg RSA -dname "CN=node103.cc.local, OU=CC, O=CC, L=CC, ST=CC, C=CC"
CN建议配置为各节点的hostname,这样不会出现验证出错。
truststore文件存储了可信任的根证书
### 在每个节点均执行操作 ###
# keytool -keystore /etc/security/keytab/truststore -alias CARoot -import -file /etc/security/keytab/hdfs_ca_cert
Enter keystore password:
Re-enter new password:
Owner: CN=CC, OU=CC, O=CC, L=CC, ST=CC, C=CC
Issuer: CN=CC, OU=CC, O=CC, L=CC, ST=CC, C=CC
Serial number: d8e316146bfe7317
Valid from: Thu Apr 11 15:37:59 CST 2024 until: Sat Mar 18 15:37:59 CST 2124
Certificate fingerprints:
SHA1: 17:85:CA:D7:86:8C:8F:9F:F4:5F:30:B7:FB:43:E0:02:BF:19:D6:F2
SHA256: 65:85:F0:29:87:0B:09:A6:BD:AD:6F:99:BE:20:3D:9D:FF:8D:7A:44:70:DB:95:C0:D4:13:49:36:27:1E:64:FA
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: F8 22 25 FF C1 89 F8 9D 7F 48 FF 3E AA E0 DF 75 ."%......H.>...u
0010: F6 B6 A7 AE ....
]
]#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:2147483647
]#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: F8 22 25 FF C1 89 F8 9D 7F 48 FF 3E AA E0 DF 75 ."%......H.>...u
0010: F6 B6 A7 AE ....
]
]Trust this certificate? [no]: yes
Certificate was added to keystore
以上命令使用 Java 工具 keytool 将之前生成的自签名 CA 证书 hdfs_ca_cert 导入到指定的 truststore 文件中,并将其命名为 CARoot。命令执行后会在各个节点目录下生成truststore文件。
#####node101######
# keytool -certreq -alias node101 -keystore /etc/security/keytab/keystore -file /etc/security/keytab/cert
Enter keystore password:Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
#####node102######
# keytool -certreq -alias node102 -keystore /etc/security/keytab/keystore -file /etc/security/keytab/cert
Enter keystore password:Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
#####node103######
# keytool -certreq -alias node103 -keystore /etc/security/keytab/keystore -file /etc/security/keytab/cert
Enter keystore password:Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
注意:--alias 需要与各节点生成keystore指定的别名一致。
使用最开始生成的hdfs_ca_cert证书文件和hdfs_ca_key密钥文件对cert进行签名,生成自签名证书
# openssl x509 -req -CA /etc/security/keytab/hdfs_ca_cert -CAkey /etc/security/keytab/hdfs_ca_key -in /etc/security/keytab/cert -out /etc/security/keytab/cert_signed -days 36500 -CAcreateserial
Signature ok
subject=/C=CC/ST=CC/L=CC/O=CC/OU=CC/CN=node101.cc.local
Getting CA Private Key
Enter pass phrase for /etc/security/keytab/hdfs_ca_key:
将之前生成的hdfs_ca_cert证书文件导入到keystore中
### 在每个节点均执行操作 ###
# keytool -keystore /etc/security/keytab/keystore -alias CARoot -import -file /etc/security/keytab/hdfs_ca_cert
Enter keystore password:
Owner: CN=CC, OU=CC, O=CC, L=CC, ST=CC, C=CC
Issuer: CN=CC, OU=CC, O=CC, L=CC, ST=CC, C=CC
Serial number: 96592715ce2f9bd9
Valid from: Thu Apr 11 16:14:28 CST 2024 until: Sat Mar 18 16:14:28 CST 2124
Certificate fingerprints:
SHA1: 12:EF:50:6F:7F:91:71:13:21:E9:F6:5D:64:6A:14:13:A4:E7:9E:AC
SHA256: F1:E5:92:5F:61:3B:D1:13:23:E1:1C:F8:ED:E1:0E:98:FD:25:10:66:C3:2B:87:B4:1F:BD:3A:50:2C:38:B9:8D
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3Extensions:
#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: 44 1F 19 D8 4A 22 FC AB 01 7B 18 3F FB 85 9B F2 D...J".....?....
0010: 33 D8 7A 1F 3.z.
]
]#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
CA:true
PathLen:2147483647
]#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: 44 1F 19 D8 4A 22 FC AB 01 7B 18 3F FB 85 9B F2 D...J".....?....
0010: 33 D8 7A 1F 3.z.
]
]Trust this certificate? [no]: yes
Certificate was added to keystoreWarning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
将生成的cert_signed自签名证书导入到keystore中
# keytool -keystore /etc/security/keytab/keystore -alias node101 -import -file /etc/security/keytab/cert_signed
Enter keystore password:
Certificate reply was installed in keystoreWarning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
# keytool -keystore /etc/security/keytab/keystore -alias node102 -import -file /etc/security/keytab/cert_signed
Enter keystore password:
Certificate reply was installed in keystoreWarning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
# keytool -keystore /etc/security/keytab/keystore -alias node103 -import -file /etc/security/keytab/cert_signed
Enter keystore password:
Certificate reply was installed in keystoreWarning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /etc/security/keytab/keystore -destkeystore /etc/security/keytab/keystore -deststoretype pkcs12".
确保hadoop用户(HDFS的启动用户)具有对所生成keystore文件的读权限
# chown -R root:hadoop /etc/security/keytab
# chmod 660 /etc/security/keytab/*
拷贝并编辑hadoop-3.3.4/etc/hadoop/ssl-server.xml
### 在每个节点均执行操作 ###
ssl.server.truststore.location
/etc/security/keytab/truststore
Truststore to be used by NN and DN. Must be specified.
ssl.server.truststore.password
hdp101
Optional. Default value is "".
ssl.server.truststore.type
jks
Optional. The keystore file format, default value is "jks".
ssl.server.truststore.reload.interval
10000
Truststore reload check interval, in milliseconds.
Default value is 10000 (10 seconds).
ssl.server.keystore.location
/etc/security/keytab/keystore
Keystore to be used by NN and DN. Must be specified.
ssl.server.keystore.password
hdp101
Must be specified.
ssl.server.keystore.keypassword
hdp!101
Must be specified.
ssl.server.keystore.type
jks
Optional. The keystore file format, default value is "jks".
ssl.server.exclude.cipher.list
TLS_ECDHE_RSA_WITH_RC4_128_SHA,SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA,
SSL_RSA_WITH_DES_CBC_SHA,SSL_DHE_RSA_WITH_DES_CBC_SHA,
SSL_RSA_EXPORT_WITH_RC4_40_MD5,SSL_RSA_EXPORT_WITH_DES40_CBC_SHA,
SSL_RSA_WITH_RC4_128_MD5
Optional. The weak security cipher suites that you want excluded
from SSL communication.
注意:启动服务时需要使用对应的操作系统用户
###切换到hadoop用户###
# su -i hadoop
###启动Namenode###
# hdfs --daemon start namenode
####启动Datanode###
# hdfs --daemon start datanode
####启动Journalnode###
hdfs --daemon start journalnode
####启动zkfc###
hdfs --daemon start zkfc
前提:已配置了hadoop用户的节点间的免密登录
修改$HADOOP_HOME/sbin/start-dfs.sh脚本,在顶部增加以下环境变量
HDFS_NAMENODE_USER=hadoop
HDFS_DATANODE_USER=hadoop
HDFS_JOURNALNODE_USER=hadoop
HDFS_ZKFC_USER=hadoop
同样,修改$HADOOP_HOME/sbin/stop-dfs.sh脚本,在顶部增加环境变量
####启动HDFS###可使用root用户###
# start-dfs.sh
访问地址:
https://192.168.121.101:9871/
https://192.168.121.102:9871/
按照官方文档设置路径访问权限
| Filesystem | Path | User:Group | Permissions | 测试用户 |
|---|---|---|---|---|
| local | dfs.namenode.name.dir | hdfs:hadoop | drwx------ | hadoop |
| local | dfs.datanode.data.dir | hdfs:hadoop | drwx------ | hadoop |
| local | $HADOOP_LOG_DIR | hdfs:hadoop | drwxrwxr-x | hadoop |
| local | $YARN_LOG_DIR | yarn:hadoop | drwxrwxr-x | hadoop |
| local | yarn.nodemanager.local-dirs | yarn:hadoop | drwxr-xr-x | hadoop |
| local | yarn.nodemanager.log-dirs | yarn:hadoop | drwxr-xr-x | hadoop |
| local | container-executor | root:hadoop | --Sr-s--* | root |
| local | conf/container-executor.cfg | root:hadoop | r-------* | root |
| hdfs | / | hdfs:hadoop | drwxr-xr-x | hadoop |
| hdfs | /tmp | hdfs:hadoop | drwxrwxrwxt | hadoop |
| hdfs | /user | hdfs:hadoop | drwxr-xr-x | hadoop |
| hdfs | yarn.nodemanager.remote-app-log-dir | yarn:hadoop | drwxrwxrwxt | hadoop |
| hdfs | mapreduce.jobhistory.intermediate-done-dir | mapred:hadoop | drwxrwxrwxt | hadoop |
| hdfs | mapreduce.jobhistory.done-dir | mapred:hadoop | drwxr-x--- | hadoop |
###hdfs-site.xml dfs.namenode.name.dir/datanode journalnode
# chown -R hadoop:hadoop /opt/hadoop
# chmod 700 /opt/hadoop/hadoop-3.3.4/data/namenode
# chmod 700 /opt/hadoop/hadoop-3.3.4/data/datanode
# chmod 700 /opt/hadoop/hadoop-3.3.4/data/journalnode###hadoop-env.sh HADOOP_LOG_DIR use HADOOP_HOME/logs###
# chmod 775 /opt/hadoop/hadoop-3.3.4/logs###yarn-env.sh YARN_LOG_DIR use default HADOOP_LOG_DIR ###
# chmod 775 /opt/hadoop/hadoop-3.3.4/logs###yarn-site.xml yarn.nodemanager.local-dirs use default hadoop.tmp.dir###
# chmod 755 /opt/hadoop/tmp###yarn-site.xml yarn.nodemanager.log-dirs use default HADOOP_LOG_DIR/userlogs ###
# chmod 755 /opt/hadoop/hadoop-3.3.4/logsuserlogs###container-executor###
# chown root:hadoop /opt/hadoop/hadoop-3.3.4/bin/container-executor
# chmod 6050 /opt/hadoop/hadoop-3.3.4/bin/container-executor###conf/container-executor.cfg###
# chown root:hadoop /opt/hadoop/hadoop-3.3.4/etc/hadoop/container-executor.cfg
# chown root:hadoop /opt/hadoop/hadoop-3.3.4/etc/hadoop
# chown root:hadoop /opt/hadoop/hadoop-3.3.4/etc
# chown root:hadoop /opt/hadoop/hadoop-3.3.4
# chown root:hadoop /opt/hadoop
# chmod 400 /opt/hadoop/hadoop-3.3.4/etc/hadoop/container-executor.cfg
#####在node101节点上执行#########
# kadmin -p kws/admin -w kws\!101 -q"addprinc hadoop/hadoop"
WARNING: no policy specified for hadoop/hadoop@CC.LOCAL; defaulting to no policy
Enter password for principal "hadoop/hadoop@CC.LOCAL": hdp101
Re-enter password for principal "hadoop/hadoop@CC.LOCAL": hdp101
Principal "hadoop/hadoop@CC.LOCAL" created.
# kinit hadoop/hadoop
Password for hadoop/hadoop@CC.LOCAL:# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hadoop/hadoop@CC.LOCALValid starting Expires Service principal
04/23/2024 18:01:48 04/24/2024 18:01:48 krbtgt/CC.LOCAL@CC.LOCAL
renew until 04/30/2024 18:01:48
# hadoop fs -chown hadoop:hadoop / /tmp /user
# hadoop fs -chmod 755 /
# hadoop fs -chmod 1777 /tmp
# hadoop fs -chmod 775 /user# hadoop fs -chown hadoop:hadoop /tmp/logs
# hadoop fs -chmod 1777 /tmp/logs# hadoop fs -chown -R hadoop:hadoop /tmp/hadoop-yarn
# hadoop fs -chmod -R 770 /tmp/hadoop-yarn/
# hadoop fs -chmod -R 1777 /tmp/hadoop-yarn/staging/history/done_intermediate
# hadoop fs -chmod -R 750 /tmp/hadoop-yarn/staging/history/done
编辑container-executor.cfg
yarn.nodemanager.linux-container-executor.group=hadoop
banned.users=hadoop
min.user.id=1000
allowed.system.users=
feature.tc.enabled=false
注意:启动服务时需要使用对应的操作系统用户
###切换到hadoop用户###
# su -i hadoop
###启动Resourcemanager###
# yarn --daemon start resourcemanager
####启动nodemanager###
# yarn --daemon start nodemanager
前提:已配置了hadoop用户的节点间的免密登录
修改$HADOOP_HOME/sbin/start-yarn.sh脚本,在顶部增加以下环境变量
YARN_RESOURCEMANAGER_USER=hadoop
YARN_NODEMANAGER_USER=hadoop
同样,修改$HADOOP_HOME/sbin/stop-yarn.sh脚本,在顶部增加环境变量
####启动Yarn###可使用root用户###
# start-yarn.sh
注意:可能报错:/opt/hadoop/hadoop-3.3.4/bin/container-executor: error while loading shared
libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory。此时需要升级openssl到1.1.1版本,可以从官网[ 1.1.1 ] - /source/old/1.1.1/index.html下载后编译生成libcrypto.so.1.1文件后拷贝到/usr/lib64下即可
访问地址:
http://192.168.121.101:8088/
http://192.168.121.102:8088/
注意:启动服务时需要使用对应的操作系统用户
###切换到hadoop用户###
# su -i hadoop
###启动historyserver###
# mapred --daemon start historyserver