openstack-ansible部署zed版本all-in-one

部署架构

可用的操作系统：
Debian11（bullseye）
Ubuntu 22.04或20.04
CentOS Stream 9 或 Rocky Linux 9
系统必备的软件：
支持公钥访问的SSH
Python 3.8.x or 3.10.x
系统语言en_US.UTF-8 as the locale
cpu推荐：硬件辅助虚拟化功能，多核心处理器
磁盘大小：每台至少100GB
网络推荐：1G或10G网卡；聚合网卡；网卡可以划分VLAN；Jumbo架构（网络包携带更多数据）

机器准备
部署节点：Rocky linux 9，4C8G+100GB，内网ip：192.168.0.10
目标节点：Rocky linux 9，8C16G+100GB+100GB+100GB，内网ip：192.168.0.11+12+13

部署节点准备

安装Rocky linux 9

配置rocky

配置静态ip

dnf config-manager --enable devel
dnf -y update
dnf install -y network-scripts 
systemctl enable network.service --now
# 禁用NetworkManager
systemctl disable NetworkManager --now

# 静态ip(如果这个不会，说明底子太薄，还不适合维护openstack)
vim /etc/sysconfig/network-scripts/ifcfg-enp1s0
-----------------------------------------------
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=eui64
NAME=enp1s0
DEVICE=enp1s0
ONBOOT=yes
UUID=75d44a82-e738-48eb-80d7-0c4f1ee32bf7
IPADDR=192.168.0.10
PREFIX=24
GATEWAY=192.168.0.1

# 重启网卡
systemctl restart network
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

生成ssh公钥，实现deployment机器到其他机器的免密登录

ssh-keygen -t rsa
# 一路回车，最后生成 ~/.ssh/id_rsa.pub
1
2

根据官方文档的指示，下载必要包

dnf upgrade
reboot
dnf install git chrony openssh-server python3-devel sudo
dnf group install "Development Tools"
systemctl stop firewalld
systemctl mask firewalld
1
2
3
4
5
6

pip加速

# 创建.pip隐藏目录
mkdir ~/.pip

# 配置文件加入国内源
vim ~/.pip/pip.conf
-------------------------------------------
[global]
index-url = https://pypi.mirrors.ustc.edu.cn/simple/
trusted-host=pypi.tuna.tsinghua.edu.cn
timeout = 120

1
2
3
4
5
6
7
8
9
10
11

下载openstack-ansible及其依赖

# 两个仓库二选一，克隆openstack-ansible源码
git clone -b 26.1.2 https://opendev.org/openstack/openstack-ansible /opt/openstack-ansible
git clone -b 26.1.2 https://github.com/openstack/openstack-ansible.git /opt/openstack-ansible

# 安装依赖
cd /opt/openstack-ansible/
scripts/bootstrap-ansible.sh
1
2
3
4
5
6
7

最终显示System is bootstrapped and ready for use.表示成功

目标节点配置

升级系统包

dnf upgrade
1

关闭selinux

systemctl disable firewalld --now
setenforce 0
vim /etc/selinux/config
-----------------------------------------
SELINUX=disabled
1
2
3
4
5

安装软件包

dnf install iputils lsof openssh-server sudo tcpdump python3
1

降低内核日志打印等级并重启

echo "kernel.printk='4 1 7 4'" >> /etc/sysctl.conf
reboot
1
2

复制deployment的公钥到控制节点。在deployment机器上执行以下命令

ssh-copy-id -i ~/.ssh/id_rsa.pub  root@192.168.0.11
1

创建LVM存储
OpenStack-Ansible automatically configures LVM on the nodes, and overrides any existing LVM configuration. If you had a customized LVM configuration, edit the generated configuration file as needed.
OpenStack-Ansible会自动在节点上配置LVM，并覆盖已有的LVM。如果你想保留已有的LVM配置，请自行修改openstack-ansible里的配置文件。
创建一个名叫cinder-volumes的卷组。

pvcreate --metadatasize 2048 /dev/sdb /dev/sdc
vgcreate cinder-volumes /dev/sdb /dev/sdc
1
2

网络配置

Host network bridges information¶

1. LXC internal: lxcbr0
   The lxcbr0 bridge is required for LXC, but OpenStack-Ansible configures it automatically. It provides external (typically Internet) connectivity to containers with dnsmasq (DHCP/DNS) + NAT.
   This bridge does not directly attach to any physical or logical interfaces on the host because iptables handles connectivity. It attaches to eth0 in each container.
   The container network that the bridge attaches to is configurable in the openstack_user_config.yml file in the provider_networks dictionary.
2. Container management: br-mgmt
   The br-mgmt bridge provides management of and communication between the infrastructure and OpenStack services.
   The bridge attaches to a physical or logical interface, typically a bond0 VLAN subinterface. It also attaches to eth1 in each container.
   The container network interface that the bridge attaches to is configurable in the openstack_user_config.yml file.
3. Storage:br-storage
   The br-storage bridge provides segregated access to Block Storage devices between OpenStack services and Block Storage devices.
   The bridge attaches to a physical or logical interface, typically a bond0 VLAN subinterface. It also attaches to eth2 in each associated container.
   The container network interface that the bridge attaches to is configurable in the openstack_user_config.yml file.
4. OpenStack Networking tunnel: br-vxlan
   The br-vxlan interface is required if the environment is configured to allow projects to create virtual networks using VXLAN. It provides the interface for encapsulated virtual (VXLAN) tunnel network traffic.
   Note that br-vxlan is not required to be a bridge at all, a physical interface or a bond VLAN subinterface can be used directly and will be more efficient. The name br-vxlan is maintained here for consistency in the documentation and example configurations.
   The container network interface it attaches to is configurable in the openstack_user_config.yml file.
5. OpenStack Networking provider: br-vlan
   The br-vlan bridge is provides infrastructure for VLAN tagged or flat (no VLAN tag) networks.
   The bridge attaches to a physical or logical interface, typically bond1. It is not assigned an IP address because it handles only layer 2 connectivity.
   The container network interface that the bridge attaches to is configurable in the openstack_user_config.yml file.
   总结：
   lxcbr0是自动配置的，用于lxc容器，不会绑定到任何物理网卡，但是会和容器里的eth0接通，相当于大家熟悉的docker0。
   br-mgmt用于openstack各个组件服务通信，需要绑定到一个物理网口，并和容器里的eth1联通。
   br-storage用于块存储服务和对象存储服务，需要绑定到一个物理网口，并和容器里的eth2联通。
   br-vxlan：为openstack提供vxlan虚拟网络功能，可以是网桥、物理网口、网口的子口等形式存在。这个网桥与容器网卡的绑定可以通过openstack_user_config.yml配置
   br-vlan：提供vlan和flat网络，需要绑定到一个物理网口，并且不需要分配ip，它提供2层交换功能。这个网桥与容器网卡的绑定可以通过openstack_user_config.yml配置。

rocky linux网卡的创建永久网桥的方法：

# 禁用NetworkManager，启动Networking服务
dnf config-manager --enable devel
dnf -y update
dnf install -y network-scripts 
systemctl enable network.service --now
# 禁用NetworkManager
systemctl disable NetworkManager --now

# 物理网卡配置，调成dhcp模式并桥接到
vim /etc/sysconfig/network-scripts/ifcfg-enp1s0
---------------------------------------------
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=enp1s0
DEVICE=enp1s0
ONBOOT=yes # 网卡开机自启动
BRIDGE=br-mgmt # 网口桥接到网桥上
BRIDGE=br-storage # 网口桥接到网桥上
BRIDGE=br-vxlan # 网口桥接到网桥上


# br-mgmt网桥配置
vim /etc/sysconfig/network-scripts/ifcfg-br-mgmt
----------------------------------------------------
TYPE="Bridge"
DEVICE="br-mgmt" #网桥的名字
ONBOOT="yes"
BOOTPROTO="static"
IPADDR="192.168.0.11" #网桥的ip地址
NETMASK="255.255.255.0" #网桥所在子网的子网掩码
GATEWAY="192.168.0.1" #网桥所在子网的网关

# br-storage网桥配置
vim /etc/sysconfig/network-scripts/ifcfg-br-storage
-------------------------------------------
TYPE="Bridge"
DEVICE="br-storage" #网桥的名字
ONBOOT="yes"
BOOTPROTO="static"
IPADDR="192.168.0.12" #网桥的ip地址
NETMASK="255.255.255.0" #网桥所在子网的子网掩码
GATEWAY="192.168.0.1" #网桥所在子网的网关

# br-vxlan网桥配置
vim /etc/sysconfig/network-scripts/ifcfg-br-vxlan
-------------------------------------------
TYPE="Bridge"
DEVICE="br-vxlan" #网桥的名字
ONBOOT="yes"
BOOTPROTO="static"
IPADDR="192.168.0.12" #网桥的ip地址
NETMASK="255.255.255.0" #网桥所在子网的子网掩码
GATEWAY="192.168.0.1" #网桥所在子网的网关

# br-vlan网桥配置
vim /etc/sysconfig/network-scripts/ifcfg-br-vlan
-------------------------------------------
TYPE="Bridge"
DEVICE="br-vlan" #网桥的名字
ONBOOT="yes"
BOOTPROTO="none"
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

部署前配置

cp -R /opt/openstack-ansible/etc/openstack_deploy /etc/
cd /etc/openstack_deploy/
cp openstack_user_config.yml.example  /etc/openstack_deploy/openstack_user_config.yml

# 检查openstack_user_config.yml


# 检查user_variables.yml，重点关注install_method变量，source和distro更关注distro

# 添加额外服务etc/openstack_deploy/conf.d

1
2
3
4
5
6
7
8
9
10
11

生成密码文件

cd /opt/openstack-ansible
# ./scripts/pw-token-gen.py --file /etc/openstack_deploy/user_secrets.yml
1
2

运行playbook开始部署

setup-hosts.yml，相当于bootstrap server
setup-infrastructure.yml，安装memcached、rabbitmq、mysql
setup-openstack.yml，安装ops服务
执行安装

cd /etc/openstack-ansible
openstack-ansible setup-infrastructure.yml --syntax-check
1
2