OpenStack离线Train版安装系列—11.2存储节点-Cinder存储服务组件

缺乏、安全感 2023-02-15 12:37 9阅读 0赞

> 本系列文章包含从OpenStack离线源制作到完成OpenStack安装的全部过程。  
> 在本系列教程中使用的OpenStack的安装版本为第20个版本Train（简称T版本），2020年5月13日，OpenStack社区发布了第21个版本Ussuri（简称U版本）。

> **OpenStack部署系列文章**  
> [OpenStack Victoria版 安装部署系列教程][OpenStack Victoria_]  
> [OpenStack Ussuri版 离线安装部署系列教程（全）][OpenStack Ussuri_]  
> [OpenStack Train版 离线安装部署系列教程（全）][OpenStack Train_]  
> 欢迎留言沟通，共同进步。

--------------------

### 文章目录 ###

*  11.2存储节点-Cinder存储服务组件
 *   *  一、存储节点准备好存储设备
     *   *  0.磁盘挂载
         *  1.安装LVM相关软件包
         *  2.创建LVM物理卷
         *  3.创建 LVM 卷组 cinder-volumes
         *  4.配置过滤器，只有实例可以访问块存储卷组,防止系统出错
     *  二、存储节点cinder相关软件安装与配置
     *   *  1.安装相关软件
         *  2.存储节点快速修改cinder配置
         *  3.存储节点启动cinder服务并配置开机自启动
     *  至此存储节点的cinder服务安装完毕
     *  三、（可选）安装和配置备份服务

--------------------

# 11.2存储节点-Cinder存储服务组件 #

在存储节点服务器安装cinder存储服务

> 官方参考：  
> [https://docs.openstack.org/install-guide/openstack-services.html][https_docs.openstack.org_install-guide_openstack-services.html]  
> [https://docs.openstack.org/train/install/][https_docs.openstack.org_train_install]  
> [https://docs.openstack.org/cinder/train/install/][https_docs.openstack.org_cinder_train_install]  
> [https://docs.openstack.org/cinder/train/install/index-rdo.html][https_docs.openstack.org_cinder_train_install_index-rdo.html]  
> [https://docs.openstack.org/cinder/train/install/cinder-storage-install-rdo.html][https_docs.openstack.org_cinder_train_install_cinder-storage-install-rdo.html]  
> （可选服务）  
> [https://docs.openstack.org/cinder/train/install/cinder-backup-install-rdo.html][https_docs.openstack.org_cinder_train_install_cinder-backup-install-rdo.html]  
> 存储节点，安装和配置备份服务，在“块存储”节点上执行这些步骤。  
> 博客：  
> [https://yinwucheng.com/?p=491][https_yinwucheng.com_p_491]  
> [https://www.cnblogs.com/tssc/p/9877026.html][https_www.cnblogs.com_tssc_p_9877026.html]  
> [https://www.cnblogs.com/wanlipenghtml/p/10721845.html][https_www.cnblogs.com_wanlipenghtml_p_10721845.html]  
> [https://docs.openstack.org/mitaka/zh\_CN/install-guide-rdo/cinder-storage-install.html][https_docs.openstack.org_mitaka_zh_CN_install-guide-rdo_cinder-storage-install.html]

*  存储节点建议单独部署服务器（最好是物理机），测试时也可以部署在控制节点或者计算节点
 *  在本文，存储节点使用LVM逻辑卷提供服务，需要提供一块空的磁盘用以创建LVM逻辑卷
 *  这里在VMware虚拟机增加一块100GB的磁盘
 *  在部署之前准备好存储设备。

## 一、存储节点准备好存储设备 ##

### 0.磁盘挂载 ###

在搭建的过程中使用的是VMware Workstation虚拟机来完成的，需要给存储节点新添加一个虚拟磁盘，并将磁盘挂载到虚拟机。  
（1）.分区  
增加了空间的硬盘是 /dev/sdb

[root@controller ~]# fdisk -l
    
    Disk /dev/sda: 68.7 GB, 68719476736 bytes, 134217728 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk label type: dos
    Disk identifier: 0x000a0797
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/sda1   *        2048     2099199     1048576   83  Linux
    /dev/sda2         2099200   134217727    66059264   8e  Linux LVM
    
    Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    
    Disk /dev/mapper/centos-root: 42.7 GB, 42651877376 bytes, 83304448 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    
    Disk /dev/mapper/centos-swap: 4160 MB, 4160749568 bytes, 8126464 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    
    Disk /dev/mapper/centos-home: 20.8 GB, 20824719360 bytes, 40673280 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    
    [root@controller ~]#

(2).分区

[root@controller ~]# fdisk /dev/sdb
    Welcome to fdisk (util-linux 2.23.2).
    
    Changes will remain in memory only, until you decide to write them.
    Be careful before using the write command.
    
    
    Command (m for help): p
    
    Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
    Units = sectors of 1 * 512 = 512 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk label type: dos
    Disk identifier: 0xfa73ca8f
    
       Device Boot      Start         End      Blocks   Id  System
    
    Command (m for help): n
    Partition type:
       p   primary (0 primary, 0 extended, 4 free)
       e   extended
    Select (default p): p
    Partition number (1-4, default 1): 3
    First sector (2048-41943039, default 2048): #回车
    Using default value 2048
    Last sector, +sectors or +size{
        K,M,G} (2048-41943039, default 41943039): #回车
    Using default value 41943039
    Partition 3 of type Linux and of size 20 GiB is set
    
    Command (m for help): t
    Selected partition 3
    Hex code (type L to list all codes): w
    Changed type of partition 'Linux' to 'unknown'
    
    Command (m for help): q
    
    [root@controller ~]# cd /dev/
    [root@controller dev]# partprobe
    [root@controller dev]# ls sd*
    sda  sda1  sda2  sdb
    [root@controller dev]#

(3).格式化

[root@controller dev]# mkfs.ext3 /dev/sdb
    mke2fs 1.42.9 (28-Dec-2013)
    /dev/sdb is entire device, not just one partition!
    Proceed anyway? (y,n) y
    Filesystem label=
    OS type: Linux
    Block size=4096 (log=2)
    Fragment size=4096 (log=2)
    Stride=0 blocks, Stripe width=0 blocks
    1310720 inodes, 5242880 blocks
    262144 blocks (5.00%) reserved for the super user
    First data block=0
    Maximum filesystem blocks=4294967296
    160 block groups
    32768 blocks per group, 32768 fragments per group
    8192 inodes per group
    Superblock backups stored on blocks:
    32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
    4096000
    
    Allocating group tables: done
    Writing inode tables: done
    Creating journal (32768 blocks): done
    Writing superblocks and filesystem accounting information: done
    
    [root@controller dev]#

### 1.安装LVM相关软件包 ###

安装LVM包

yum install lvm2 device-mapper-persistent-data -y

启动LVM的metadata服务并配置开机自启动

systemctl start lvm2-lvmetad.service
    systemctl status lvm2-lvmetad.service
    
    systemctl enable lvm2-lvmetad.service
    systemctl list-unit-files |grep lvm2-lvmetad |grep enabled

### 2.创建LVM物理卷 ###

创建LVM物理卷/dev/sdb

# 检查磁盘状态
    [root@controller ~]# fdisk -l
    
    # 创建LVM 物理卷 /dev/sdb
    [root@controller ~]# pvcreate /dev/sdb
      Physical volume "/dev/sdb" successfully created.

### 3.创建 LVM 卷组 cinder-volumes ###

(1)块存储服务会在这个卷组中创建逻辑卷

[root@controller ~]# vgcreate cinder-volumes /dev/sdb
      Volume group "cinder-volumes" successfully created
    [root@controller ~]# pvdisplay
      --- Physical volume ---
      PV Name               /dev/sda2
      VG Name               centos
      PV Size               <63.00 GiB / not usable 3.00 MiB
      Allocatable           yes
      PE Size               4.00 MiB
      Total PE              16127
      Free PE               1
      Allocated PE          16126
      PV UUID               aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ
      --- Physical volume ---
      PV Name               /dev/sdb
      VG Name               cinder-volumes
      PV Size               20.00 GiB / not usable 4.00 MiB
      Allocatable           yes
      PE Size               4.00 MiB
      Total PE              5119
      Free PE               5119
      Allocated PE          0
      PV UUID               2GmNUZ-03a6-I2dH-hG0W-4ISc-5Liy-30XbrO
    
    [root@controller ~]#

### 4.配置过滤器，只有实例可以访问块存储卷组,防止系统出错 ###

(1)新增磁盘sdb为LVM卷组

*  默认只会有openstack实例访问块存储卷组，不过，底层的操作系统也会管理这些设备并尝试将逻辑卷与系统关联。
 *  默认情况下LVM卷扫描工具会扫描整个/dev目录，查找所有包含lvm卷的块存储设备。如果其他项目在某个磁盘设备sda，sdc等上使用了lvm卷，那么扫描工具检测到这些卷时会尝试缓存这些lvm卷，可能导致底层操作系统或者其他服务无法正常调用他们的lvm卷组，从而产生各种问题，需要手动配置LVM，让LVM卷扫描工具只扫描包含"cinder-volume"卷组的设备/dev/sdb，我这边磁盘分区都是格式化的手工分区，目前不存在这个问题，以下是配置演示

将LVM重新配置为仅扫描包含cinder-volumes卷组的设备

vim /etc/lvm/lvm.conf
    -----------------------------
    devices {
        
    filter = [ "a/sdb/", "r/.*/"]
    }
    -----------------------------

在devices部分中，添加一个接受/dev/sdb设备并拒绝所有其他设备的过滤器：

*  配置规则：
 *  a用于接受，r用于拒绝。
 *  每个过滤器组中的元素都以a开头accept接受，或以 r 开头reject拒绝，后面连接设备名称的正则表达式规则。
 *  过滤器组必须以"r/.\*/"结束，过滤所有保留设备。
 *  可以使用命令:vgs -vvvv来测试过滤器。

(2)、查看系统磁盘卷类型

[root@controller ~]# vgdisplay
      WARNING: Device for PV aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ not found or rejected by a filter.
      Couldn't find device with uuid aTMOV5-l175-850d-vuur-0LRK-HGiz-JBLcwZ.
      WARNING: Couldn't find all devices for LV centos/swap while checking used and assumed devices.
      WARNING: Couldn't find all devices for LV centos/home while checking used and assumed devices.
      WARNING: Couldn't find all devices for LV centos/root while checking used and assumed devices.
      --- Volume group ---
      VG Name               centos
      System ID
      Format                lvm2
      Metadata Areas        0
      Metadata Sequence No  4
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                3
      Open LV               3
      Max PV                0
      Cur PV                1
      Act PV                0
      VG Size               <63.00 GiB
      PE Size               4.00 MiB
      Total PE              16127
      Alloc PE / Size       16126 / 62.99 GiB
      Free  PE / Size       1 / 4.00 MiB
      VG UUID               Ws4wQS-K0VZ-vp9q-TB9Y-sLP9-OKyb-SZbTd8
    
      --- Volume group ---
      VG Name               cinder-volumes
      System ID
      Format                lvm2
      Metadata Areas        1
      Metadata Sequence No  18
      VG Access             read/write
      VG Status             resizable
      MAX LV                0
      Cur LV                4
      Open LV               0
      Max PV                0
      Cur PV                1
      Act PV                1
      VG Size               <20.00 GiB
      PE Size               4.00 MiB
      Total PE              5119
      Alloc PE / Size       4874 / <19.04 GiB
      Free  PE / Size       245 / 980.00 MiB
      VG UUID               AOqwgQ-yYNb-7anP-1tr4-1e3m-Bxth-b3R9rP
    
    [root@controller ~]#

*  **注意**：
 *  **如果存储节点的操作系统磁盘/dev/sda使用的是LVM卷组，也需要将该设备添加到过滤器中**，配置文件/etc/lvm/lvm.conf添加如下：

devices {
        
      ......
    filter = [ "a/sda/", "a/sdb/", "r/.*/"]
      ......
    }

*  **如果计算节点的操作系统磁盘/dev/sda使用的是LVM卷组，也需要将该设备添加到过滤器中**，配置文件/etc/lvm/lvm.conf添加如下：

devices {
        
      ......
    filter = [ "a/sda/", "r/.*/"]
      ......
    }

## 二、存储节点cinder相关软件安装与配置 ##

### 1.安装相关软件 ###

yum install openstack-cinder targetcli python-keystone -y
    #用于快速配置
    yum install openstack-utils -y

### 2.存储节点快速修改cinder配置 ###

/etc/cinder/cinder.conf

cp /etc/cinder/cinder.conf /etc/cinder/cinder.conf.bak
    grep -Ev '#|^$' /etc/cinder/cinder.conf.bak>/etc/cinder/cinder.conf

openstack-config --set  /etc/cinder/cinder.conf database connection  mysql+pymysql://cinder:CINDER_DBPASS@controller/cinder
    openstack-config --set  /etc/cinder/cinder.conf DEFAULT transport_url  rabbit://openstack:RABBIT_USER_PASS@controller
    openstack-config --set  /etc/cinder/cinder.conf DEFAULT auth_strategy  keystone
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken www_authenticate_uri  http://controller:5000
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_url  http://controller:5000
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken memcached_servers  controller:11211
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken auth_type  password
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_domain_name  default
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken user_domain_name  default
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken project_name  service
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken username  cinder
    openstack-config --set  /etc/cinder/cinder.conf keystone_authtoken password  CINDER_PASS
    openstack-config --set  /etc/cinder/cinder.conf DEFAULT my_ip 10.0.0.41
    openstack-config --set  /etc/cinder/cinder.conf DEFAULT enabled_backends  lvm
    openstack-config --set  /etc/cinder/cinder.conf DEFAULT glance_api_servers  http://controller:9292
    openstack-config --set  /etc/cinder/cinder.conf lvm volume_driver cinder.volume.drivers.lvm.LVMVolumeDriver
    openstack-config --set  /etc/cinder/cinder.conf lvm volume_group cinder-volumes
    openstack-config --set  /etc/cinder/cinder.conf lvm target_protocol  iscsi
    openstack-config --set  /etc/cinder/cinder.conf lvm target_helper  lioadm
    openstack-config --set  /etc/cinder/cinder.conf oslo_concurrency lock_path  /var/lib/cinder/tmp
    
    #Replace RABBIT_PASS with the password you chose for the openstack account in RabbitMQ.
    #Replace MANAGEMENT_INTERFACE_IP_ADDRESS with the IP address of the management network interface on your storage node, typically 10.0.0.41 for the first node.
    
    #注意具体配置与之前版本有所变更
    #[lvm]
    # volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver
    # volume_group = cinder-volumes
    # target_protocol = iscsi
    # target_helper = lioadm
    
    egrep -v "^#|^$" /etc/cinder/cinder.conf
    grep '^[a-z]' /etc/cinder/cinder.conf

### 3.存储节点启动cinder服务并配置开机自启动 ###

需要启动2个服务

systemctl start openstack-cinder-volume.service target.service
    systemctl status openstack-cinder-volume.service target.service
    
    systemctl enable openstack-cinder-volume.service target.service
    systemctl list-unit-files |grep openstack-cinder |grep enabled
    systemctl list-unit-files |grep target.service |grep enabled

## 至此存储节点的cinder服务安装完毕 ##

至此，在存储节点安装cinder服务就完成了

## 三、（可选）安装和配置备份服务 ##

#安装软件包：
    yum install openstack-cinder
    
    #编辑/etc/cinder/cinder.conf文件并完成以下操作
    ---------------------------------------------------------------
    [DEFAULT]
    # ...
    backup_driver = cinder.backup.drivers.swift.SwiftBackupDriver
    backup_swift_url = SWIFT_URL
    ---------------------------------------------------------------
    #替换SWIFT_URL为对象存储服务的URL。可以通过显示对象库API端点来找到URL：
    
    openstack catalog show object-store
    
    #确定安装,启动块存储备份服务，并将其配置为在系统启动时启动：
    systemctl start openstack-cinder-backup.service
    systemctl status openstack-cinder-backup.service
    systemctl enable openstack-cinder-backup.service
    systemctl list-unit-files |grep openstack-cinder-backup.service |grep enabled

[OpenStack Victoria_]: https://blog.csdn.net/u013469753/article/details/116122782
[OpenStack Ussuri_]: https://blog.csdn.net/u013469753/article/details/106549085
[OpenStack Train_]: https://blog.csdn.net/u013469753/article/details/106549917
[https_docs.openstack.org_install-guide_openstack-services.html]: https://docs.openstack.org/install-guide/openstack-services.html
[https_docs.openstack.org_train_install]: https://docs.openstack.org/train/install/
[https_docs.openstack.org_cinder_train_install]: https://docs.openstack.org/cinder/train/install/
[https_docs.openstack.org_cinder_train_install_index-rdo.html]: https://docs.openstack.org/cinder/train/install/index-rdo.html
[https_docs.openstack.org_cinder_train_install_cinder-storage-install-rdo.html]: https://docs.openstack.org/cinder/train/install/cinder-storage-install-rdo.html
[https_docs.openstack.org_cinder_train_install_cinder-backup-install-rdo.html]: https://docs.openstack.org/cinder/train/install/cinder-backup-install-rdo.html
[https_yinwucheng.com_p_491]: https://yinwucheng.com/?p=491
[https_www.cnblogs.com_tssc_p_9877026.html]: https://www.cnblogs.com/tssc/p/9877026.html
[https_www.cnblogs.com_wanlipenghtml_p_10721845.html]: https://www.cnblogs.com/wanlipenghtml/p/10721845.html
[https_docs.openstack.org_mitaka_zh_CN_install-guide-rdo_cinder-storage-install.html]: https://docs.openstack.org/mitaka/zh_CN/install-guide-rdo/cinder-storage-install.html