[rhel6.5]ISCSI配置过程详解

末蓝、 2022-05-30 11:05 229阅读 0赞

*  **关于数据存储的几种方式**  
    
    
     *  1.DAS:表示直接访问存储,例如本机上面的磁盘
     *  2.SAN:存储区域网,来自局域网里面其他存储设备提供的磁盘;
     *  3.NAS:`Network Attached Storge`:表示来自NAS提供的文件系统,只能够独立使用,不可以进行格式化;
 *  **关于NAS以及SAN**  
    
    
     *  NAS:`Network Attached Storage` 表示网络附加存储,提供的是文件系统上面的服务,可以提供锁控制等机制,`NFS ,SAMBA, FTP`
     *  SAN:`Storage Area Networks`,表示的含义是存储区域网,通过提供块设备,来提供服务,通常包括`SCSI`等;
     *  ISCSI:表示的含义是`Internet SCSI` 主要通过`TCP/IP`技术将存储设备通过`iSCSI target`,做成提供磁盘设备的服务端,在通过`iscsi Initiator`做成iscsi target\` 客户端;
 *  **ISCSI主要包含两个部分:**  
    
    
     *  iSCSI target:存储设备的服务器端,存放磁盘或者RAID设备,也可以用于将Linux主机仿真成为Linux设备,用于提供其他主机使用的磁盘;
     *  iSCSI initiator:能够使用target设备的Client 通常是服务器,Client必须使用这个功能才能够连接到iSCSI提供的磁盘;
 *  **iSCSI Target的设置:**  
    
    
     *  所需要的软件:  
        
        
         *  `scsi-target-utils`:用来将Linux系统仿真成为iSCSI target的功能;
         *  `iscsi-target-utils`:挂在来自target的磁盘到Linux本机上面;
     *  `scsi-target-utils`提供的配置文件:  
        
        
         *  `/etc/sysconfig/tgtd`: 主配置文件用于设置共享的磁盘文件以及磁盘文件的格式;
         *  `/usr/sbin/tgt-admin`: 在线查询,删除target等功能的设置工具;
         *  `/usr/sbin/tgt-setup-lun`:建立target以及设置共享磁盘以及可以使用的客户端等工具软件;
         *  `/usr/sbin/tgtadm`: 手动直接管理的管理员工具;
         *  `/usr/sbin/tgtd`: 提供scsi-target服务的主程序;
         *  `/usr/sbin/tgtimg`: 搭建预计共享的影响文件设备的工具;
 *  **scsi-target支持的共享设备类型:**  
    
    
     *  1.使用dd命令建立的大型文件,不需要事先进行格式化
     *  2.使用单一的分区;
     *  3.使用但以完整的磁盘,无需事先分区;
     *  4.使用磁盘阵列进行共享;
     *  5.使用软件磁盘阵列(software RAID)共享成单一磁盘;
     *  6.使用LVM的LV设备;
 *  接下来进行规划
    
    > 1.`server23.com`作为`scsi-target`,  
    > 2.`server22.com`作为`iscsi-initiator`,用来事先`SCSI`设备的访问;
    > 
    > 1.  这里的共享计划使用`dd`命令生成的大型文件,普通的分区`[/dev/vdb1]`以及使用`LVM[//dev/vdb2]`卷组的方式;
 *  **首先来准备共享的设备文件**
 *  1.首先建立`500MB`的一个大型文件

[root@server23 ~]# mkdir /srv/iscsi
    [root@server23 ~]# dd if=/dev/zero of=/srv/iscsi/disk1.img bs=1M count=500
    500+0 records in
    500+0 records out
    524288000 bytes (524 MB) copied, 8.46649 s, 61.9 MB/s
    [root@server23 ~]# chcon -Rv -t tgtd_var_lib_t /srv/iscsi/ //用于修改文件的安全上下文,比关闭selinux更加安全,selinux以后会补充说明;
    changing security context of `/srv/iscsi/disk1.img' changing security context of `/srv/iscsi/'
    [root@server23 ~]# ls -lh /srv/iscsi/disk1.img 
    -rw-r--r--. 1 root root 500M 3月  11 10:47 /srv/iscsi/disk1.img

*  建立分区,`[关于kvm添加磁盘的使用方法参照以前的文章]`

[root@server23 ~]# fdisk  /dev/vdb 
    Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
    Building a new DOS disklabel with disk identifier 0xff1fad05.
    Changes will remain in memory only, until you decide to write them.
    After that, of course, the previous content won't be recoverable.
    
    Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
    
    WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
             switch off the mode (command 'c') and change display units to
             sectors (command 'u').
    
    Command (m for help): p
    
    Disk /dev/vdb: 3221 MB, 3221225472 bytes
    16 heads, 63 sectors/track, 6241 cylinders
    Units = cylinders of 1008 * 512 = 516096 bytes
    Sector size (logical/physical): 512 bytes / 512 bytes
    I/O size (minimum/optimal): 512 bytes / 512 bytes
    Disk identifier: 0xff1fad05
    
       Device Boot      Start End Blocks Id System Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-6241, default 1): Using default value 1 Last cylinder, +cylinders or +size{K,M,G} (1-6241, default 6241): +1G Command (m for help): t Selected partition 1 Hex code (type L to list codes): 83 Changed system type of partition 1 to 8e (Linux) Command (m for help): p Disk /dev/vdb: 3221 MB, 3221225472 bytes 16 heads, 63 sectors/track, 6241 cylinders Units = cylinders of 1008 * 512 = 516096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xff1fad05 Device Boot Start End Blocks Id System /dev/vdb1 1 2082 1049296+ 83 Linux Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 2 First cylinder (2083-6241, default 2083): Using default value 2083 Last cylinder, +cylinders or +size{K,M,G} (2083-6241, default 6241): +1G Command (m for help): t Partition number (1-4): 2 Hex code (type L to list codes): 8e Changed system type of partition 2 to 8e (Linux LVM) Command (m for help): p Disk /dev/vdb: 3221 MB, 3221225472 bytes 16 heads, 63 sectors/track, 6241 cylinders Units = cylinders of 1008 * 512 = 516096 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xff1fad05 Device Boot Start End Blocks Id System /dev/vdb1 1 2082 1049296+ 83 Linux /dev/vdb2 2083 4164 1049328 8e Linux LVM Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks. [root@server23 ~]# partx /dev/vdb # 1: 63- 2098655 ( 2098593 sectors, 1074 MB) # 2: 2098656- 4197311 ( 2098656 sectors, 1074 MB) # 3: 0- -1 ( 0 sectors, 0 MB) # 4: 0- -1 ( 0 sectors, 0 MB)

*  建立`LV`设备

[root@server23 ~]# pvcreate /dev/vdb2 
      Physical volume "/dev/vdb2" successfully created
    
    [root@server23 ~]# vgcreate server /dev/vdb2 
      Volume group "server" successfully created
    [root@server23 ~]# vgscan 
      Reading all physical volumes.  This may take a while...
      Found volume group "server" using metadata type lvm2
      Found volume group "VolGroup" using metadata type lvm2
    [root@server23 ~]# lvcreate -l 255 -n iscsi server
      Logical volume "iscsi" created
    [root@server23 ~]# lvscan 
      ACTIVE            '/dev/server/iscsi' [1020.00 MiB] inherit
      ACTIVE            '/dev/VolGroup/lv_root' [7.61 GiB] inherit
      ACTIVE            '/dev/VolGroup/lv_swap' [920.00 MiB] inherit

*  关于共享磁盘的文件写法

iqn.yyyy-mm.<reversed domain name>:identifier
    iqn.年-月.域名的反转写法:共享的target的名称
    例如:
    iqn.2018-03.com.linux:testtarget

*  对于每一个`target`来说可以包含多个磁盘设备,通常称为逻辑单位编号 `LUN [Logical Unit Number]`
 *  来修改主配置文件`/etc/tgt/targets.conf`

<target iqn.2018-03.com.server23:testscsi>
            backing-store /srv/iscsi/disk1.img
            backing-store /dev/vdb1
            backing-store /dev/server/iscsi
    
        initiator-address 172.25.23.0/24
        incominguser westos westos
        write-cache off
    </target>

*  上面这个文件的配置语法

<target iqn.相关设备target的名称> backing-store /虚拟设备的完整路径名称 <=====自动进行LUN编号 LUN1 backing-store /虚拟设备的完整路径名称 <=====自动进行LUN编号 LUN1 backing-store /虚拟设备的完整路径名称 <=====自动进行LUN编号 LUN1 </target>

*  关于这个文件里面一些参数的说明  
    
    
     *  1.`backing-store`和`direct-store`的区别:在设置设备时,如果需要使用整块设备来作为`iscci`就可以使用`direct-store`,否则不建议使用;
     *  2.`initiator-address`:如果需要限制使用`target`的客户端来源,如果不填写,就表示任何用户都可以访问,也可以使用`iptables`来进行限制;
     *  3.`incominguser`:关于用户帐号密码的设置,可以通过帐号和密码来进行访问,后面依次是帐号和密码;
     *  4.`write-cache`:`tgtd`是否使用缓存来加快速度,但是存在数据丢失的风险;
 *  接下来启动`target`并且查看端口以及磁盘信息

[root@server23 ~]# vim /etc/tgt/targets.conf 
    [root@server23 ~]# /etc/init.d/tgtd start
    Starting SCSI target daemon:                               [  OK  ]
    [root@server23 ~]# chkconfig tgtd on 
    如果需要开机自动启动 
    [root@server23 ~]# chkconfig --list tgtd
    tgtd            0:off   1:off   2:on    3:on    4:on    5:on    6:off

*  同时防火墙存在需要添加防火墙例外,防火墙以后会特别说明
 *  查看端口是否正启动

[root@server23 ~]# netstat -tlunp | grep tgt
    tcp        0      0 0.0.0.0:3260                0.0.0.0:*                   LISTEN      1617/tgtd           
    tcp        0      0 :::3260                     :::*                        LISTEN      1617/tgtd

*  使用`使用命令来查看target`信息

[root@server23 ~]# tgt-admin --show
    Target 1: iqn.2018-03.com.server23:testscsi
        System information:
            Driver: iscsi
            State: ready
        I_T nexus information:
        LUN information:
            LUN: 0
                Type: controller
                SCSI ID: IET     00010000
                SCSI SN: beaf10
                Size: 0 MB, Block size: 1
                Online: Yes
                Removable media: No
                Prevent removal: No
                Readonly: No
                Backing store type: null
                Backing store path: None
                Backing store flags: 
            LUN: 1
                Type: disk
                SCSI ID: IET     00010001
                SCSI SN: beaf11
                Size: 1070 MB, Block size: 512
                Online: Yes
                Removable media: No
                Prevent removal: No
                Readonly: No
                Backing store type: rdwr
                Backing store path: /dev/server/iscsi
                Backing store flags: 
            LUN: 2
                Type: disk
                SCSI ID: IET     00010002
                SCSI SN: beaf12
                Size: 1074 MB, Block size: 512
                Online: Yes
                Removable media: No
                Prevent removal: No
                Readonly: No
                Backing store type: rdwr
                Backing store path: /dev/vdb1
                Backing store flags: 
            LUN: 3
                Type: disk
                SCSI ID: IET     00010003
                SCSI SN: beaf13
                Size: 524 MB, Block size: 512
                Online: Yes
                Removable media: No
                Prevent removal: No
                Readonly: No
                Backing store type: rdwr
                Backing store path: /srv/iscsi/disk1.img
                Backing store flags: 
        Account information:
            westos
        ACL information:
            172.25.23.0/24

*  *\* 接下来配置`iscsi-initiator`\**
 *  关于`initiator`的软件架构  
    
    
     *  需要安装的软件`iscsi-initiator-utils.x86_64`  
        
    
    > /etc/iscsi/iscsid.conf： 主配置文件，用于配置连接`iSCSI target`的配置;  
    > /sbin/iscsid: 启动`iSCSI initiator`的主要配置文件程序;  
    > /sbin/iscsiadm: 用来管理`iSCSI initiator`的主要设置程序;  
    > /sbin/iscsid: 让本机成为`iSCSI initiator`的主要服务;

*  首先修改配置文件,并且需要添加密码信息
 *      [root@server22 ~]# vim /etc/iscsi/iscsid.conf
        更改或者添加
        node.session.auth.username = westos
        node.session.auth.password = westos
        discovery.sendtargets.auth.username = westos
        discovery.sendtargets.auth.password = westos
    
    ![这里写图片描述][70]  
    \* 启动服务并且进行连接尝试
    
        [root@server22 ~]# /etc/init.d/iscsi
        iscsi   iscsid  
        [root@server22 ~]# /etc/init.d/iscsi start [root@server22 ~]# iscsiadm -m discovery -t sendtargets -p 172.25.23.23 Starting iscsid: [ OK ] iscsiadm: cannot make connection to 172.25.23.23: No route to host iscsiadm: cannot make connection to 172.25.23.23: No route to host iscsiadm: cannot make connection to 172.25.23.23: No route to host iscsiadm: cannot make connection to 172.25.23.23: No route to host
    
     *  上面的问题多半是因为没有添加防火墙例外,或者网络不同导致的;
     *  查看启动过程中是否存在错误信息
    
        (Jun 06, 2013)
        Mar 11 15:13:54 server22 kernel: iscsi: registered transport (bnx2i)
        Mar 11 15:13:54 server22 kernel: iscsi: registered transport (be2iscsi)
        Mar 11 15:13:54 server22 kernel: In beiscsi_module_init, tt=ffffffffa0461e80
        Mar 11 15:13:54 server22 iscsid: iSCSI logger with pid=1488 started!
        Mar 11 15:13:54 server22 iscsid: iSCSI daemon with pid=1489 started!
    
     *  使用命令探测`target`连接
    
        [root@server22 ~]# iscsiadm -m discovery -t sendtargets -p 172.25.23.23
        172.25.23.23:3260,1 iqn.2018-03.com.server23:testscsi
    
     *  重新启动服务,然后执行登陆操作
    
        [root@server22 ~]# /etc/init.d/iscsi restart
        Stopping iscsi:                                            [  OK  ]
        Starting iscsi:                                            [  OK  ]
        [root@server22 ~]# iscsiadm -m node -T iqn.2018-03.com.server23:testscsi --login
    
     *  使用命令查看新增的设备
    
        [root@server22 ~]# fdisk -l
        
        Disk /dev/vda: 9663 MB, 9663676416 bytes
        16 heads, 63 sectors/track, 18724 cylinders
        Units = cylinders of 1008 * 512 = 516096 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x000c4b5f
        
           Device Boot      Start         End      Blocks   Id  System
        /dev/vda1   *           3        1018      512000   83  Linux
        Partition 1 does not end on cylinder boundary.
        /dev/vda2            1018       18725     8924160   8e  Linux LVM
        Partition 2 does not end on cylinder boundary.
        
        Disk /dev/mapper/VolGroup-lv_root: 8170 MB, 8170504192 bytes
        255 heads, 63 sectors/track, 993 cylinders
        Units = cylinders of 16065 * 512 = 8225280 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x00000000
        
        
        Disk /dev/mapper/VolGroup-lv_swap: 964 MB, 964689920 bytes
        255 heads, 63 sectors/track, 117 cylinders
        Units = cylinders of 16065 * 512 = 8225280 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x00000000
        
        
        Disk /dev/sda: 1069 MB, 1069547520 bytes
        33 heads, 62 sectors/track, 1020 cylinders
        Units = cylinders of 2046 * 512 = 1047552 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x00000000
        
        
        Disk /dev/sdb: 1074 MB, 1074479616 bytes
        34 heads, 61 sectors/track, 1011 cylinders
        Units = cylinders of 2074 * 512 = 1061888 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x00000000
        
        
        Disk /dev/sdc: 524 MB, 524288000 bytes
        17 heads, 59 sectors/track, 1020 cylinders
        Units = cylinders of 1003 * 512 = 513536 bytes
        Sector size (logical/physical): 512 bytes / 512 bytes
        I/O size (minimum/optimal): 512 bytes / 512 bytes
        Disk identifier: 0x00000000
    
     *  通过`iscsi`共享的设备被识别为`/dev/sda`等设备;  
        接下来如果需要进行使用,就可以通过格式化等方式来使用`iscsi`设备
     *  1.需要注意的是提供服务为的主机`target`主机必须开机比`iscsi initiator`开机要早;
     *  **更新添加删除`target`的方法**
     *  可能需要使用以下几个命令
    
        [root@server22 ~]# iscsiadm -m node -T iqn.2018-03.com.server23:testscsi --logout
        Logging out of session [sid: 1, target: iqn.2018-03.com.server23:testscsi, portal: 172.25.23.23,3260]
        Logout of [sid: 1, target: iqn.2018-03.com.server23:testscsi, portal: 172.25.23.23,3260] successful.
        表示退出登陆,但是不会删除/var/lib/iscsi/nodes/iqn.2018-03.com.server23
        [root@server22 ~]# ll /var/lib/iscsi/nodes/iqn.2018-03.com.server23\:testscsi/
        total 4
        drw-------. 2 root root 4096 3月  11 15:15 172.25.23.23,3260,1
        删除|添加|更新target的方法
        [root@server22 ~]# iscsiadm -m node -o [delete|new|update] -T targetname
    
     *  如果执行了删除操作,上面的文件就不存在了
    
        [root@server22 ~]# iscsiadm -m node -o delete -T iqn.2018-03.com.server23:testscsi
        [root@server22 ~]# iscsiadm -m node
        iscsiadm: No records found
    
     *  暂时没有遇到比较大的错误,就不进行错误的总结了;

[70]: /images/20220530/35999c171dd44e5d93cd760d9f4934d3.png