P460阵列卡 raid扩容

发布于 2022-10-14  1.21k 次阅读


一、lvm 数据盘raid 5磁盘扩容

1.1 安装Arcconf 以及lspci命令所需的软件包

将Arcconf-3.00-23488.x86_64.rpm 下载或者复制到 /tmp 下
通过 lspci命令:列出所有的PCI设备信息
lspci命令属于pciutils工具包
执行yum install whatprovides */lspci 或者直接yum -y install pciutils 安装

1.2 查看裸金属的磁盘阵列信息

arcconf getconfig 1 ld (查看控制器 1的磁盘阵列信息)

[root@localhost ~]# arcconf getconfig 1 ld
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical Device number 0
   Logical Device name                      : Logical Drive 1
   Disk Name                                : /dev/sdb
   Block Size of member drives              : 512 Bytes
   Array                                    : 0
   RAID level                               : 1
   Status of Logical Device                 : Optimal
   Size                                     : 457830 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 256 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : /boot/efi 500 MB  Partition Number 1 /boot 1024 MB  Partition Number 2
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001C2C9E041A27F27AC60F24
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 0                                 : Present (457862MB, SATA, SSD, Channel:0, Device:0) S45PNA0NC00444    
   Device 1                                 : Present (457862MB, SATA, SSD, Channel:0, Device:1) S45PNA0NC00442    


Logical Device number 1
   Logical Device name                      : Logical Drive 2
   Disk Name                                : /dev/sdc
   Block Size of member drives              : 512 Bytes
   Array                                    : 1
   RAID level                               : 5
   Status of Logical Device                 : Optimal
   Parity Initialization Status             : Completed
   Size                                     : 1831366 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 512 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : Not Applicable
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001CA58EA2ACC77C0FABB034
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 3                                 : Present (915715MB, SATA, SSD, Channel:0, Device:3)         1933235A894B
   Device 4                                 : Present (915715MB, SATA, SSD, Channel:0, Device:4)         19332359F8B3
   Device 5                                 : Present (915715MB, SATA, SSD, Channel:0, Device:5)         19332359FB98




Command completed successfully.
[root@localhost ~]#

记录下SN 信息,以及device的序号
以上截图中显示该机器下有两个磁盘阵列分别是:
device0和device1组成的raid 1,对应分区是/dev/sdbDevice3、device4、device5组成的raid 5,对应分区是/dev/sdc

1.3 查看服务器上所有物理盘信息

arcconf getconfig 1 pd(查控制器1上连接sata盘的信息,可以确认sata盘槽位)

[root@localhost ~]# arcconf getconfig 1 pd
..................................................
 Device #6
         Device is a Hard drive
         State                              : Ready
         Drive has stale RIS data           : False
         Disk Name                          : /dev/sda
         Block Size                         : 512 Bytes
         Physical Block Size                : 4K Bytes
         Transfer Speed                     : SATA 6.0 Gb/s
         Reported Channel,Device(T:L)       : 0,6(6:0)
         Reported Location                  : Enclosure Direct Attached, Slot 6(Connector 1:CN1)
         Vendor                             : ATA
         Model                              : Micron_5200_MTFD
         Firmware                           :  D1MH027
         Serial number                      : 19332359F39D
         World-wide name                    : 38465695EB81E001
         Reserved Size                      : 32768 KB
         Used Size                          : 0 MB
         Unused Size                        : 915683 MB
         Total Size                         : 915715 MB
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         SSD                                : Yes
         NCQ supported                      : Supported
         NCQ status                         : Enabled
         Boot Type                          : None
         Current Temperature                : 28 deg C
         Maximum Temperature                : 41 deg C
         Threshold Temperature              : 60 deg C
         PHY Count                          : 1
         Drive Configuration Type           : Unassigned
         Mount Point(s)                     : Not Mounted
         Drive Exposed to OS                : True
         Sanitize Erase Support             : True
         Sanitize Lock Freeze Support       : True
         Sanitize Lock Anti-Freeze Support  : True
         Sanitize Lock Setting              : None
         Usage Remaining                    : 100 percent
         Estimated Life Remaining           : Not Applicable
         SSD Smart Trip Wearout             : False
         56 Day Warning Present             : False
         Drive Unique ID                    : C30143D16E95C57EBAE00891528B0285
         Last Failure Reason                : No Failure
      ----------------------------------------------------------------
      Device Phy Information
      ----------------------------------------------------------------
         Phy #0
            Negotiated Physical Link Rate   : 6 Gbps
            Negotiated Logical Link Rate    : 6 Gbps
            Maximum Link Rate               : 6 Gbps

      ----------------------------------------------------------------
      Device Error Counters
      ----------------------------------------------------------------
         Aborted Commands                   : 0
         Bad Target Errors                  : 0
         Ecc Recovered Read Errors          : 0
         Failed Read Recovers               : 0
         Failed Write Recovers              : 0
         Format Errors                      : 0
         Hardware Errors                    : 0
         Hard Read Errors                   : 0
         Hard Write Errors                  : 0
         Hot Plug Count                     : 0
         Media Failures                     : 0
         Not Ready Errors                   : 0
         Other Time Out Errors              : 0
         Predictive Failures                : 0
         Retry Recovered Read Errors        : 0
         Retry Recovered Write Errors       : 0
         Scsi Bus Faults                    : 0
         Sectors Reads                      : 1249537714
         Sectors Written                    : 1148083105
         Service Hours                      : 211

   Channel #2:
      Device #2
         Device is an Enclosure Services Device
         Reported Channel,Device(T:L)       : 2,2(2:0)
         Enclosure ID                       : 0
         Enclosure Logical Identifier       : 58465695EB81E008
         Type                               : SES2
         Vendor                             : UN
         Model                              : Virtual SGPIO
         Firmware                           : 3.21
         Status of Enclosure Services Device
            Speaker status                  : Not Available


Command completed successfully.

通过arcconf getconfig 1 pd 中显示device 信息列表,device 6是新加入磁盘未加入磁盘阵列,记录下槽位Reported Channel,Device(T:L): 0,6(6:0)

1.4 磁盘扩容到阵列中

在/data 中存放一些文件并记录其MD5值

将磁盘扩容到阵列里:
arcconf modify 1 array 1 expand 0 6
array 1:的1是阵列号  磁盘加在哪个raid里就写对应raid的Array,这里是raid5 对应Array1
arcconf modify 1:的1为controllers found 显示的阵列卡控制器号 咱们服务器只有一个控制器 显示的是1
expand 0 6 是加入磁盘槽位信息

将磁盘扩容到阵列命令,硬盘容量越大,这一步操作的时间就会越长,可以通过  arcconf getconfig 1 ld 查看同步信息:

[root@localhost ~]# arcconf getconfig 1 ld
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical Device number 0
   Logical Device name                      : Logical Drive 1
   Disk Name                                : /dev/sdb
   Block Size of member drives              : 512 Bytes
   Array                                    : 0
   RAID level                               : 1
   Status of Logical Device                 : Optimal
   Size                                     : 457830 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 256 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : /boot/efi 500 MB  Partition Number 1 /boot 1024 MB  Partition Number 2
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001C2C9E041A27F27AC60F24
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 0                                 : Present (457862MB, SATA, SSD, Channel:0, Device:0) S45PNA0NC00444    
   Device 1                                 : Present (457862MB, SATA, SSD, Channel:0, Device:1) S45PNA0NC00442    


Logical Device number 1
   Logical Device name                      : Logical Drive 2
   Disk Name                                : /dev/sdc
   Block Size of member drives              : 512 Bytes
   Array                                    : 1
   RAID level                               : 5
   Status of Logical Device                 : Logical device Reconfiguring(Expanding : 22 %)
   Parity Initialization Status             : Completed
   Size                                     : 1831366 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 512 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : Not Applicable
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001CA58EA2ACC77C0FABB034
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 3                                 : Present (915715MB, SATA, SSD, Channel:0, Device:3)         1933235A894B
   Device 4                                 : Present (915715MB, SATA, SSD, Channel:0, Device:4)         19332359F8B3
   Device 5                                 : Present (915715MB, SATA, SSD, Channel:0, Device:5)         19332359FB98




Command completed successfully.
[root@localhost ~]#

Parity Initialization Status 是Optimal 表示成功加入阵列中,目前960GB SATA SSD,也就是平均1TG磁盘大概要2-3小时

1.5 把磁盘阵列扩容到最大空间

arcconf modify 1 from 1 to max 5 0 3 0 4 0 5 0 6
表示:将控制器1(modify 1)的逻辑卷1(from 1)扩到最大空间,RAID级别是5(max 5),device3-6 的槽位是0 3 0 4 0 5 0 6  

通过arcconf getconfig 1 ld 可以看到新的磁盘已经扩容到 /dev/sdc 上,这时候0% 是扩容后,数据均分布过程

1.6 扩容磁盘空间

将sdc 未使用空间扩容到/dev/centos_data/data 上
pvresize /dev/sdc
lvextend -l +100%FREE /dev/centos_data/data
xfs_growfs /dev/centos_data/data

通过比较扩容前后数据md5值可以看出扩容并没有影响数据

二、标准格式数据盘raid 5磁盘扩容

2.1 安装Arcconf 以及lspci命令所需的软件包

将Arcconf-3.00-23488.x86_64.rpm 下载或者复制到 /tmp 下

通过 lspci命令:列出所有的PCI设备信息
lspci命令属于pciutils工具包
执行yum install whatprovides */lspci 或者直接yum -y install pciutils 安装

2.2 查看裸金属的磁盘阵列信息

arcconf getconfig 1 ld (查看控制器 1的磁盘阵列信息)

[root@localhost ~]# arcconf getconfig 1 ld
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical Device number 0
   Logical Device name                      : Logical Drive 1
   Disk Name                                : /dev/sdb
   Block Size of member drives              : 512 Bytes
   Array                                    : 0
   RAID level                               : 1
   Status of Logical Device                 : Optimal
   Size                                     : 457830 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 256 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : /boot/efi 500 MB  Partition Number 1 /boot 1024 MB  Partition Number 2
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001C2C9E041A27F27AC60F24
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 0                                 : Present (457862MB, SATA, SSD, Channel:0, Device:0) S45PNA0NC00444    
   Device 1                                 : Present (457862MB, SATA, SSD, Channel:0, Device:1) S45PNA0NC00442    


Logical Device number 1
   Logical Device name                      : Logical Drive 2
   Disk Name                                : /dev/sdc
   Block Size of member drives              : 512 Bytes
   Array                                    : 1
   RAID level                               : 5
   Status of Logical Device                 : Optimal
   Parity Initialization Status             : Completed
   Size                                     : 1831366 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 512 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : Not Applicable
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001CA58EA2ACC77C0FABB034
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 3                                 : Present (915715MB, SATA, SSD, Channel:0, Device:3)         1933235A894B
   Device 4                                 : Present (915715MB, SATA, SSD, Channel:0, Device:4)         19332359F8B3
   Device 5                                 : Present (915715MB, SATA, SSD, Channel:0, Device:5)         19332359FB98




Command completed successfully.
[root@localhost ~]#

记录下SN 信息,以及device的序号
以上截图中显示该机器下有两个磁盘阵列分别是:
device0和device1组成的raid 1,对应分区是/dev/sdbDevice3、device4、device5组成的raid 5,对应分区是/dev/sdc

2.3 查看服务器上所有物理盘信息

arcconf getconfig 1 pd(查控制器1上连接sata盘的信息,可以确认sata盘槽位)

[root@localhost ~]# arcconf getconfig 1 pd
..................................................
 Device #6
         Device is a Hard drive
         State                              : Ready
         Drive has stale RIS data           : False
         Disk Name                          : /dev/sda
         Block Size                         : 512 Bytes
         Physical Block Size                : 4K Bytes
         Transfer Speed                     : SATA 6.0 Gb/s
         Reported Channel,Device(T:L)       : 0,6(6:0)
         Reported Location                  : Enclosure Direct Attached, Slot 6(Connector 1:CN1)
         Vendor                             : ATA
         Model                              : Micron_5200_MTFD
         Firmware                           :  D1MH027
         Serial number                      : 19332359F39D
         World-wide name                    : 38465695EB81E001
         Reserved Size                      : 32768 KB
         Used Size                          : 0 MB
         Unused Size                        : 915683 MB
         Total Size                         : 915715 MB
         S.M.A.R.T.                         : No
         S.M.A.R.T. warnings                : 0
         SSD                                : Yes
         NCQ supported                      : Supported
         NCQ status                         : Enabled
         Boot Type                          : None
         Current Temperature                : 28 deg C
         Maximum Temperature                : 41 deg C
         Threshold Temperature              : 60 deg C
         PHY Count                          : 1
         Drive Configuration Type           : Unassigned
         Mount Point(s)                     : Not Mounted
         Drive Exposed to OS                : True
         Sanitize Erase Support             : True
         Sanitize Lock Freeze Support       : True
         Sanitize Lock Anti-Freeze Support  : True
         Sanitize Lock Setting              : None
         Usage Remaining                    : 100 percent
         Estimated Life Remaining           : Not Applicable
         SSD Smart Trip Wearout             : False
         56 Day Warning Present             : False
         Drive Unique ID                    : C30143D16E95C57EBAE00891528B0285
         Last Failure Reason                : No Failure
      ----------------------------------------------------------------
      Device Phy Information
      ----------------------------------------------------------------
         Phy #0
            Negotiated Physical Link Rate   : 6 Gbps
            Negotiated Logical Link Rate    : 6 Gbps
            Maximum Link Rate               : 6 Gbps

      ----------------------------------------------------------------
      Device Error Counters
      ----------------------------------------------------------------
         Aborted Commands                   : 0
         Bad Target Errors                  : 0
         Ecc Recovered Read Errors          : 0
         Failed Read Recovers               : 0
         Failed Write Recovers              : 0
         Format Errors                      : 0
         Hardware Errors                    : 0
         Hard Read Errors                   : 0
         Hard Write Errors                  : 0
         Hot Plug Count                     : 0
         Media Failures                     : 0
         Not Ready Errors                   : 0
         Other Time Out Errors              : 0
         Predictive Failures                : 0
         Retry Recovered Read Errors        : 0
         Retry Recovered Write Errors       : 0
         Scsi Bus Faults                    : 0
         Sectors Reads                      : 1249537714
         Sectors Written                    : 1148083105
         Service Hours                      : 211

   Channel #2:
      Device #2
         Device is an Enclosure Services Device
         Reported Channel,Device(T:L)       : 2,2(2:0)
         Enclosure ID                       : 0
         Enclosure Logical Identifier       : 58465695EB81E008
         Type                               : SES2
         Vendor                             : UN
         Model                              : Virtual SGPIO
         Firmware                           : 3.21
         Status of Enclosure Services Device
            Speaker status                  : Not Available


Command completed successfully.

通过arcconf getconfig 1 pd 中显示device 信息列表,device 6是新加入磁盘未加入磁盘阵列,记录下槽位Reported Channel,Device(T:L): 0,6(6:0)

2.4 磁盘扩容到阵列中

在/data 中存放一些文件并记录其MD5值

将磁盘扩容到阵列里:
arcconf modify 1 array 1 expand 0 6
array 1:的1是阵列号  磁盘加在哪个raid里就写对应raid的Array,这里是raid5 对应Array1
arcconf modify 1:的1为controllers found 显示的阵列卡控制器号 咱们服务器只有一个控制器 显示的是1
expand 0 6 是加入磁盘槽位信息

将磁盘扩容到阵列命令,硬盘容量越大,这一步操作的时间就会越长,可以通过  arcconf getconfig 1 ld 查看同步信息:

[root@localhost ~]# arcconf getconfig 1 ld
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical Device number 0
   Logical Device name                      : Logical Drive 1
   Disk Name                                : /dev/sdb
   Block Size of member drives              : 512 Bytes
   Array                                    : 0
   RAID level                               : 1
   Status of Logical Device                 : Optimal
   Size                                     : 457830 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 256 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : /boot/efi 500 MB  Partition Number 1 /boot 1024 MB  Partition Number 2
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001C2C9E041A27F27AC60F24
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 0                                 : Present (457862MB, SATA, SSD, Channel:0, Device:0) S45PNA0NC00444    
   Device 1                                 : Present (457862MB, SATA, SSD, Channel:0, Device:1) S45PNA0NC00442    


Logical Device number 1
   Logical Device name                      : Logical Drive 2
   Disk Name                                : /dev/sdc
   Block Size of member drives              : 512 Bytes
   Array                                    : 1
   RAID level                               : 5
   Status of Logical Device                 : Logical device Reconfiguring(Expanding : 22 %)
   Parity Initialization Status             : Completed
   Size                                     : 1831366 MB
   Stripe-unit size                         : 256 KB
   Full Stripe Size                         : 512 KB
   Interface Type                           : SATA SSD
   Device Type                              : Data
   Boot Type                                : None
   Heads                                    : 255
   Sectors Per Track                        : 32
   Cylinders                                : 65535
   Caching                                  : Disabled
   Mount Points                             : Not Applicable
   LD Acceleration Method                   : SSD IO Bypass
   Volume Unique Identifier                 : 600508B1001CA58EA2ACC77C0FABB034
   --------------------------------------------------------
   Array Physical Device Information
   --------------------------------------------------------
   Device 3                                 : Present (915715MB, SATA, SSD, Channel:0, Device:3)         1933235A894B
   Device 4                                 : Present (915715MB, SATA, SSD, Channel:0, Device:4)         19332359F8B3
   Device 5                                 : Present (915715MB, SATA, SSD, Channel:0, Device:5)         19332359FB98




Command completed successfully.
[root@localhost ~]#

Parity Initialization Status 是Optimal 表示成功加入阵列中,目前960GB SATA SSD,也就是平均1TG磁盘大概要2-3小时

2.5 把磁盘阵列扩容到最大空间

arcconf modify 1 from 1 to max 5 0 3 0 4 0 5 0 6
表示:将控制器1(modify 1)的逻辑卷1(from 1)扩到最大空间,RAID级别是5(max 5),device3-6 的槽位是0 3 0 4 0 5 0 6  

通过arcconf getconfig 1 ld 可以看到新的磁盘已经扩容到 /dev/sdc 上,这时候0% 是扩容后,数据均分布过程

通过partprobe 无法刷新分区,需要重启服务器来刷新分区

2.6 扩容磁盘空间

yum -y install cloud-utils-growpart 使用growpart 命令
growpart /dev/sdb 1 

xfs_growfs /dev/sdb1

查看扩容后文件得md5值和扩容前一致