Dell服務器RAID卡介紹及監控
本文包含如下內容
一、Dell服務器 RAID卡介紹
二、陣列卡的Stripe size介紹
三、megacli介紹、安裝、使用、crontab監控腳本
四、查看SAS 6/iR卡的信息
五、DELL服務器的各種RAID卡的詳細參數
一、Dell服務器 RAID卡介紹
以下在系統下使用相關命令得到的數據，dell支持的各種RAID卡的詳細參數參見本文第五部分。
1、內部RAID控制器
使用megacli命令查看得到的RAID卡數據：
SAS 6/iR ：只支持2塊硬盤做 raid0，或者raid1，或者不做陣列卡 (Dell硬件聊天室獲得)
PERC 6/i Integrated
Memory Size: 256MB
RAID Level Supported: RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported
Supported Drives: SAS, SATA
Min Stripe Size: 8 KB
Max Stripe Size: 1.0 MB
接口速度Link Speed: 3.0Gb/s
PERC H700 Integrated
Memory Size: 512MB
RAID Level Supported: RAID0, RAID1, RAID5, RAID6, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning
Supported Drives: SAS, SATA
Min Stripe Size: 8 KB
Max Stripe Size: 1.0 MB
接口速度Link Speed: 6.0Gb/s
官方資料：
PERC H200（6 Gb/秒）
PERC H700（6Gb /秒），配備512 MB非易失性高速緩存
PERC H700（6 Gb/秒），配備512 MB電池后備高速緩存；512 MB、1 G非易失性電池后備高速緩存
PERC 6/i，配備256 MB電池后備高速緩存
2、外部RAID控制器
官方資料：
PERC H800（6Gb/秒），配備512 MB非易失性高速緩存
PERC H800（6Gb/秒），配備512 MB電池后備高速緩存；512 MB、1 G非易失性電池后備高速緩存
PERC 6/E，配備256 MB或512 MB電池后備高速緩存
二、陣列卡的Stripe size介紹
Stripe size 默認64KB。
1、Stripe size百科介紹
在每個磁盤上連續寫入數據的總量，也稱作“條帶深度”。你可以指定每個邏輯驅動器的條帶容量從2KB，4KB，8KB一直到128KB。為了獲得更高的性能，要選擇條帶的容量等於或小於操作系統的簇的大小。大容量的條帶會產生更高的讀取性能，尤其在讀取連續數據的時候。而讀取隨機數據的時候，最好設定條帶的容量小一點。如果指定128KB的條帶將需要8MB內存。
不同Stripe size的選擇直接影響性能，如IOPS和吞吐量。
Stripe size值小，通過多塊磁盤響應多個I/O請求，可以增加I/O訪問速率（IOPS）
Stripe size值大，通過多塊磁盤響應一個I/O請求，可以增加數據傳輸速率（Mbps）
我們知道順序和隨機對於帶寬和iops的需求成反比，即順序的讀寫比較消耗帶寬，隨機的讀寫比較消耗iops；
2、Stripe size一般設置
通常，小Stripe size適用於數據庫，一般Stripe size適用於文件服務器，大Stripe size適用於多煤體應用。實際應用stripe size大小應在效率測試后具體設置。
數據庫一般4-16KB
web服務器默認即可，16-64KB
流媒體等大文件建議128KB以上
三、megacli介紹、安裝、使用、crontab監控腳本
查看RAID信息的工具有mpt-status、megarc、dell OpenManage。經測試只有megacli可以實現目標，dell OpenManage由於較大90M左右沒有安裝。
megacli工具非常強大，不僅能查看RAID卡信息，而且還能設置RAID卡，但是不支持SAS 6/iR的RAID卡，不推薦采用megacli在線設置RAID卡，本文也僅用於監控掛接在RAID上的硬盤狀態。
1、查看raid卡的型號：
root@3:~# lspci |grep RAID H700
02:00.0 RAID bus controller: LSI Logic / Symbios Logic Device 0079 (rev 04)
root@3:~# lspci |grep RAID PERC 6/i
03:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 04)
root@k:~# lspci|grep SAS SAS 6/iR
03:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
2、安裝megacli
root@3:~# more /etc/apt/sources.list
增加如下源：
##add monitor dell perc 6/i raidcontroller's source
deb http://hwraid.le-vert.net/debian lenny main
root@3:~# apt-get update
root@3:~# apt-get install megacli
3、使用megacli查看RAID卡信息
術語：BBU 電池備份單元(Battery Backup Unit)
常用查看命令：
megacli -LDInfo -Lall -aALL 查raid級別
megacli -AdpAllInfo -aALL 查raid卡信息
megacli -PDList -aALL 查看硬盤信息
megacli -AdpBbuCmd -aAll 查看電池信息
megacli -FwTermLog -Dsply -aALL 查看raid卡日志
megacli -adpCount 顯示適配器個數
megacli -AdpGetTime –aALL 顯示適配器時間
megacli -AdpAllInfo -aAll 顯示所有適配器信息
megacli -LDInfo -LALL -aAll 顯示所有邏輯磁盤組信息
megacli -PDList -aAll 顯示所有的物理信息
megacli -AdpBbuCmd -GetBbuStatus -aALL |grep ‘Charger Status’ 查看充電狀態
megacli -AdpBbuCmd -GetBbuStatus -aALL 顯示BBU狀態信息
megacli -AdpBbuCmd -GetBbuCapacityInfo -aALL 顯示BBU容量信息
megacli -AdpBbuCmd -GetBbuDesignInfo -aALL 顯示BBU設計參數
megacli -AdpBbuCmd -GetBbuProperties -aALL 顯示當前BBU屬性
megacli -cfgdsply -aALL 顯示Raid卡型號，Raid設置，Disk相關信息
4、使用crontab定時監控硬盤狀態的小腳本
該腳本實現在監控到硬盤故障時發出報警郵件，並關聯到手機短信，以便通知運維人員及時處理。
root@3:~# more /backup/autobackup/autoshell/check_raid_disk_health
#!/bin/sh
log_dir=/backup/autobackup/logsum/
log_name=_raid_disk_monitor
logtime=$(date +%Y%m%d --date='1 days ago')
fix=.log
host=_`hostname`
STATUS=0
echo "Checking RAID status on $host" >$log_dir$logtime$log_name$host$fix
RAID_Contrller=`megacli -AdpAllInfo -aALL |grep "Product Name" | cut -d: -f2`
echo "Controller : $RAID_Contrller" >>$log_dir$logtime$log_name$host$fix
Online_disk_num=`megacli -PDList -aALL | grep Online | wc -l`
echo "Totall number of Physical disks online : $Online_disk_num" >>$log_dir$logtime$log_name$host$fix
Degrade_disk=`megacli -AdpAllInfo -a0 |grep "Degrade"`
echo $Degrade_disk >>$log_dir$logtime$log_name$host$fix
Degrade_disk_num=`echo $Degrade_disk |cut -d" " -f3`
Failed_disk=`megacli -AdpAllInfo -a0 |grep "Failed Disks"`
echo $Failed_disk >>$log_dir$logtime$log_name$host$fix
Failed_disk_num=`echo $Failed_disk |cut -d" " -f4`
##將raid_disk_monitor.log scp到mail上
scp -P120 $log_dir$logtime$log_name$host$fix lai@60.*.*.*:/data/autobackup/logsum
ssh -p120 lai@60.1.1.1 "cat /data/autobackup/logsum//$(date +%Y%m%d --date='1 days ago')_raid*| mail -s "`echo "
/$(date +%Y'年'%m'月'%d'日' --date='1 days ago')ERROR:tv233_RAID卡硬盤報警"|iconv -f utf-8 -t gbk`" postmaster@126.co
m,1860531****@wo.com.cn -- -f www@xp.com"
[ "$Degrade_disk_num" -ne 0 ] && STATUS=1
[ "$Failed_disk_num" -ne 0 ] && STATUS=1
exit $STATUS
四、查看SAS 6/iR卡的信息
因為megacli不支持SAS 6/iR卡，使用一款叫做lsiutil的綠色免安裝工具來查看SAS 6/iR卡的信息。
1、下載lsiutil
官網下載地址ftp://ftp.lsil.com/HostAdapterDrivers/linux/lsiutil/lsiutil.tar.gz
2、解壓運行lsiutil
直接解壓,運行目錄里面的lsiutil 即可.
[root@k]# tar -zxvf lsiutil.tar.gz
[root@k]# cd lsiutil
[root@k]# ./lsiutil 【交互式運行】
LSI Logic MPT Configuration Utility, Version 1.56, March 19, 2008
1 MPT Port found
Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev IOC
1. /proc/mpt/ioc0 LSI Logic SAS1068E B3 105 00192f00 0
Select a device: [1-1 or 0 to quit] 1
1. Identify firmware, BIOS, and/or FCode
2. Download firmware (update the FLASH)
4. Download/erase BIOS and/or FCode (update the FLASH)
8. Scan for devices
10. Change IOC settings (interrupt coalescing)
13. Change SAS IO Unit settings
16. Display attached devices
20. Diagnostics
21. RAID actions
22. Reset bus
23. Reset target
42. Display operating system names for devices
45. Concatenate SAS firmware and NVDATA files
60. Show non-default settings
61. Restore default settings
69. Show board manufacturing information
97. Reset SAS link, HARD RESET
98. Reset SAS link
99. Reset port
e Enable expert mode in menus
p Enable paged mode
w Enable logging
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 69 --- 查看Raid 卡的信息板子名稱【SAS6IR】
Seg/Bus/Dev/Fun Board Name Board Assembly Board Tracer
0 2 0 0 SAS6IR
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 8 ---掃描硬盤設備並打印設備狀態
SAS1068E's links are 3.0 G, 3.0 G, down, down, off, off, off, off
B___T___L Type Vendor Product Rev SASAddress PhyNum
0 0 0 Disk Dell VIRTUAL DISK 1028
0 8 0 EnclServ DP BACKPLANE 1.07 5942b0b02dcf1a00 8
Hidden RAID Devices:
B___T Device Vendor Product Rev SASAddress PhyNum
0 1 PhysDisk 0 SEAGATE ST3600057SS ES62 5000c50023c4b95d 1
0 9 PhysDisk 1 SEAGATE ST3600057SS ES62 5000c50023c4bd95 0
【查看Raid 狀態】: 進入21 選項:
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 21
1. Show volumes
2. Show physical disks
3. Get volume state
4. Wait for volume resync to complete
23. Replace physical disk
26. Disable drive firmware update mode
27. Enable drive firmware update mode
30. Create volume
31. Delete volume
32. Change volume settings
33. Change volume name
50. Create hot spare
99. Reset port
e Enable expert mode in menus
p Enable paged mode
w Enable logging
RAID actions menu, select an option: [1-99 or e/p/w or 0 to quit] 2 ---查看設備在線狀態
1 volume is active, 2 physical disks are active
PhysDisk 0 is Bus 0 Target 1
PhysDisk State: online
PhysDisk Size 572325 MB, Inquiry Data: SEAGATE ST3600057SS ES62
Warning, page length in reply does not match page length in buffer
(9 vs. 15)
PhysDisk 1 is Bus 0 Target 9
PhysDisk State: online
PhysDisk Size 572325 MB, Inquiry Data: SEAGATE ST3600057SS ES62
Warning, page length in reply does not match page length in buffer
(9 vs. 15)
注意，因為我的兩塊硬盤做得是Raid 0 ，如果你的磁盤做得是Raid 1 可以通過 【4. Wait for volume resync to complete】兩塊硬盤的同步
狀態
RAID actions menu, select an option: [1-99 or e/p/w or 0 to quit] 4 ---我的是Raid 0
Volume 0 State: optimal, enabled
以下幾個菜單可以在線創建陣列！
30. Create volume
31. Delete volume
32. Change volume settings
33. Change volume name
50. Create hot spare
#注意以上所有的退出使用阿拉數字零
3、非交互方式查看SAS 6/iR卡信息
如果你覺得上面的菜單選項比較麻煩，你也可以使用命令行模式完成！如下:
lsiutil –p1 –a 8,21,4,0,0
-p1 指1 MPT Port found
-a 指你選擇的菜單號,並使用逗號分開.
0表示退出參數------即進入幾個菜單，就是用幾個0 做退出！
[root@k]# ./lsiutil -p1 -a 8,21,4,0,0
LSI Logic MPT Configuration Utility, Version 1.56, March 19, 2008
1 MPT Port found
Port Name Chip Vendor/Type/Rev MPT Rev Firmware Rev IOC
1. /proc/mpt/ioc0 LSI Logic SAS1068E B3 105 00192f00 0
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 8
SAS1068E's links are 3.0 G, 3.0 G, down, down, off, off, off, off
B___T___L Type Vendor Product Rev SASAddress PhyNum
0 0 0 Disk Dell VIRTUAL DISK 1028
0 8 0 EnclServ DP BACKPLANE 1.07 5942b0b02dcf1a00 8
Hidden RAID Devices:
B___T Device Vendor Product Rev SASAddress PhyNum
0 1 PhysDisk 0 SEAGATE ST3600057SS ES62 5000c50023c4b95d 1
0 9 PhysDisk 1 SEAGATE ST3600057SS ES62 5000c50023c4bd95 0
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 21
RAID actions menu, select an option: [1-99 or e/p/w or 0 to quit] 4
Volume 0 State: optimal, enabled
RAID actions menu, select an option: [1-99 or e/p/w or 0 to quit] 0
Main menu, select an option: [1-99 or e/p/w or 0 to quit] 0
[root@k]#
五、DELL服務器的各種RAID卡的詳細參數
1、SAS 6/iR Features
This section provides the specifications of Dell邃｢ Serial-Attached SCSI (SAS) 6/iR controller. The following table compares the specifications of the SAS 6/iR Adapter and SAS 6/iR Integrated.
Table 3-1. Specifications of SAS 6/iR
| Specification |
SAS 6/iR Adapter |
SAS 6/iR Integrated |
| SAS technology |
Yes |
Yes |
| Support for x4, x8, or x16 PCI Express Host Interface |
Yes |
Yes |
| Form Factor |
Standard-Height, Half-Length PCI Adapter |
Standard-Height, Half-Length PCI on all systems except on blade servers (where the dimension do not follow industry standards) and some Precision workstations (where the controller has been integrated on the mother board) |
| I/O controller (IOC) |
LSI SAS 1068e |
LSI SAS 1068e |
| Core Speed: 255 MHz |
Core Speed: 255 MHz |
|
| Operating voltage requirements |
+12V, +3.3V, +3.3Vaux |
+12V, +3.3V, +3.3Vaux |
| Communication to the system |
PCI-E lanes |
System dependent |
| Communication to end devices |
SAS Links |
SAS Links |
| SAS Connectors |
2 x4 Internal |
2 x4 Internal connectors on all systems, with two exceptions: 4 x1 on Precision workstations with controller integrated on the motherboard, no SAS connector on the controller for the blades (I/O is routed though the PCI connector) |
| Lead Free |
Yes |
Yes |
| Supported operating systems |
Microsoftﾂｮ Windows Serverﾂｮ2003 Family, Windowsﾂｮ XP, Microsoft Windows Server 2008 Family, Windows Vista邃｢, Red Hatﾂｮ Enterprise Linuxﾂｮ Versions 4 and 5, SUSEﾂｮ Linux Enterprise Server Version 10. |
|
| Dell-compliant SAS and SATA compatibility |
Yes |
Yes |
| Dell supported direct connected end devices |
Dell-compliant physical disks |
Dell-compliant physical disks |
| SMART error support through management applications |
Yes |
Yes |
| Backplane supported systems |
Yes |
Yes |
| Hardware-based RAID |
RAID 0, RAID 1 |
RAID 0, RAID 1 |
| Maximum number of virtual disks |
2 |
2 |
| Storage management software |
OpenManage邃｢ Storage Services, SAS RAID Storage Manager |
OpenManage Storage Services, SAS RAID Storage Manager |
| NOTE:The management software that is supported depends on the specific platform. |
||
| Support for internal tape drive |
No |
No |
| Support for Global Hotspare |
Yes |
Yes |
| Maximum number of Hotspares |
2 |
2 |
2、Dell所有RAID參數對比表
DellTM PERC Products
| Model |
Interface Support |
PCI Support |
SAS Connectors |
Cache Memory Size |
Write Back Cache |
RAID Levels |
Max Drive Support |
RAID Support |
|
| PERC H800 Adapter |
6Gb/s SAS |
PCI-Express 2.0 |
2x4 external |
512MB |
Yes (TBBU) |
0,1,5,6,10,50,60 |
192 |
Hardware RAID |
|
| PERC H700 Integrated / Adapter |
6Gb/s SAS |
PCI-Express 2.0 |
2x4 internal |
512MB |
Yes (BBU) |
0,1,5,6,10,50,60 |
16 |
Hardware RAID |
|
| PERC H700 Modular |
6Gb/s SAS |
PCI-Express 2.0 |
1x4 internal |
512MB |
Yes (BBU) |
0,1,5,6,10 |
4 |
Hardware RAID |
|
| PERC H200 Integrated / Adapter |
6Gb/s SAS |
PCI-Express 2.0 |
2x4 internal |
- |
- |
0,1,10,supports Non-RAID |
16 |
Hardware RAID |
|
| PERC H200 Modular |
6Gb/s SAS |
PCI-Express 2.0 |
1x4 internal |
- |
- |
0,1,10,supports Non-RAID |
4 |
Hardware RAID |
|
| 6Gbps SAS HBA |
6Gb/s SAS |
PCI-Express 2.0 |
2x4 External |
- |
- |
- |
- |
HBA Hardware |
|
| PERC 6/E Adapter |
3Gb/s SAS |
PCI-Express 1.0 |
2x4 external |
512MB |
Yes (TBBU) |
0,1,5,6,10,50,60 |
144 |
Hardware RAID |
|
| PERC 6/I Integrated / Adapter |
3Gb/s SAS |
PCI-Express 1.0 |
2x4 internal |
256MB |
Yes (BBU) |
0,1,5,6,10,50,60 |
16 |
Hardware RAID |
|
| PERC 6/I Modular |
3Gb/s SAS |
PCI-Express 1.0 |
1x4 internal |
256MB |
Yes (BBU) |
0,1,5,6,10 |
4 |
Hardware RAID |
|
| CERC 6/I Modular |
3Gb/s SAS |
PCI-Express 1.0 |
1x4 internal |
128MB |
- |
0,1,5,6,10 |
4 |
Hardware RAID |
|
| SAS 6/IR Integrated / Adapter |
3Gb/s SAS |
PCI-Express 1.0 |
2x4 internal |
- |
- |
0,1 |
8 |
Hardware RAID |
|
| SAS 6/IR Modular |
3Gb/s SAS |
PCI-Express 1.0 |
1x4 internal |
- |
- |
0,1 |
4 |
Hardware RAID |
|
| PERC S300 |
3Gb/s SAS |
PCI-Express 1.0 |
2x4 internal |
- |
- |
0,1,10,5 |
8 |
Software RAID on HBA |
|
| PERC S100 |
3Gb/s SAS |
- |
4 cabled SATA |
- |
- |
0,1,10,5 |
4 |
Software RAID on SATA chipset |
|
| SAS 5/E HBA |
3Gb/s SAS |
PCI-Express 1.0 |
2x4 external |
- |
- |
||||