1、fio 簡單介紹
fio 磁盤性能測試工具，磁盤IO是檢查磁盤性能的重要指標，可以按照負載情況分成順序讀寫、隨機讀寫兩大類
fio最初是用來節省為特定負載寫專門測試程序，或是進行性能測試，或是找到和重現bug的時間。寫這么一個測試應用是非常浪費時間的。因此需要一個工具來模擬給定的io負載，而不用重復的寫一個又一個的特定的測試程序。但是test負載很難定義。因為可能會產生很多進程或線程，他們每一個都用他們自己的方式產生io。fio需要足夠靈活得來模擬這些case
2、磁盤讀寫原理與順序/隨機讀寫說明以及區別
許多小伙伴不太明白順序讀寫和隨機讀寫的意思和區別（比如我，查閱了不少資料做了以下簡單的描述）
（1）磁盤讀寫原理
信息存儲在硬盤里，打開也看不見里面有任何東西只有些盤片。如果把盤片放大，會看見盤片表面凹凸不平，凸起的地方被磁化，凹的地方是沒有被磁化；凸起的地方代表數字1（磁化為1），凹的地方代表數字0。因此硬盤可以以二進制來存儲表示文字、圖片等信息。
（2）順序讀寫和隨機讀寫
順序讀寫：指的是本次 I/O 給出的初始扇區地址和上一次 I/O 的結束扇區地址是完全連續或者相隔不多的。反之，如果相差很大，則算作一次隨機 I/O。
隨機讀寫：隨機I/O可能是因為磁盤碎片導致磁盤空間不連續，或者當前block空間小於文件大小導致的。
SSD固態盤主要看的就是隨機讀寫性能-（由控制單元和固態存儲單元（DRAM或FLASH芯片）組成，存儲單元負責存儲數據，控制單元負責讀取、寫入數據。由於固態硬盤沒有普通硬盤的機械結構，也不存在機械硬盤的尋道問題）
SAS機械硬盤主要看的是順序讀寫性能-（數據是存儲的扇區的，讀寫是依靠磁頭的擺動尋址的。順序讀寫主要時間花費在了傳輸時間，隨機讀寫需要多次尋道和旋轉延遲。）
（3）順序讀寫與隨機讀寫區別
順序讀寫：文件指針只能從頭移動到尾。
隨機讀寫：文件指針可以隨意移動，根據需要。
連續讀寫比隨機讀寫效率高
因為在做連續讀寫的時候，磁頭幾乎不用換道或者換道的時間很短；
而對於隨機讀寫，如果這個讀寫很多的話，會導致磁頭不停地換道，造成效率的降低。
3、參數說明（更多參數詳見“fio --help”）
-filename=/dev/sdb1 測試文件名稱，通常選擇需要測試的盤的data目錄。 只能是分區，不能是目錄，會破壞數據
-direct=1 測試過程繞過機器自帶的buffer。使測試結果更真實。
-rw=randrw 測試隨機寫和讀的I/O
-rw=rw/readwrite 測試順序讀寫的I/O
-rw=read 順序讀
-rw=write 順序寫
-rw=randread 隨機讀
-rw=randwrite 隨機寫
-iodepth=1 隊列深度，只有使用libaio時才有意義，這是一個可以影響IOPS的參數，通常情況下為1。
-ioengine=psync/libaio io引擎使用pync方式
-bs=4k 單次io的塊文件大小為4k (單次IO越小的時候，單次IO所耗費的時間也越少，相應的IOPS也就越大)
-size=2G 測試文件大小為2g，以每次4k的io進行測試,此大小不能超過filename的大小，否則會報錯。
-numjobs=10 測試線程為10.
-runtime=10 測試時間為10秒，如果不寫則一直將2g文件分4k每次寫完為止。
-lockmem=1g 只使用1g內存進行測試。
-rwmixwrite=30 在混合讀寫的模式下，寫占30%
-group_reporting 關於顯示結果的，匯總每個進程的信息。
-name=name 直接測試讀會在測試設備寫入name.*文件
-output=rw.txt 將輸出內容寫入文件
4、測試范例 （視情況自行選擇參數並更改）
（1）順序讀
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=read -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=read, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
name: Laying out IO file(s) (1 file(s) / 2048MB)
Jobs: 1 (f=1): [R] [100.0% done] [53528KB/0KB/0KB /s] [13.4K/0/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=15754: Sun May 9 00:04:40 2021
read : io=608940KB, bw=60888KB/s, iops=15221, runt= 10001msec
clat (usec): min=42, max=634, avg=64.26, stdev=13.58
lat (usec): min=42, max=635, avg=64.40, stdev=13.58
clat percentiles (usec):
| 1.00th=[ 47], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 52],
| 30.00th=[ 53], 40.00th=[ 56], 50.00th=[ 66], 60.00th=[ 69],
| 70.00th=[ 72], 80.00th=[ 74], 90.00th=[ 77], 95.00th=[ 81],
| 99.00th=[ 107], 99.50th=[ 124], 99.90th=[ 171], 99.95th=[ 195],
| 99.99th=[ 266]
bw (KB /s): min=51072, max=74128, per=100.00%, avg=61282.95, stdev=8940.17
lat (usec) : 50=3.15%, 100=95.56%, 250=1.28%, 500=0.01%, 750=0.01%
cpu : usr=0.07%, sys=75.35%, ctx=140335, majf=0, minf=28
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=152235/w=0/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=608940KB, aggrb=60887KB/s, minb=60887KB/s, maxb=60887KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=149199/2, merge=0/1, ticks=7981/0, in_queue=7978, util=80.90%（2）順序寫
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=write -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
Jobs: 1 (f=1): [W] [100.0% done] [0KB/48488KB/0KB /s] [0/12.2K/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=15893: Sun May 9 00:07:27 2021
write: io=527800KB, bw=52775KB/s, iops=13193, runt= 10001msec
clat (usec): min=53, max=751, avg=74.34, stdev=16.13
lat (usec): min=53, max=751, avg=74.52, stdev=16.14
clat percentiles (usec):
| 1.00th=[ 57], 5.00th=[ 58], 10.00th=[ 59], 20.00th=[ 60],
| 30.00th=[ 62], 40.00th=[ 64], 50.00th=[ 74], 60.00th=[ 80],
| 70.00th=[ 83], 80.00th=[ 87], 90.00th=[ 90], 95.00th=[ 94],
| 99.00th=[ 123], 99.50th=[ 141], 99.90th=[ 187], 99.95th=[ 213],
| 99.99th=[ 422]
bw (KB /s): min=42824, max=62568, per=100.00%, avg=52800.00, stdev=6626.28
lat (usec) : 100=97.08%, 250=2.89%, 500=0.02%, 750=0.01%, 1000=0.01%
cpu : usr=0.05%, sys=85.36%, ctx=80042, majf=0, minf=29
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=131950/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=527800KB, aggrb=52774KB/s, minb=52774KB/s, maxb=52774KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=0/130571, merge=0/1, ticks=0/8390, in_queue=8388, util=83.89%（3）隨機讀
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=randread -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=randread, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
Jobs: 1 (f=1): [r] [100.0% done] [53036KB/0KB/0KB /s] [13.3K/0/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=15974: Sun May 9 00:09:00 2021
read : io=550584KB, bw=55053KB/s, iops=13763, runt= 10001msec
clat (usec): min=45, max=2140, avg=71.00, stdev=17.94
lat (usec): min=45, max=2141, avg=71.13, stdev=17.95
clat percentiles (usec):
| 1.00th=[ 48], 5.00th=[ 51], 10.00th=[ 52], 20.00th=[ 56],
| 30.00th=[ 67], 40.00th=[ 70], 50.00th=[ 73], 60.00th=[ 74],
| 70.00th=[ 77], 80.00th=[ 79], 90.00th=[ 83], 95.00th=[ 89],
| 99.00th=[ 110], 99.50th=[ 131], 99.90th=[ 181], 99.95th=[ 213],
| 99.99th=[ 812]
bw (KB /s): min=46928, max=69160, per=99.99%, avg=55045.47, stdev=6973.14
lat (usec) : 50=2.19%, 100=96.14%, 250=1.64%, 500=0.02%, 750=0.01%
lat (usec) : 1000=0.01%
lat (msec) : 2=0.01%, 4=0.01%
cpu : usr=0.09%, sys=78.34%, ctx=121541, majf=0, minf=28
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=137646/w=0/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=550584KB, aggrb=55052KB/s, minb=55052KB/s, maxb=55052KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=136380/2, merge=0/1, ticks=7707/0, in_queue=7708, util=77.09%（4）隨機寫
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=randwrite -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=randwrite, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
Jobs: 1 (f=1): [w] [100.0% done] [0KB/56419KB/0KB /s] [0/14.2K/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=16046: Sun May 9 00:10:27 2021
write: io=568740KB, bw=56868KB/s, iops=14217, runt= 10001msec
clat (usec): min=54, max=896, avg=68.65, stdev=13.78
lat (usec): min=54, max=897, avg=68.85, stdev=13.78
clat percentiles (usec):
| 1.00th=[ 59], 5.00th=[ 60], 10.00th=[ 61], 20.00th=[ 62],
| 30.00th=[ 63], 40.00th=[ 64], 50.00th=[ 65], 60.00th=[ 67],
| 70.00th=[ 69], 80.00th=[ 73], 90.00th=[ 78], 95.00th=[ 85],
| 99.00th=[ 113], 99.50th=[ 129], 99.90th=[ 223], 99.95th=[ 290],
| 99.99th=[ 450]
bw (KB /s): min=54344, max=59800, per=100.00%, avg=56980.63, stdev=1192.52
lat (usec) : 100=97.66%, 250=2.27%, 500=0.06%, 750=0.01%, 1000=0.01%
cpu : usr=0.08%, sys=91.91%, ctx=41937, majf=0, minf=27
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=0/w=142185/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: io=568740KB, aggrb=56868KB/s, minb=56868KB/s, maxb=56868KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=0/141304, merge=0/1, ticks=0/9746, in_queue=9744, util=97.26%（5）混合順序讀寫
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=rw -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=rw, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
Jobs: 1 (f=1): [M] [100.0% done] [23032KB/23336KB/0KB /s] [5758/5834/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=16102: Sun May 9 00:11:33 2021
read : io=300452KB, bw=30042KB/s, iops=7510, runt= 10001msec
clat (usec): min=41, max=741, avg=60.08, stdev=15.11
lat (usec): min=41, max=744, avg=60.22, stdev=15.11
clat percentiles (usec):
| 1.00th=[ 46], 5.00th=[ 49], 10.00th=[ 50], 20.00th=[ 51],
| 30.00th=[ 51], 40.00th=[ 52], 50.00th=[ 53], 60.00th=[ 55],
| 70.00th=[ 64], 80.00th=[ 73], 90.00th=[ 77], 95.00th=[ 81],
| 99.00th=[ 102], 99.50th=[ 122], 99.90th=[ 177], 99.95th=[ 211],
| 99.99th=[ 442]
bw (KB /s): min=22960, max=34376, per=100.00%, avg=30410.95, stdev=4527.95
write: io=300316KB, bw=30029KB/s, iops=7507, runt= 10001msec
clat (usec): min=53, max=994, avg=70.09, stdev=16.20
lat (usec): min=53, max=994, avg=70.29, stdev=16.21
clat percentiles (usec):
| 1.00th=[ 57], 5.00th=[ 58], 10.00th=[ 59], 20.00th=[ 59],
| 30.00th=[ 60], 40.00th=[ 61], 50.00th=[ 63], 60.00th=[ 66],
| 70.00th=[ 76], 80.00th=[ 85], 90.00th=[ 89], 95.00th=[ 93],
| 99.00th=[ 113], 99.50th=[ 135], 99.90th=[ 185], 99.95th=[ 223],
| 99.99th=[ 434]
bw (KB /s): min=23336, max=34048, per=100.00%, avg=30375.16, stdev=4390.51
lat (usec) : 50=2.54%, 100=95.58%, 250=1.85%, 500=0.03%, 750=0.01%
lat (usec) : 1000=0.01%
cpu : usr=0.11%, sys=81.47%, ctx=103856, majf=0, minf=29
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=75113/w=75079/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=300452KB, aggrb=30042KB/s, minb=30042KB/s, maxb=30042KB/s, mint=10001msec, maxt=10001msec
WRITE: io=300316KB, aggrb=30028KB/s, minb=30028KB/s, maxb=30028KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=74842/74787, merge=0/1, ticks=4097/4760, in_queue=8855, util=88.22%（6）混合隨機讀寫
[root@centos7 test]# fio -fimename=/dev/sdc1 -direct=1 -rw=randrw -iodepth=1 -ioengine=psync -bs=4k -size=2G -numjobs=1 -runtime=10 -group_reporting -name=name
name: (g=0): rw=randrw, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1
fio-2.1.10
Starting 1 process
Jobs: 1 (f=1): [m] [100.0% done] [23160KB/23648KB/0KB /s] [5790/5912/0 iops] [eta 00m:00s]
name: (groupid=0, jobs=1): err= 0: pid=16210: Sun May 9 00:13:45 2021
read : io=290024KB, bw=29000KB/s, iops=7249, runt= 10001msec
clat (usec): min=44, max=771, avg=62.03, stdev=14.93
lat (usec): min=44, max=771, avg=62.16, stdev=14.93
clat percentiles (usec):
| 1.00th=[ 48], 5.00th=[ 50], 10.00th=[ 51], 20.00th=[ 52],
| 30.00th=[ 53], 40.00th=[ 55], 50.00th=[ 57], 60.00th=[ 60],
| 70.00th=[ 66], 80.00th=[ 73], 90.00th=[ 78], 95.00th=[ 84],
| 99.00th=[ 102], 99.50th=[ 118], 99.90th=[ 179], 99.95th=[ 221],
| 99.99th=[ 470]
bw (KB /s): min=23384, max=32232, per=100.00%, avg=29312.00, stdev=3044.28
write: io=289732KB, bw=28970KB/s, iops=7242, runt= 10001msec
clat (usec): min=54, max=771, avg=72.58, stdev=14.81
lat (usec): min=54, max=771, avg=72.78, stdev=14.82
clat percentiles (usec):
| 1.00th=[ 59], 5.00th=[ 60], 10.00th=[ 61], 20.00th=[ 62],
| 30.00th=[ 63], 40.00th=[ 65], 50.00th=[ 67], 60.00th=[ 71],
| 70.00th=[ 78], 80.00th=[ 85], 90.00th=[ 90], 95.00th=[ 96],
| 99.00th=[ 114], 99.50th=[ 131], 99.90th=[ 187], 99.95th=[ 225],
| 99.99th=[ 298]
bw (KB /s): min=23704, max=32056, per=100.00%, avg=29257.26, stdev=2921.99
lat (usec) : 50=1.26%, 100=96.35%, 250=2.35%, 500=0.03%, 750=0.01%
lat (usec) : 1000=0.01%
cpu : usr=0.17%, sys=85.70%, ctx=81379, majf=0, minf=27
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=72506/w=72433/d=0, short=r=0/w=0/d=0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: io=290024KB, aggrb=28999KB/s, minb=28999KB/s, maxb=28999KB/s, mint=10001msec, maxt=10001msec
WRITE: io=289732KB, aggrb=28970KB/s, minb=28970KB/s, maxb=28970KB/s, mint=10001msec, maxt=10001msec
Disk stats (read/write):
sdc: ios=72006/71942, merge=0/1, ticks=4119/4861, in_queue=8976, util=89.73%5、輸出結果解釋說明
IOPS (英文：Input/Output Operations Per Second)，即每秒進行讀寫（I/O）操作的次數，多用於數據庫、存儲等場合，衡量隨機訪問的性能。
主要查看iops
iops ：磁盤的每秒讀寫次數，這個是隨機讀寫考察的重點
bw ：磁盤的吞吐量，這個是順序讀寫考察的重點，類似於下載速度
avg ：每個IO請求的平均相應時間
io ：總的輸入輸出量
runt ：總運行時間
msec ：毫秒
usec ：微秒
lat (msec)：延遲(毫秒)隨機關注IOPS 因為隨機讀寫的話，每次IO操作的尋址時間和旋轉延時都不能忽略不計，而這兩個時間的存在也就限制了IOPS的大小
而順序讀寫關注bw 可以忽略不計尋址時間和旋轉延時，主要花費在數據傳輸的時間上。
注意：盡量避免對有數據的磁盤或者分區做測試，謹慎操作，可能理解不夠透徹的話參數使用錯誤會破壞已存在的數據