三層包測試
**發包方式：**192.168.20.205
**發包工具：**pktgen
**發包網卡：**i350
**收包設備：**1922.168.20.185
**CPU：**Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
**收包方式：**網卡ixgeb驅動，~~單隊列，~~promisc計數
收包網卡： 82599EB
**傳輸鏈路：**千兆RJ45網線直連
發包方法：
Pktgen:/> set 0 size 500
Pktgen:/> set 0 count 1000000
Pktgen:/> set 0 proto tcp
Pktgen:/> disable 0 vlan
Pktgen:/> start 0
Pktgen:/> clear 0 stats
收包設置：
驅動設置
- 重載驅動，清空計數，同時可以調整隊列數。
- 設置混雜模式
- 禁用PAUSE幀
irq設置
綁定IRQ
echo "1,13" > /proc/irq/eth0/smp_affinity_list
CMD
modprobe -r ixgbe
#modprobe ixgbe MQ=1,0
modprobe ixgbe
ifconfig eth0 promisc
ethtool --pause eth0 autoneg off rx off tx off
irq_bind.sh 1
測試結果：
| 包長(byte) |
pps |
Mbps |
seconds |
socket數 |
收包physics core數 |
隊列數 |
發包數 |
收包數 |
| 500 |
230000 |
960 |
|
1 |
1 |
24 |
10000000 |
10000000 |
| 400 |
284000 |
950 |
|
1 |
1 |
24 |
10000000 |
10000000 |
| 300 |
367000 |
940 |
|
1 |
1 |
24 |
10000000 |
10000000 |
| 200 |
542000 |
952 |
|
1 |
1 |
24 |
10000000 |
10000000 |
| 100 |
1039000 |
997 |
9.62 |
1 |
1 |
24 |
10000000 |
10000000 |
| 64 |
1483000 |
996 |
6.74 |
1 |
1 |
24 |
10000000 |
9907134 |
| 64 |
1483000 |
996 |
67.4 |
1 |
1 |
24 |
100000000 |
99147516 |
| 64 |
1483000 |
996 |
67.4 |
1 |
1 |
1 |
100000000 |
99671467 |
| 64 |
1483000 |
996 |
67.4 |
1 |
2 |
24 |
100000000 |
99252877 |
| 64 |
1483000 |
996 |
67.4 |
1 |
4 |
24 |
100000000 |
99084322 |
| 64 |
1483000 |
996 |
67.4 |
2 |
8 |
24 |
100000000 |
99177716 |
| 64 |
1483000 |
996 |
67.4 |
2 |
24 |
24 |
100000000 |
99110338 |
設置 Ring 為4096, 之前是512
ethtool -G eth0 rx 4096
| 包長(byte) |
pps |
Mbps |
seconds |
socket數 |
收包physics core數 |
隊列數 |
發包數 |
收包數 |
| 64 |
1483000 |
996 |
67.4 |
2 |
8 |
24 |
100000000 |
99177716 |
| 64 |
1483000 |
996 |
67.4 |
2 |
24 |
24 |
100000000 |
98947571 |
查看丟包數的另一種方法
[root@T185 tong]# ethtool -S eth0 |grep rx_missed_errors
rx_missed_errors: 1052429
設置 dev_weight, 默認值64
echo 512 > /proc/sys/net/core/dev_weight
| 包長(byte) |
pps |
Mbps |
seconds |
socket數 |
收包physics core數 |
隊列數 |
dev_weight |
發包數 |
收包數 |
| 64 |
1483000 |
996 |
67.4 |
2 |
24 |
24 |
4096 |
100000000 |
98992914 |
參考： https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Performance_Tuning_Guide/s-network-common-queue-issues.html#s-network-commonque-nichwbuf
換用dpdk
dpdk-app收包測試結果
./dpdk-app -l 1,2,3 ../../conf/dpdk-app.conf
| 包長(byte) |
pps |
Mbps |
發包數 |
收包數 |
| 500 |
230000 |
960 |
10000000 |
10000000 |
| 400 |
284000 |
950 |
10000000 |
10000000 |
| 300 |
367000 |
940 |
10000000 |
10000000 |
| 200 |
541000 |
952 |
10000000 |
10000000 |
| 100 |
1039000 |
997 |
10000000 |
10000000 |
| 64 |
1483000 |
996 |
10000000 |
10000000 |
| 64 |
1483000 |
996 |
100000000 |
100000000 |
| 64 |
1483000 |
996 |
1000000000 |
1000000000 |
IP/UDP 分片包測試
DNS包 測試
真實包測試
| 包名 |
pps |
Mbps |
隊列/線程數 |
發包數 |
收包數 |
TSC/pkt |
| onescap-4.4g.pcap |
150000 |
940 |
1 |
3800000 |
601200 |
87712 |
| onescap-4.4g.pcap |
150000 |
940 |
1 |
3800000 |
631847 |
82141 |
| onescap-4.4g.pcap |
150000 |
940 |
1 |
3800000 |
615956 |
84308 |
| onescap-4.4g.pcap |
150000 |
940 |
1 |
3800000 |
689366 |
75871 |
| onescap-4.4g.pcap |
150000 |
940 |
1 |
3800000 |
950270 |
53670 |
| onescap-4.4g.pcap |
150000 |
940 |
2 |
3800000 |
803675 |
AVG(105741,102386)=104063 |
| onescap-4.4g.pcap |
150000 |
940 |
2 |
3800000 |
727040 |
AVG(111395,117661)=114528 |
| onescap-4.4g.pcap |
150000 |
940 |
2 |
3800000 |
782799 |
AVG(108618,102867)=105742 |
| onescap-4.4g.pcap |
150000 |
940 |
2 |
3800000 |
730686 |
AVG(113122,113510)=113316 |
| onescap-4.4g.pcap |
150000 |
940 |
2 |
3800000 |
717762 |
AVG(113489,116942)=115215 |
| onescap-4.4g.pcap |
150000 |
940 |
4 |
3800000 |
753101 |
AVG(153469,125007,167986,167374)=153459 |
| onescap-4.4g.pcap |
150000 |
940 |
4 |
3800000 |
897236 |
AVG(108565,170719,122691,121693)=130917 |
| onescap-4.4g.pcap |
150000 |
940 |
4 |
3800000 |
765500 |
AVG(142303,156840,122374,193267)=153696 |
結論
單包處理的TSC隨着線程數增加而增加，說明線程間有鎖。以至性能不能隨着並行數的增加線性增長。從測試結果看，並行處理幾乎沒有為性能增長帶來任何好處。
自構建http測試
| 包名 |
pps |
Mbps |
隊列/線程數 |
會話數 |
發包數 |
收包數 |
TSC/pkt |
| http_get.pcap |
none |
line |
1 |
1 |
10 |
10 |
159457 |
| http_get.pcap |
none |
line |
1 |
1 |
10 |
10 |
12384 |
| http_get.pcap |
none |
line |
1 |
1 |
10 |
10 |
160807 |
| http_get.pcap |
none |
line |
1 |
1 |
10 |
10 |
12730 |
| http_get.pcap |
none |
line |
1 |
1 |
10 |
10 |
12074 |
