kubernetes网络性能测试三兄弟

所有的测试都需要分两种情况，同宿主机和跨宿主机。

1 netperf多流测试

1.1 获取镜像

docker pull oldyoutiao/netperf

该镜像内已经将以下三个脚本打进去了，可以根据自身需要更改或者不使用。

vim streamserver.sh

#!/bin/bash
size=1
port_range = `seq 16001 16064 `
killall netserver
for j in $port_range
do
port=$j
netserver -p $port > netserverstream.log 2>&1 &
done

vim tcpStreamClient.sh

#!/bin/bash
server_ip=$1
port_range=`seq 16001 16064 `
killall netperf
for j in $port_range
do
port=$j
netperf -H ${server_ip} -l 60 -t TCP_STREAM -p $port -- -m 1400 -D > tcp-stream.log 2>&1 &
done

vim udpStreamClient.sh

#!/bin/bash
server_ip=$1
port_range=`seq 16001 16064 `
killall netperf
for j in $port_range
do
port=$j
netperf -H ${server_ip} -l 60 -t UDP_STREAM -p $port  -- -m 1 -D -R 1 > udp-stream.log 2>&1 &
done

1.2 TCP_STREAM测试

在k8s集群内创建netperf deployment，并运行两个副本：

 kubectl create deployment --name netperf --image=oldyoutiao/netperf:latest --replicas=2
 kubectl get po -owide | grep netperf

其中一台为server

另外一台client

sh tcpStreamClient.sh 10.244.1.193

server端sar

客户端sar

1.3 UDP_STREAM

server端

client端

2 iperf3带宽测试

在k8s集群创建iperf3应用，使用镜像为：docker pull oldyoutiao/iperf3:latest

2.1 TCP测试

server端运行iperf3：iperf3 -s

client端运行tcp测试： iperf3 -c $server_ip -b 1000MB -n 1GB

2.2 UDP测试

server端运行iperf3：iperf3 -s

client端运行udp测试： iperf3 -u -c $server_ip -b 1000MB

3 qperf延迟测试

在k8s集群创建2副本的qperf deployment，使用镜像为：docker pull xridge/qperf

同宿主机测试，运行两个pod在同一个worker节点

server端直接运行：qperf

只测试一次

/ # qperf 172.17.0.3  -vu tcp_bw tcp_lat
tcp_bw:
    bw  =  350 MB/sec
tcp_lat:
    latency  =  93 us

循环测试1bytes-64KiB的带宽和延迟

/ # qperf 172.17.0.3 -oo msg_size:1:64K:*2 -vu tcp_bw tcp_lat
tcp_bw:
    bw        =  176 KB/sec
    msg_size  =    1 bytes
tcp_bw:
    bw        =  400 KB/sec
    msg_size  =    2 bytes
tcp_bw:
    bw        =  752 KB/sec
    msg_size  =    4 bytes
tcp_bw:
    bw        =  1.41 MB/sec
    msg_size  =     8 bytes
tcp_bw:
    bw        =  2.34 MB/sec
    msg_size  =    16 bytes
tcp_bw:
    bw        =  4.27 MB/sec
    msg_size  =    32 bytes
tcp_bw:
    bw        =  6.56 MB/sec
    msg_size  =    64 bytes
tcp_bw:
    bw        =  10.5 MB/sec
    msg_size  =   128 bytes
tcp_bw:
    bw        =  16.9 MB/sec
    msg_size  =   256 bytes
tcp_bw:
    bw        =  17.2 MB/sec
    msg_size  =   512 bytes
tcp_bw:
    bw        =  25.1 MB/sec
    msg_size  =     1 KiB (1,024)
tcp_bw:
    bw        =  37 MB/sec
    msg_size  =   2 KiB (2,048)
tcp_bw:
    bw        =  70.6 MB/sec
    msg_size  =     4 KiB (4,096)
tcp_bw:
    bw        =  141 MB/sec
    msg_size  =    8 KiB (8,192)
tcp_bw:
    bw        =  229 MB/sec
    msg_size  =   16 KiB (16,384)
tcp_bw:
    bw        =  393 MB/sec
    msg_size  =   32 KiB (32,768)
tcp_bw:
    bw        =  468 MB/sec
    msg_size  =   64 KiB (65,536)
tcp_lat:
    latency   =  83.9 us
    msg_size  =     1 bytes
tcp_lat:
    latency   =  90.8 us
    msg_size  =     2 bytes
tcp_lat:
    latency   =  95.8 us
    msg_size  =     4 bytes
tcp_lat:
    latency   =  76.8 us
    msg_size  =     8 bytes
tcp_lat:
    latency   =  87 us
    msg_size  =  16 bytes
tcp_lat:
    latency   =  91.2 us
    msg_size  =    32 bytes
tcp_lat:
    latency   =  100 us
    msg_size  =   64 bytes
tcp_lat:
    latency   =  90.9 us
    msg_size  =   128 bytes
tcp_lat:
    latency   =  82.1 us
    msg_size  =   256 bytes
tcp_lat:
    latency   =  101 us
    msg_size  =  512 bytes
tcp_lat:
    latency   =  97.6 us
    msg_size  =     1 KiB (1,024)
tcp_lat:
    latency   =  119 us
    msg_size  =    2 KiB (2,048)
tcp_lat:
    latency   =  184 us
    msg_size  =    4 KiB (4,096)
tcp_lat:
    latency   =  136 us
    msg_size  =    8 KiB (8,192)
tcp_lat:
    latency   =  175 us
    msg_size  =   16 KiB (16,384)
tcp_lat:
    latency   =  251 us
    msg_size  =   32 KiB (32,768)
tcp_lat:
    latency   =  251 us
    msg_size  =   64 KiB (65,536)