k8s 壓測工具之perf-test

1、安裝go環境

下載go

配置環境變量

export GO_HOME=/usr/local/go
export PATH=$GO_HOME/bin:$PATH

 

2、下載perftest

https://github.com/kubernetes/perf-tests/

 

3、解壓后進入clusterloader2目錄

cd perf-tests-master/clusterloader2/

 

4、運行時可以直接go運行，或者編譯后

go run cmd/clusterloader.go --kubeconfig=kubeConfig.yaml --testconfig=config.yaml

或先編譯，編譯會生成clusterloader

./run-e2e.sh --testconfig=config.yaml
./clusterloader --kubeconfig=kubeConfig.yaml --testconfig=config.yaml

 

5、配置待測試文件

pod形式

1、pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: {{.Name}}
#  name: nginx
  labels:
    group: {{.Group}}
spec:
  containers:
  - image: nginx:1.12
    name: nginx

2、測試文件 config-pod.yaml

{{$namespaces := 1}}
{{$podPerNamespace := 2}}
{{$totalPods := 2}}  #pod數量

name: pod-test
automanagedNamespaces: {{$namespaces}}
tuningSets:
- name: Sequence
  parallelismLimitedLoad:
    parallelismLimit: 1
steps:
# start measurements
- measurements:
  - Identifier: PodStartupLatency
    Method: PodStartupLatency
    Params:
      action: start
      labelSelector: group = latency
- measurements:
  - Identifier: SchedulingMetrics
    Method: SchedulingMetrics
    Params:
      action: reset
      labelSelector: group = latency
- measurements:
  - Identifier: SchedulingThroughput
    Method: SchedulingThroughput
    Params:
      action: start
# create pods
- phases:
  - namespaceRange:
      min: 1
      max: {{$namespaces}}
    replicasPerNamespace: {{$podPerNamespace}}
    tuningSet: Sequence
    objectBundle:
    - basename: latency-pod
      objectTemplatePath: pod.yaml
      templateFillMap:
        Group: latency
        AppName: latency-pod
- measurements:
  - Identifier: SchedulingThroughput
    Method: SchedulingThroughput
    Params:
      action: gather
- measurements:
  - Identifier: WaitForRunningPods
    Method: WaitForRunningPods
    Params:
      desiredPodCount: {{$totalPods}}
      labelSelector: group = latency
      timeout: 15m
 gather measurements
- measurements:
  - Identifier: PodStartupLatency
    Method: PodStartupLatency
    Params:
      action: gather
- measurements:
  - Identifier: SchedulingMetrics
    Method: SchedulingMetrics
    Params:
      action: gather

 

deploy形式

1、deploy文件 deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{.Name}}
  labels:
    group: {{.Group}}
spec:
  replicas: {{.Replicas}}
  selector:
    matchLabels:
      name: {{.Name}}
  template:
    metadata:
      labels:
        name: {{.Name}}
        group: {{.Group}}
    spec:
      containers:
      - name: {{.Name}}
        image: nginx:1.12

 

2、config-deployment-old.yaml

{{$namespaces := 1}}
{{$deploymentPerNamespace := 200}}

name: deployment-test
automanagedNamespaces: {{$namespaces}}
tuningSets:
- name: Sequence
  parallelismLimitedLoad:
    parallelismLimit: 1
steps:
# start measurements
- measurements:
  - Identifier: WaitForRunningDeployments
    Method: WaitForControlledPodsRunning
    Params:
      action: start
      apiVersion: apps/v1
      kind: Deployment
      labelSelector: group = latency
      operationTimeout: 15m
# create deployment
- phases:
  - namespaceRange:
      min: 1
      max: {{$namespaces}}
    replicasPerNamespace: {{$deploymentPerNamespace}}
    tuningSet: Sequence
    objectBundle:
    - basename: latency-deployment
      objectTemplatePath: deployment.yaml
      templateFillMap:
        Replicas: 3
        Group: latency
- measurements:
  - Identifier: WaitForRunningDeployments
    Method: WaitForControlledPodsRunning
    Params:
      action: gather
# create deployment
- phases:
  - namespaceRange:
      min: 1
      max: {{$namespaces}}
    replicasPerNamespace: {{$deploymentPerNamespace}}
    tuningSet: Sequence
    objectBundle:
    - basename: latency-deployment
      objectTemplatePath: deployment.yaml
      templateFillMap:
        Replicas: 5
        Group: latency
- measurements:
  - Identifier: WaitForRunningDeployments
    Method: WaitForControlledPodsRunning
    Params:
      action: gather

- phases:
  - namespaceRange:
      min: 1
      max: {{$namespaces}}
    replicasPerNamespace: {{$deploymentPerNamespace}}
    tuningSet: Sequence
    objectBundle:
    - basename: latency-deployment
      objectTemplatePath: deployment.yaml
      templateFillMap:
        Replicas: 0
        Group: latency
- measurements:
  - Identifier: WaitForRunningDeployments
    Method: WaitForControlledPodsRunning
    Params:
      action: gather

 

6、完整測試命令

樣例：

./clusterloader --kubeconfig=/root/.kube/config  --alsologtostderr --provider=local --masterip=<集群masterIP> --master-internal-ip=<集群masterIP> --mastername=<集群master hostname> --report-dir=<測試結果生成目錄> --testconfig=<測試文件>

 

實際命令

#測試pod
./clusterloader --kubeconfig=/root/.kube/config  --alsologtostderr --provider=local --masterip=192.168.7.7 --master-internal-ip=192.168.7.7 --mastername=master --report-dir=result --testconfig=config-pod.yaml

#測試deployment
./clusterloader --kubeconfig=/root/.kube/config  --alsologtostderr --provider=local --masterip=192.168.7.7 --master-internal-ip=192.168.7.7 --mastername=master --report-dir=result --testconfig=config-deployment.yaml

 

7、結果展示