碼上歡樂
相關內容    簡體    繁體

Kubernetes 使用Nginx-Ingress實現藍綠發布/金絲雀發布/AB測試

本文轉載自   查看原文   2021-03-08 09:42   283    kubernetes

https://juejin.cn/post/6844903927318577159

背景介紹

某些情況下，我們在使用Kubernetes作為業務應用的雲平台，想要實現應用的藍綠部署用來迭代應用版本，用lstio太重太復雜，而且它本身定位於流控和網格治理；Ingress-Nginx在0.21版本引入了Canary功能，可以為網關入口配置多個版本的應用程序，使用annotation來控制多個后端服務的流量分配

Ingress-Nginx-Annotation Canary 功能介紹

如果想啟用Canary功能，要先設置nginx.ingress.kubernetes.io/canary: "true"，然后可以啟用以下注釋來配置Canary

  • nginx.ingress.kubernetes.io/canary-weight 請求到Canary ingress中指定的服務的請求百分比，值為0-100的整數，根據設置的值來決定大概有百分之多少的流量會分配Canary Ingress中指定的后端s服務
  • nginx.ingress.kubernetes.io/canary-by-header 基於request header 的流量切分，適用於灰度發布或者A/B測試，當設定的hearder值為always是，請求流量會被一直分配到Canary入口，當hearder值被設置為never時，請求流量不會分配到Canary入口，對於其他hearder值，將忽略，並通過優先級將請求流量分配到其他規則
  • nginx.ingress.kubernetes.io/canary-by-header-value 這個配置要和nginx.ingress.kubernetes.io/canary-by-header 一起使用，當請求中的hearder key和value 和nginx.ingress.kubernetes.io/canary-by-header nginx.ingress.kubernetes.io/canary-by-header-value匹配時，請求流量會被分配到Canary Ingress入口，對於其他任何hearder值，將忽略，並通過優先級將請求流量分配到其他規則
  • nginx.ingress.kubernetes.io/canary-by-cookie 這個配置是基於cookie的流量切分，也適用於灰度發布或者A/B測試，當cookie值設置為always時，請求流量將被路由到Canary Ingress入口，當cookie值設置為never時，請求流量將不會路由到Canary入口，對於其他值，將忽略，並通過優先級將請求流量分配到其他規則

金絲雀規則按優先順序進行如下排序：canary-by-header - > canary-by-cookie - > canary-weight

1.基於權重的小規模版本測試

  • v1版本編排文件
apiVersion: extensions/v1beta1 kind: Ingress metadata:  annotations: kubernetes.io/ingress.class: nginx  labels:  app: echoserverv1  name: echoserverv1  namespace: echoserver spec:  rules:  - host: echo.chulinx.com  http:  paths:  - backend:  serviceName: echoserverv1  servicePort: 8080  path: / --- kind: Service apiVersion: v1 metadata:  name: echoserverv1  namespace: echoserver spec:  selector:  name: echoserverv1  type: ClusterIP  ports:  - name: echoserverv1  port: 8080  targetPort: 8080 --- apiVersion: extensions/v1beta1 kind: Deployment metadata:  name: echoserverv1  namespace: echoserver  labels:  name: echoserverv1 spec:  template:  metadata:  labels:  name: echoserverv1  spec:  containers:  - image: mirrorgooglecontainers/echoserver:1.10  name: echoserverv1  ports:  - containerPort: 8080  name: echoserverv1 復制代碼
  • 查看v1版本創建的資源
$ [K8sSj] kubectl get pod,service,ingress -n echoserver
NAME                                READY   STATUS    RESTARTS   AGE
pod/echoserverv1-657b966cb5-7grqs   1/1     Running   0          24h

NAME                   TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
service/echoserverv1   ClusterIP   10.99.68.72   <none>        8080/TCP   24h

NAME                              HOSTS              ADDRESS   PORTS   AGE
ingress.extensions/echoserverv1   echo.chulinx.com             80      24h
復制代碼
  • 訪問v1的服務,可以看到10個請求都是訪問到一個pod上也就是v1版本的服務
$ [K8sSj] for i in `seq 10`;do curl -s echo.chulinx.com|grep Hostname;done Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs 復制代碼
  • 創建v2版本的服務

我們開啟canary功能，將v2版本的權重設置為50%,這個百分比並不能精確的將請求平均分配到兩個版本的服務，而是在50%上下浮動

apiVersion: extensions/v1beta1 kind: Ingress metadata:  annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "50"  labels:  app: echoserverv2  name: echoserverv2  namespace: echoserver spec:  rules:  - host: echo.chulinx.com  http:  paths:  - backend:  serviceName: echoserverv2  servicePort: 8080  path: / --- kind: Service apiVersion: v1 metadata:  name: echoserverv2  namespace: echoserver spec:  selector:  name: echoserverv2  type: ClusterIP  ports:  - name: echoserverv2  port: 8080  targetPort: 8080 --- apiVersion: extensions/v1beta1 kind: Deployment metadata:  name: echoserverv2  namespace: echoserver  labels:  name: echoserverv2 spec:  template:  metadata:  labels:  name: echoserverv2  spec:  containers:  - image: mirrorgooglecontainers/echoserver:1.10  name: echoserverv2  ports:  - containerPort: 8080  name: echoserverv2 復制代碼
  • 再次查看創建的資源
$ [K8sSj] kubectl get pod,service,ingress -n echoserver
NAME                                READY   STATUS    RESTARTS   AGE
pod/echoserverv1-657b966cb5-7grqs   1/1     Running   0          24h
pod/echoserverv2-856bb5758-f9tqn 1/1 Running 0 4s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/echoserverv1 ClusterIP 10.99.68.72 <none> 8080/TCP 24h service/echoserverv2 ClusterIP 10.111.103.170 <none> 8080/TCP 4s NAME HOSTS ADDRESS PORTS AGE ingress.extensions/echoserverv1 echo.chulinx.com 80 24h ingress.extensions/echoserverv2 echo.chulinx.com 80 4s 復制代碼
  • 訪問測試

可以看到請求有4個落到v2版本，6個落到v1版本，理論上來說，請求說越多，落到v2版本的請求數越接近設置的權重50%

$ [K8sSj] for i in `seq 10`;do curl -s echo.chulinx.com|grep Hostname;done Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs 復制代碼

2.基於header的A/B測試

  • 更改v2版本的編排文件

增加headernginx.ingress.kubernetes.io/canary-by-header: "v2"

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "50" nginx.ingress.kubernetes.io/canary-by-header: "v2" labels: app: echoserverv2 name: echoserverv2 namespace: echoserver spec: rules: - host: echo.chulinx.com http: paths: - backend: serviceName: echoserverv2 servicePort: 8080 path: / --- kind: Service apiVersion: v1 metadata: name: echoserverv2 namespace: echoserver spec: selector: name: echoserverv2 type: ClusterIP ports: - name: echoserverv2 port: 8080 targetPort: 8080 --- apiVersion: extensions/v1beta1 kind: Deployment metadata: name: echoserverv2 namespace: echoserver labels: name: echoserverv2 spec: template: metadata: labels: name: echoserverv2 spec: containers: - image: mirrorgooglecontainers/echoserver:1.10 name: echoserverv2 ports: - containerPort: 8080 name: echoserverv2 復制代碼
  • 更新訪問測試

測試了header 為v2:always v2:never v2:true這三個hearder值，可以看到當hearder為v2:always時，流量會全部流入v2，當v2:never時，流量會全部流入v1，當v2:true時，也就是非always/never,流量會按照配置的權重流入對應版本的服務

$ [K8sSj] kubectl apply -f appv2.yml ingress.extensions/echoserverv2 configured service/echoserverv2 unchanged deployment.extensions/echoserverv2 unchanged $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:always" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:never" echo.chulinx.com|grep Hostname;done Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:true" echo.chulinx.com|grep Hostname;done Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn 復制代碼
  • 自定義header-value
apiVersion: extensions/v1beta1 kind: Ingress metadata:  annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "50" nginx.ingress.kubernetes.io/canary-by-header: "v2" nginx.ingress.kubernetes.io/canary-by-header-value: "true"  labels:  app: echoserverv2  name: echoserverv2  namespace: echoserver spec:  rules:  - host: echo.chulinx.com  http:  paths:  - backend:  serviceName: echoserverv2  servicePort: 8080  path: / --- kind: Service apiVersion: v1 metadata:  name: echoserverv2  namespace: echoserver spec:  selector:  name: echoserverv2  type: ClusterIP  ports:  - name: echoserverv2  port: 8080  targetPort: 8080 --- apiVersion: extensions/v1beta1 kind: Deployment metadata:  name: echoserverv2  namespace: echoserver  labels:  name: echoserverv2 spec:  template:  metadata:  labels:  name: echoserverv2  spec:  containers:  - image: mirrorgooglecontainers/echoserver:1.10  name: echoserverv2  ports:  - containerPort: 8080  name: echoserverv2 復制代碼
  • 更新測試

可以看到只有header為v2:never時，請求流量才會流入v2版本，其他值流量都會按照權重設置流入不通版本的服務

$ [K8sSj] kubectl apply -f appv2.yml ingress.extensions/echoserverv2 configured service/echoserverv2 unchanged deployment.extensions/echoserverv2 unchanged $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:true" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:always" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn $ [K8sSj] for i in `seq 10`;do curl -s -H "v2:never" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs 復制代碼

3.基於cookie的流控

cookie其實和header原理大致相同，也是ingress自動cookie值，客戶訪問如果cookie匹配，流量就會流入與之匹配的后端服務

  • 更新v2版本的編排文件
apiVersion: extensions/v1beta1 kind: Ingress metadata:  annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/canary: "true" nginx.ingress.kubernetes.io/canary-weight: "50" nginx.ingress.kubernetes.io/canary-by-header: "v2" nginx.ingress.kubernetes.io/canary-by-header-value: "true" nginx.ingress.kubernetes.io/canary-by-cookie: "user_from_shanghai"  labels:  app: echoserverv2  name: echoserverv2  namespace: echoserver spec:  rules:  - host: echo.chulinx.com  http:  paths:  - backend:  serviceName: echoserverv2  servicePort: 8080  path: / --- kind: Service apiVersion: v1 metadata:  name: echoserverv2  namespace: echoserver spec:  selector:  name: echoserverv2  type: ClusterIP  ports:  - name: echoserverv2  port: 8080  targetPort: 8080 --- apiVersion: extensions/v1beta1 kind: Deployment metadata:  name: echoserverv2  namespace: echoserver  labels:  name: echoserverv2 spec:  template:  metadata:  labels:  name: echoserverv2  spec:  containers:  - image: mirrorgooglecontainers/echoserver:1.10  name: echoserverv2  ports:  - containerPort: 8080  name: echoserverv2 復制代碼
  • 訪問測試

可以看和header的訪問效果是一樣的，只不過cookie不能自定義value

$ [K8sSj] kubectl apply -f appv2.yml ingress.extensions/echoserverv2 configured service/echoserverv2 unchanged deployment.extensions/echoserverv2 unchanged $ [K8sSj] for i in `seq 10`;do curl -s --cookie "user_from_shanghai" echo.chulinx.com|grep Hostname;done Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn # zlx @ zlxdeMacBook-Pro in ~/Desktop/unicom/k8syml/nginx-ingress-canary-deployment [16:01:52] $ [K8sSj] for i in `seq 10`;do curl -s --cookie "user_from_shanghai:always" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv1-657b966cb5-7grqs Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn # zlx @ zlxdeMacBook-Pro in ~/Desktop/unicom/k8syml/nginx-ingress-canary-deployment [16:02:25] $ [K8sSj] for i in `seq 10`;do curl -s --cookie "user_from_shanghai=always" echo.chulinx.com|grep Hostname;done Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn Hostname: echoserverv2-856bb5758-f9tqn


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Kubernetes 使用Nginx-Ingress實現藍綠發布/金絲雀發布/AB測試 手把手教你使用 Nginx Ingress 實現金絲雀發布 k8s 使用 ingress 實現簡單的灰度發布（金絲雀發布） 藍綠發布、金絲雀發布(灰度發布)和滾動發布 藍綠部署、滾動部署、灰度發布、金絲雀發布 服務發布方法（藍綠、灰度、金絲雀） 通過 Traefik 使用 Kubernetes Service APIs 進行流量路由 (http,https,金絲雀發布) 軟件上線常用的發布方式 ---- 金絲雀、藍綠、紅黑灰發布 Spring Cloud實踐：降級、限流、滾動、灰度、AB、金絲雀的實現思路 Kubernetes部署nginx-ingress
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM