IngressController的session stick問題

周末爬坑，IngressController恢復因為鏡像下載和版本問題折騰一下午，晚上終於折騰出個眉目。

首先，在Kubernetes的service中是可以設置Session Affinity的。例子如下：

[root@k8s-master ~]# cat rc.yaml
apiVersion: v1
kind: ReplicationController
metadata:
  name: helloworld-service
spec:
  replicas: 2 
  template:
    metadata:
      labels:
        weblogic-app: "helloworld"
        version: "0.1"
    spec:
      containers:
      - name: weblogichelloworld
        image: 1213-helloworld:v2
        ports:
        - containerPort: 7001
---
apiVersion: v1
kind: Service
metadata:
  name: helloworldsvc
  labels:
    weblogic-app: helloworld
spec:
  type: NodePort
  ports:
  - port: 7001
    protocol: TCP
    targetPort: 7001
    name: http
    nodePort: 30005
  selector:
    weblogic-app: helloworld
  sessionAffinity: ClientIP

最核心就是最后那句，會基於客戶端訪問服務的ip進行hash運算后把同一ip的請求路由到同一個pod.這樣通過nodePort方式過來的請求就不會到處分發了。

但這並不意味着通過IngressController過來的請求不會到處發，實際上在gcr.io/google_containers/nginx-ingress-controller:0.61版本就加上了session stick的功能，需要你在

創建Ingress對象的時候添加Annotation,比如:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: dashboard-weblogic-ingress
 annotations: ingress.kubernetes.io/affinity: "cookie" ingress.kubernetes.io/session-cookie-name: "route" ingress.kubernetes.io/session-cookie-hash: "sha1"
spec:
  rules:
  - host: helloworld.paic.test
    http:
      paths:
      - backend:
          serviceName: helloworldsvc 
          servicePort: 7001
        path: /

 

如何驗證這個annotation已經生效了呢？這個問題我折騰了很久，因為按照文檔的說法通過

curl -v http://<ingress-svc-address> -H 'Host: example.com'

kubectl  exec -n kube-system nginx-ingress-lb-303jx  -- curl -v localhost  -H'Host: helloworld.test' 

全部返回的是404,503,並沒有以下的cookie

< Set-Cookie: route=dc89ae303c62a8bfce8bf32f06d27c31f0980ef7; Path=/; HttpOnly

無奈進入IngressController,查看具體的最終Nginx配置

在 /etc/nginx/nginx.conf 中。

 

 而Pod的信息是

 

 可見Nginx是繞開了Service,直接把pod的ip寫入到配置文件，所以也就是Service中的Session Affinity設置應該不起作用。

但Cookie到底有沒有設置成功呢，通過chrome訪問網站，然后查看Cookie

 

 成功看到，驗證的時候也沒問題，所以應該配置生效。

最后貼一個ingress-controller的yaml

[root@k8s-master ingress]# cat nginx-ingress-controller.yaml 
apiVersion: v1
kind: ReplicationController
metadata:
  name: nginx-ingress-lb
  labels:
    name: nginx-ingress-lb
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: nginx-ingress-lb
      annotations:
        prometheus.io/port: '10254'
        prometheus.io/scrape: 'true'
    spec:
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.7
        name: nginx-ingress-lb
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: KUBERNETES_MASTER
            value: http://192.168.0.104:8080
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend
        - --apiserver-host=http://192.168.0.104:8080