Kubernetes之Ingress-Nginx

簡介

ingress-Nginx和ingress-Nginx-Controller的區別

ingress-Nginx:是每個服務自己創建的ingress,就是nginx的轉發規則,生成Nginx的配置文件

ingress-Nginx-Controller:相當於Nginx的服務,監聽API Server,根據用戶編寫的ingress-nginx規則(ingress.yaml文件),動態的去更改Nginx服務的配置文件,並且reload使其生效,此過程是自動化的,通過lua實現

ingress-Nginx-Controller 的Service類型

NodePort:用Deployment的方式部署一個ingress-nginx-controller,再創建一個type為NodePort的Service,這樣就在集群的所有Node節點暴露了ingress-nginx-controller的端口,然后找幾台機器掛在公有雲的ELB后面,然后把域名解析到公有雲的ELB就實現的服務的對外暴露   

LoadBalance:用Deployment部署一個ingress-nginx-controller,再創建一個type為LoadBalancer的Service關聯這組Pod.大部分公有雲,都會為LoadBalancer的Service自動創建一個負載均衡器,通常還綁定的公網地址,只要把域名指向該地址,就實現了服務的對外暴露

部署ingress-Nginx-Controller

1.ingress-Nginx-Crontoller所需的ServiceAccount,用來訪問API Server

apiVersion: v1
kind: ServiceAccount
metadata:
  
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.2
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress
  namespace: se
secrets:
- name: nginx-ingress-token-9bbd4

2.ingress-Nginx-Controller中ServiceAccount所需的Secret(通過base64加密之后的ca和token)

apiVersion: v1
data:
  ca.crt: LS0tLS1CUJBZ0lVUXVqazcwRmhXQm43dXQ1M3liMWdLeXNkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKb3N01DVGpPK2VNd0h3WURWUjBqQkJnd0ZvQVVXYTVCSzQvSApOMjdteEVvaVB3N01DVGpPK2VNd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFJcDlveFJTb29OelNGQmJrMEMvCmIwbVNvTUFlSU5vOVYrNWFEdGg3eExjWjZPazJCYVFWV1ZLK2ZVYW45WQpjaTQ9Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
  namespace: c2U=
  token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklpSjkuZZoWTJOdmRXNTBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5dVlXMWxjM0JoWTJVaU9pSnpaU0lzSW10MVltVnlibRTRXVlSjN2U0NlcTc5S25ENFdaWnoybXBvR1RuLVZHUFI4ai1B
kind: Secret
metadata:
 
  name: nginx-ingress-token-9bbd4
  namespace: se

type: kubernetes.io/service-account-token

3.ingress-Nginx-Controller的 Deployment配置文件

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.2
    component: controller
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
  namespace: se
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-ingress
      component: controller
      release: nginx-ingress
  template:
    metadata:
      labels:
        app: nginx-ingress
        component: controller
        release: nginx-ingress
    spec:
      containers:
      - args:
        - /nginx-ingress-controller
        - --default-backend-service=se/nginx-ingress-default-backend
        - --election-id=ingress-controller-leader
        - --ingress-class=nginx
        - --configmap=se/nginx-ingress-controller
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: metadata.namespace
        image: hrb.xxxx.com/library/nginx-ingress-controller:0.26.1
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: nginx-ingress-controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 33
      serviceAccount: nginx-ingress
      serviceAccountName: nginx-ingress

4.ingress-Nginx-Controller的Service配置文件

apiVersion: v1
kind: Service
metadata:
  labels:
    app: nginx-ingress
    chart: nginx-ingress-1.26.2
    component: controller
    heritage: Helm
    release: nginx-ingress
  name: nginx-ingress-controller
  namespace: se

spec:
  ports:
  - name: http
    nodePort: 30080
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    nodePort: 30443
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: nginx-ingress
    component: controller
    release: nginx-ingress
  type: NodePort

5.查看ingress-Nginx-Controller的Service

kubectl get  svc  nginx-ingress-controller -n se
NAME                       TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
nginx-ingress-controller   NodePort   192.168.2.67   <none>        80:30080/TCP,443:30443/TCP   1d

進行到這步,ingress-Nginx-Controller已經部署完了,所有的集群Node節點都已經監聽30080和30443端口

AWS上申請ELB,然后找兩個固定的Node節點,專門做轉發用,不做Pod調度

• ELB的80端口-->Node節點的NodePort30080端口
• ELB的443端口-->Node節點的NodePort30443端口

部署一個測試用的服務

1.測試服務的Deployment配置文件

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: test-docker
    env: stg
  name: test-docker
  namespace: test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-docker
  template:
    metadata: 
      labels:
        app: test-docker
        env: stg
    spec:
      containers:
      - env:
        - name: K8S_ENV
          value: stg
        - name: K8S_CLUSTER
          value: aws
        - name: CPU_REQUEST
          valueFrom:
            resourceFieldRef:
              containerName: test-docker
              divisor: "0"
              resource: requests.cpu
        - name: MEM_REQUEST
          valueFrom:
            resourceFieldRef:
              containerName: test-docker
              divisor: "0"
              resource: requests.memory
        - name: CPU_LIMIT
          valueFrom:
            resourceFieldRef:
              containerName: test-docker
              divisor: "0"
              resource: limits.cpu
        - name: MEM_LIMIT
          valueFrom:
            resourceFieldRef:
              containerName: test-docker
              divisor: "0"
              resource: limits.memory
        - name: TZ
          value: Asia/Shanghai
        - name: POD_IP
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: status.podIP
        image: hrb.xxx.com/test-docker:1.0.428.7eb2128
        imagePullPolicy: IfNotPresent
        name: test-docker
        ports:
        - containerPort: 8025
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /status
            port: 8025
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 5
        resources:
          limits:
            cpu: "1"
            memory: 2000Mi
          requests:
            cpu: 100m
            memory: 2000Mi
        volumeMounts:
        - mountPath: /etc/localtime
          name: host-time
          readOnly: true
        - mountPath: /data/logs
          name: log
        - mountPath: /app/conf
          name: config-volume
          readOnly: true

      volumes:
      - hostPath:
          path: /etc/localtime
          type: ""
        name: host-time
      - hostPath:
          path: /data/logs/test-docker-stg
          type: ""
        name: log
      - configMap:
          defaultMode: 420
          name: test-docker
        name: config-volume

2.測試服務的Service配置文件

apiVersion: v1
kind: Service
metadata:
  name: test-docker
  namespace: test
spec:
  ports:
  - name: http-8025
    port: 8025
    protocol: TCP
    targetPort: 8025
  selector:
    app: test-docker
  type: ClusterIP

3.測試服務的ingress配置文件

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: test-docker
  namespace: test
spec:
  rules:
  - host: test.baidu.com
    http:
      paths:
      - backend:
          serviceName: test-docker
          servicePort: 8025
        path: /

到這里,測試服務部署完成,測試服務的域名可以解析到ELB,然后就完成通過域名訪問了.

 

服務怎么通過ingress暴露出去,然后用域名訪問的?

1.先創建ingress-nginx-controller,ingress-nginx-controller的Server通過NodePort方式暴露端口,這樣所有K8S集群的Node節點全部監聽NodePort端口,這個就相當於是Nginx的服務

　　那Nginx的配置文件怎么來呢,ingress-nginx-controller監聽API Server,用戶在K8S集群內創建完服務的ingress之后,ingress-nginx-controller就會加載這個ingress里面的規則信息,並更新到ingress-nginx-controller的配置文件里

2.創建一個AWS的ELB,解析到隨便兩台Node節點

3.用戶創建一個服務,先創建Deployment、Service、ingress,然后ingress里寫上域名 轉發到 某個Service上，然后service會轉發到 具體的Pod上的

4.ingress里配置的域名解析到ELB的地址,就行了,就能通過域名訪問K8S集群的服務了.