前文我們了解了k8s上的service資源的相關話題，回顧請參考：https://www.cnblogs.com/qiuhom-1874/p/14161950.html；今天我們來了解下k8s上的Ingress資源的相關話題；
我們知道在k8s上service是用來解決Pod訪問問題，它是通過kube-proxy在每個節點上創建iptables規則或ipvs規則，在用戶請求某個pod時，用戶的請求會被其service規則所捕獲，從而實現訪問對應pod；對於service來講，用戶請求直接在傳輸層就被捕獲轉發，效率很高效，但這同時也引入了一個新問題；比如我們運行的pod對外客戶端訪問需要https通信，如果使用service這種4層調度，那就意味着每個pod上我們要配置證書，這很顯然不是我們想要做的；那有沒有什么辦法做到在用戶訪問pod對應的service時使用https，而對應pod里又不用https協議呢？答案是有的；比如我們可以使用nginx來做https會話卸載器；我們只需要在代理上配置證書即可；又比如我們在k8s上運行了各種各樣的pod，這些pod的功能每個都不一樣，有的是專門處理用戶認證的，有的是專門處理站點主頁的，有的專門處理支付的等等，而這些pod對外都是提供一個獨有的url，那么這些pod需要怎么才能被集群外部訪問到呢？我們知道對於一個站點來講，如果后端有多個server同時提供一種服務，我們可以把這些同功能的server定義成一個組，然后使用nginx代理將不同功能url的訪問代理到不同組上即可；這樣一來就解決了后端多server被負載訪問的問題；那么對於k8s上這種同功能的pod怎么歸並成一個組呢？用戶訪問不同url怎么調度到不同的組上呢？很顯然要想實現這些功能，在k8s上應該有一個類似nginx一樣的代理存在；這個代理就叫做ingress 控制器；ingress 控制器和k8s上的其他控制不一樣，ingress控制器並不能直接運行為kube-controller-manager的一部分，它類似k8s集群上的coredns，需要在集群上單獨部署，本質上就是一個pod，我們可以使用k8s上的ds或deploy控制器來創建它；ingress controller pod的作用主要是引入集群外部流量，並實時監控着apiserver上ingress資源的變動，並將其ingress中定義的規則轉化為對應ingress控制器對應應用程序的專有配置，然后動態的重載或重啟對應守護進程來使其配置文件生效；在k8s上ingress是一種標准資源，它本質上就是我們定義的基於dns名稱（host）或url路徑把請求轉發至指定service資源的規則；簡單講ingress就是我們用來定義代理的配置所創建的資源；ingress控制器就是把對應ingress規則轉換為對應ingress控制器中應用程序的專有配置，然后重啟或重載對應配置文件使其生效的組件；
ingress和ingress controller pod的關系
提示：如上圖所示，ingress就是ingress 控制器pod的代理規則；用戶請求某個后端pod所提供的服務時，首先會通過ingress controller pod把流量引入到集群內部，然后ingress controller pod根據ingress定義的規則，把對應ingress規則轉化為對應ingress controller pod實現的對應應用的配置（ingress controller 可以由任何具有七層反向代理功能的服務實現，比如nginx,haproxy等等）然后再適配用戶請求，把對應請求反代到對應service上；而對於pod的選擇上，ingress控制器可以基於對應service中的標簽選擇器，直接同pod直接通信，無須通過service對象api的再次轉發，從而省去了用戶請求到kube-proxy實現的代理開銷（本質上ingress controller 也是運行為一個pod，和其他pod在同一網段中）；
ingress controller部署
在k8s上ingress controller的實現有很多，比如基於nginx的，基於haproxy的等等，這里以nginx為例；
下載ingress-nginx包
wget https://github.com/kubernetes/ingress-nginx/archive/nginx-0.28.0.tar.gz
解壓包，找到對應的部署清單
[root@master01 ~]# ll total 92144 -rw------- 1 root root 65586688 Dec 8 15:16 flannel-v0.13.1-rc1.tar drwxr-xr-x 2 root root 4096 Dec 21 21:04 manifests -rw-r--r-- 1 root root 28760559 Dec 21 21:02 nginx-0.28.0.tar.gz [root@master01 ~]# tar xf nginx-0.28.0.tar.gz [root@master01 ~]# ls flannel-v0.13.1-rc1.tar ingress-nginx-nginx-0.28.0 manifests nginx-0.28.0.tar.gz [root@master01 ~]# cd ingress-nginx-nginx-0.28.0/ [root@master01 ingress-nginx-nginx-0.28.0]# ls build code-of-conduct.md docs hack labels.yaml mkdocs.yml README.md SECURITY_CONTACTS version Changelog.md CONTRIBUTING.md go.mod images LICENSE OWNERS requirements-docs.txt test cmd deploy go.sum internal Makefile OWNERS_ALIASES rootfs vendor [root@master01 ingress-nginx-nginx-0.28.0]# cd deploy/ [root@master01 deploy]# ls aws cloud-generic grafana prometheus static with-validating-webhook.yaml.tpl baremetal cluster-wide minikube README.md validating-webhook.yaml.tpl [root@master01 deploy]# cd static/ [root@master01 static]# ls configmap.yaml mandatory.yaml namespace.yaml provider rbac.yaml with-rbac.yaml [root@master01 static]# pwd /root/ingress-nginx-nginx-0.28.0/deploy/static [root@master01 static]#
提示：資源配置清單在ingress-nginx-nginx-0.28.0/deploy/static下，名為mandatory.yaml；
資源配置清單內容
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: apps/v1 kind: Deployment metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: kubernetes.io/os: linux containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown --- apiVersion: v1 kind: LimitRange metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: limits: - default: min: memory: 90Mi cpu: 100m type: Container
提示：以上清單主要定義了一個名稱ingress-nginx的名稱空間，在其名稱空間下創建了幾個configmap，最重要的是用deployment創建了一個ingress-nginx pod；
這里說一下，對於ingress-nginx控制器，它本質還是運行為一個pod，對於pod來說要想接入外部訪問流量到集群內部來，有三種方式，一種是使用NodePort類型的service；第二種是使用ds或deploy控制器，在定義pod模板時使用hostPort把pod端口映射到宿主機方式；第三種是定義pod模板時使用hostNetwork，直接共享宿主機網絡名稱空間；如下所示
使用專有NodePort service來引入外部流量
提示：這種使用deploy控制管理ingress controller pod，如果在pod模板中沒有暴露端口，則需要創建一個service資源來暴露ingress controller pod的端口來引入外部流量到集群內部；
使用ds控制器管理ingress controller pod在pod模板中使用hostPort方式暴露端口
提示：使用ds控制器能夠保證每個節點上只運行一個ingress controller,所以我們可以把對應ingress controller pod端端口通過端口映射的方式映射到宿主機上的某一固定端口；
使用ds控制器在pod模板中使用hostNetwork方式共享宿主機網絡名稱空間
提示：共享宿主機網絡名稱空間，也必須使用ds控制器來確保對應每個節點上只能運行一個ingress controller pod，這樣才能確保每個ingress controller pod能夠正常把端口暴露出去，以供集群外部客戶端訪問；
選擇上述其中一種方式暴露ingress controller pod的端口即可；如果選擇使用ds控制器來暴露端口，我們就需要修改其對應資源配置清單中的pod模板，如下所示
使用ds控制器來管理ingress controller pod在pod模板中使用hostPort方式暴露端口
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: apps/v1 kind: DaemonSet metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: kubernetes.io/os: linux containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 hostPort: 30080 protocol: TCP - name: https containerPort: 443 hostPort: 30443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown --- apiVersion: v1 kind: LimitRange metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: limits: - default: min: memory: 90Mi cpu: 100m type: Container
提示：只需把對應控制器類型更改為DaemonSet，在pod模板中spec字段下把replicas去掉；在spec.template.spec.containers.ports字段中加上nodePort字段指定要把容器的端口映射到宿主機上某個端口；如果暴露的端口是非標准端口，在對應k8s集群外部我們還需要部署反代，比如使用nginx，haproxy,lvs；
使用ds控制器管理ingress controller pod在ds控制器資源配置中使用hostNetwork
apiVersion: v1 kind: Namespace metadata: name: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: nginx-configuration namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: tcp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- kind: ConfigMap apiVersion: v1 metadata: name: udp-services namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: v1 kind: ServiceAccount metadata: name: nginx-ingress-serviceaccount namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRole metadata: name: nginx-ingress-clusterrole labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - endpoints - nodes - pods - secrets verbs: - list - watch - apiGroups: - "" resources: - nodes verbs: - get - apiGroups: - "" resources: - services verbs: - get - list - watch - apiGroups: - "" resources: - events verbs: - create - patch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses verbs: - get - list - watch - apiGroups: - "extensions" - "networking.k8s.io" resources: - ingresses/status verbs: - update --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: Role metadata: name: nginx-ingress-role namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx rules: - apiGroups: - "" resources: - configmaps - pods - secrets - namespaces verbs: - get - apiGroups: - "" resources: - configmaps resourceNames: # Defaults to "<election-id>-<ingress-class>" # Here: "<ingress-controller-leader>-<nginx>" # This has to be adapted if you change either parameter # when launching the nginx-ingress-controller. - "ingress-controller-leader-nginx" verbs: - get - update - apiGroups: - "" resources: - configmaps verbs: - create - apiGroups: - "" resources: - endpoints verbs: - get --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: RoleBinding metadata: name: nginx-ingress-role-nisa-binding namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: nginx-ingress-role subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: nginx-ingress-clusterrole-nisa-binding labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: nginx-ingress-clusterrole subjects: - kind: ServiceAccount name: nginx-ingress-serviceaccount namespace: ingress-nginx --- apiVersion: apps/v1 kind: DaemonSet metadata: name: nginx-ingress-controller namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: selector: matchLabels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx template: metadata: labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx annotations: prometheus.io/port: "10254" prometheus.io/scrape: "true" spec: # wait up to five minutes for the drain of connections terminationGracePeriodSeconds: 300 serviceAccountName: nginx-ingress-serviceaccount nodeSelector: kubernetes.io/os: linux hostNetwork: true containers: - name: nginx-ingress-controller image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.28.0 args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --publish-service=$(POD_NAMESPACE)/ingress-nginx - --annotations-prefix=nginx.ingress.kubernetes.io securityContext: allowPrivilegeEscalation: true capabilities: drop: - ALL add: - NET_BIND_SERVICE # www-data -> 101 runAsUser: 101 env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace ports: - name: http containerPort: 80 protocol: TCP - name: https containerPort: 443 protocol: TCP livenessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP initialDelaySeconds: 10 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 readinessProbe: failureThreshold: 3 httpGet: path: /healthz port: 10254 scheme: HTTP periodSeconds: 10 successThreshold: 1 timeoutSeconds: 10 lifecycle: preStop: exec: command: - /wait-shutdown --- apiVersion: v1 kind: LimitRange metadata: name: ingress-nginx namespace: ingress-nginx labels: app.kubernetes.io/name: ingress-nginx app.kubernetes.io/part-of: ingress-nginx spec: limits: - default: min: memory: 90Mi cpu: 100m type: Container
提示：把對應控制器類型更改外DaemonSet，在pod模板中spec字段下的replicas字段去掉；在spec.template.spec字段下加上hostNetwork: true即可；以上兩種使用ds控制器管理ingress controller pod也可以使用node選擇器，來篩選在某個節點上創建ingress controller pod；
使用deploy控制器管理ingress controller pod，就直接應用mandatory.yaml即可
[root@master01 ~]# kubectl apply -f mandatory.yaml namespace/ingress-nginx created configmap/nginx-configuration created configmap/tcp-services created configmap/udp-services created serviceaccount/nginx-ingress-serviceaccount created Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created Warning: rbac.authorization.k8s.io/v1beta1 Role is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 Role role.rbac.authorization.k8s.io/nginx-ingress-role created Warning: rbac.authorization.k8s.io/v1beta1 RoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 RoleBinding rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created Warning: rbac.authorization.k8s.io/v1beta1 ClusterRoleBinding is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRoleBinding clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created deployment.apps/nginx-ingress-controller created limitrange/ingress-nginx created [root@master01 ~]#
查看應用資源清單創建的資源對象
[root@master01 ~]# kubectl get all -n ingress-nginx NAME READY STATUS RESTARTS AGE pod/nginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 80s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/nginx-ingress-controller 1/1 1 1 80s NAME DESIRED CURRENT READY AGE replicaset.apps/nginx-ingress-controller-5466cb8999 1 1 1 80s [root@master01 ~]#
提示：可以看到在ingress-nginx名稱空間下創建了一個deploy控制器，對應控制器創建了一個nginx-ingress-controller控制器pod；但是此pod現在不能被外部客戶端訪問到，我們需要創建一個service來引入外部流量到此pod上；
查看pod標簽
[root@master01 ~]# kubectl get pod -n ingress-nginx --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 4m38s app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=5466cb8999 [root@master01 ~]#
根據上述標簽來寫一個創建ingress-service資源的配置清單
[root@master01 ~]# cat ingress-nginx-service.yaml
apiVersion: v1
kind: Service
metadata:
name: ingress-nginx-svc
namespace: ingress-nginx
spec:
type: NodePort
ports:
- port: 80
name: http
nodePort: 30080
- port: 443
name: https
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
[root@master01 ~]#
提示：以上配置清單主要把滿足對應標簽選擇器的pod關聯起來；並把對應pod的80和443端口分別映射到對應主機上的30080和30443端口；
應用配置清單
[root@master01 ~]# kubectl apply -f ingress-nginx-service.yaml service/ingress-nginx-svc created [root@master01 ~]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-svc NodePort 10.98.4.208 <none> 80:30080/TCP,443:30443/TCP 13s [root@master01 ~]#
訪問集群任意節點ip的30080和30443端口，看看是否訪問到對應pod？
提示：30080是能夠正常訪問的，只是它顯示404，是因為我們沒有對應的主頁；
訪問30443端口
提示：30443是一個https端口，所以訪問必須用https協議訪問，這里提示訪問頁面有風險是因為瀏覽器不信任證書引起的，我們可以點擊高級，信任證書即可；同樣30443端口也是返回404，是因為沒有主頁的原因；兩個端口能夠正常訪問，說明我們在k8s上部署的ingress-nginx controller就部署好了；
ingress資源的使用
在k8s上創建一個deploy控制器，讓其管理2個 ikubernetes/myapp:v1鏡像運行的pod，然后再創建一個對應的service
[root@master01 manifests]# cat myapp-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: myapp
rel: stable
template:
metadata:
namespace: default
labels:
app: myapp
rel: stable
spec:
containers:
- name: myapp
image: ikubernetes/myapp:v1
---
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: default
spec:
selector:
app: myapp
rel: stable
ports:
- name: http
port: 80
targetPort: 80
[root@master01 manifests]#
提示：一個清單中定義多個資源，需要用“---”來分割資源；
應用資源清單
[root@master01 manifests]# kubectl apply -f myapp-demo.yaml deployment.apps/myapp created service/myapp created [root@master01 manifests]# kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES myapp-6479b786f5-9d4mh 1/1 Running 0 11s 10.244.2.98 node02.k8s.org <none> <none> myapp-6479b786f5-k252c 1/1 Running 0 11s 10.244.4.20 node04.k8s.org <none> <none> [root@master01 manifests]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h52m myapp ClusterIP 10.105.208.218 <none> 80/TCP 21s [root@master01 manifests]# kubectl describe svc myapp Name: myapp Namespace: default Labels: <none> Annotations: <none> Selector: app=myapp,rel=stable Type: ClusterIP IP Families: <none> IP: 10.105.208.218 IPs: 10.105.208.218 Port: http 80/TCP TargetPort: 80/TCP Endpoints: 10.244.2.98:80,10.244.4.20:80 Session Affinity: None Events: <none> [root@master01 manifests]#
創建ingress資源來反代以上資源
示例：創建ingress資源
[root@master01 manifests]# cat ingress-myapp.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: www.myapp.com
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: 80
[root@master01 manifests]#
提示：創建ingress資源apiVersion的值要寫成extensions/v1beta1，kind為Ingress；對應metadata中的annotations的配置表示把ingress資源通知給那個類別的ingress controller，如果k8s集群上有多個類別的ingress controller時，這一項特別有用；在spec字段主要內嵌了三個字段，rules字段用來定義反代規則列表，其值為一個對象列表；其中rules字段里主要host和http字段；host用來指定虛擬主機的fqdn名稱，如果不寫表示匹配任意虛擬主機名稱；http是用來定義指向后端的http選擇器列表；其值為一個對象，里面只有一個paths字段，用於指定把請求映射到后端的某個路徑；其值為一個對象列表；對應paths字段中可以定義path，用來指定映射后端的路徑；backend用於指定后端pod的service，其值為一個對象；serviceName用於指定對應pod的service名稱；servicePort用於指定后端服務的端口；以上配置表示把www.myapp.com這個虛擬主機的訪問全部反代至服務名稱為myapp端口為80的pod上；
應用配置清單
[root@master01 manifests]# kubectl apply -f ingress-myapp.yaml Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress ingress.extensions/ingress-myapp created [root@master01 manifests]# kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE ingress-myapp <none> www.myapp.com 80 29s [root@master01 manifests]#
查看ingress資源的詳細信息
[root@master01 manifests]# kubectl describe ingress ingress-myapp
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
www.myapp.com
/ myapp:80 (10.244.2.98:80,10.244.4.20:80)
Annotations: kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 81s nginx-ingress-controller Ingress default/ingress-myapp
[root@master01 manifests]#
提示：可以看到對應滿足service名稱為myapp並且其端口為80的pod有兩個；
進入ingress controller pod里，看看對應配置文件是否有www.myapp.com的配置？
[root@master01 manifests]# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-5466cb8999-4lsjc 1/1 Running 0 78m
[root@master01 manifests]# kubectl exec -it -n ingress-nginx pod/nginx-ingress-controller-5466cb8999-4lsjc -- /bin/sh
/etc/nginx $ cd /etc/nginx/
/etc/nginx $ ls
fastcgi.conf koi-utf modsecurity owasp-modsecurity-crs uwsgi_params.default
fastcgi.conf.default koi-win modules scgi_params win-utf
fastcgi_params lua nginx.conf scgi_params.default
fastcgi_params.default mime.types nginx.conf.default template
geoip mime.types.default opentracing.json uwsgi_params
/etc/nginx $ grep "www.myapp.com" nginx.conf
## start server www.myapp.com
server_name www.myapp.com ;
## end server www.myapp.com
/etc/nginx $
提示：可以看到在對應ingress-nginx 控制器pod中能夠搜索到www.myapp.com的配置；說明我們定義的ingress資源已經被ingress-nginx controller 捕獲；
用瀏覽器訪問www.myapp.com看看是否能夠訪問到內容？
提示：使用www.myapp.com訪問，需要確保對應域名能夠正常解析到k8s集群任意一節點上；可以看到訪問www.myapp.com:30080能夠訪問到對應pod內容；
刪除ingress代理規則
[root@master01 manifests]# kubectl delete -f ingress-myapp.yaml Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress ingress.extensions "ingress-myapp" deleted [root@master01 manifests]# kubectl get ingress No resources found in default namespace. [root@master01 manifests]#
示例：配置基於url路徑進行流量分發
[root@master01 manifests]# cat ingress-myapp1.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: www.myapp.com
http:
paths:
- path: /bbs
backend:
serviceName: myapp
servicePort: 80
- path: /blog
backend:
serviceName: myapp
servicePort: 80
[root@master01 manifests]#
提示：以上配置表示把www.myapp.com/bbs反代到service名稱為myapp並且端口為80的pod上；把www.myapp.com/blog反代到ervice名稱為myapp並且端口為80的pod上；我這里是因為k8s上只有這一種應用，生成環境中按照對應的業務邏輯來反代對應url到對應pod上即可；
應用配置清單
[root@master01 manifests]# kubectl apply -f ingress-myapp1.yaml
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.extensions/ingress-myapp created
[root@master01 manifests]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-myapp <none> www.myapp.com 80 5s
[root@master01 manifests]# kubectl describe ingress ingress-myapp
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
www.myapp.com
/bbs myapp:80 (10.244.2.98:80,10.244.4.20:80)
/blog myapp:80 (10.244.2.98:80,10.244.4.20:80)
Annotations: ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 30s nginx-ingress-controller Ingress default/ingress-myapp
[root@master01 manifests]#
提示：可以看到對應ingress上就有兩個url分別指向后端service名稱為myapp端口為80的pod上；
訪問對應url，看看是否訪問到內容？
提示：這里訪問不到內容的原因是對應pod內部並沒有對應url的頁面；
進入ingress controller pod內部，查看是否有對應配置？
提示：可以看到對應在ingress中定義的配置，都轉為對應該ingress controller pod中的配置，說明我們定義基於url分發流量的ingress沒有問題；
示例：定義ingress規則基於主機名稱的虛擬主機
[root@master01 manifests]# cat ingress-myapp2.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: www.myapp.com
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
- host: blog.myapp.com
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
[root@master01 manifests]#
提示：以上配置表示把www.myapp.com這個虛擬主機名稱的訪問流量分發至service名稱為myapp端口為80的pod上；把blog.myapp.com的流量分發至至service名稱為myapp端口為80的pod上；生成環境按照對應的service名稱來分發即可；
應用配置清單
[root@master01 manifests]# kubectl apply -f ingress-myapp2.yaml
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.extensions/ingress-myapp created
[root@master01 manifests]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-myapp <none> www.myapp.com,blog.myapp.com 80 16s
[root@master01 manifests]# kubectl describe ingress ingress-myapp
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
www.myapp.com
myapp:80 (10.244.2.98:80,10.244.4.20:80)
blog.myapp.com
myapp:80 (10.244.2.98:80,10.244.4.20:80)
Annotations: ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 32s nginx-ingress-controller Ingress default/ingress-myapp
[root@master01 manifests]#
驗證配置信息
訪問對應虛擬主機，看看是否能夠訪問對應pod？
提示：可以看到兩個虛擬主機名稱都可以正常訪問到，對應也做了調度；
示例：創建tls類型的ingress資源
創建證書
[root@master01 manifests]# openssl genrsa -out tls.key 2048 Generating RSA private key, 2048 bit long modulus .........................................+++ ........+++ e is 65537 (0x10001) [root@master01 manifests]# openssl req -x509 -key tls.key -out tls.crt -subj /C=CN/ST=SiChuan/L=GuangYuan/O=Test/CN=www.myapp.com -days 3650 [root@master01 manifests]#
提示：以上兩條命令創建了一個名為tls.key的私鑰和一個自簽名證書，其名為tls.crt；
創建Secret資源
[root@master01 manifests]# kubectl create secret tls www-myapp-com-ingress-secret --cert=./tls.crt --key=./tls.key secret/www-myapp-com-ingress-secret created [root@master01 manifests]# kubectl get secret NAME TYPE DATA AGE default-token-xvd4c kubernetes.io/service-account-token 3 13d www-myapp-com-ingress-secret kubernetes.io/tls 2 21s [root@master01 manifests]#
提示：在ingress控制器上配置https主機時，不能直接使用私鑰和證書文件，而是需要使用secret資源對象來傳遞相關數據；
定義tls類型ingress資源清單
[root@master01 manifests]# cat www-myapp-com-ingress-secret.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-myapp-tls
namespace: default
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
tls:
- hosts:
- www.myapp.com
secretName: www-myapp-com-ingress-secret
rules:
- host: www.myapp.com
http:
paths:
- path: /
backend:
serviceName: myapp
servicePort: 80
[root@master01 manifests]#
提示：定義tls類型ingress資源清單，需要在spec字段下用tls字段來指定對應主機名稱，以及secret資源對象的名稱；
應用資源清單
[root@master01 manifests]# kubectl apply -f www-myapp-com-ingress-secret.yaml
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
ingress.extensions/ingress-myapp-tls created
[root@master01 manifests]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress-myapp <none> www.myapp.com,blog.myapp.com 80 31m
ingress-myapp-tls <none> www.myapp.com 80, 443 8s
[root@master01 manifests]# kubectl describe ingress ingress-myapp-tls
Name: ingress-myapp-tls
Namespace: default
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
www-myapp-com-ingress-secret terminates www.myapp.com
Rules:
Host Path Backends
---- ---- --------
www.myapp.com
/ myapp:80 (10.244.2.98:80,10.244.4.20:80)
Annotations: kubernetes.io/ingress.class: nginx
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 26s nginx-ingress-controller Ingress default/ingress-myapp-tls
[root@master01 manifests]#
驗證：訪問對應虛擬主機名稱，看看對應的https端口是否能夠正常訪問到內容？
提示：可以看到使用https協議訪問對應的30443端口能夠正常訪問到對應后端pod提供的內容；