(八)Kubernetes Ingress資源

前言

Kubernetes提供了兩種內建的雲端負載均衡機制（cloud load balancing）用於發布公共應用，一種是工作於傳輸層的Service資源，它實現的是“TCP負載均衡器”，另一種是Ingress資源，它實現的是“HTTP(S)負載均衡器”。

• TCP負載均衡器

  無論是iptables還是ipvs模型的Service資源都配置於Linux內核中的Netfilter之上進行四層調度，是一種類型更為通用的調度器，支持調度HTTP、MySQL等應用層服務。不過，也正是由於工作於傳輸層從而使得它無法做到類似卸載HTTPS中的SSL會話等一類操作，也不支持基於URL的請求調度機制，而且，Kubernetes也不支持為此類負載均衡器配置任何類型的健康狀態檢查機制。

• HTTP(S)負載均衡器

  HTTP(S)負載均衡器是應用層負載均衡機制的一種，支持根據環境做出更好的調度決策。與傳輸層調度器相比，它提供了諸如可自定義URL映射和TLS卸載等功能，並支持多種類型的后端服務器健康狀態檢查機制。

Ingress概述

什么是Ingress？

通常情況下，service和pod僅可在集群內部網絡中通過IP地址訪問。所有到達邊界路由器的流量或被丟棄或被轉發到其他地方。從概念上講，可能像下面這樣：

 internet
     |
------------
[ Services ]

Ingress是授權入站連接到達集群服務的規則集合。

 internet
     |
[ Ingress ]
--|-----|--
[ Services ]

你可以給Ingress配置提供外部可訪問的URL、負載均衡、SSL、基於名稱的虛擬主機等。用戶通過POST Ingress資源到API Server的方式來請求Ingress。Ingress controller負責實現Ingress，通常使用負載平衡器，它還可以配置邊界路由和其他前端，這有助於以HA方式處理流量。

Ingress和Ingress Controller

Ingress是Kubernetes API的標准資源類型之一，它其實就是一組基於DNS名稱（host）或URL路徑把請求轉發至指定的Service資源的規則，用於將集群外部的請求流量轉發至集群內部完成服務發布。然而，Ingress資源自身並不能進行“流量穿透”，它僅是一組路由規則的集合，這些規則要想真正發揮作用還需要其他功能的輔助，如監聽某套接字，然后根據這些規則的匹配機制路由請求流量。這種能夠為Ingress資源監聽套接字並轉發流量的組件稱為Ingress控制器（Ingress Controller）。

Ingress控制器並不直接運行為kube-controller-manager的一部分，它是Kubernetes集群的一個重要組件，類似CoreDNS，需要在集群上單獨部署。

Ingress工作流程

如下圖所示，流量到達外部負載均衡器（externalLB）后，首先轉發至Service資源Ingres-nginx上，然后通過Ingress控制器基於Ingress資源定義的規則將客戶端請求流量直接轉發至與Service對應的后端Pod資源之上。這種轉發機制會繞過Service資源(app Service；api Service)，從而省去了由kube-proxy實現的端口代理開銷。Ingress規則需要由一個Service資源對象輔助識別相關的所有Pod資源。如下Ingress通過app service資源去匹配后端的pod1和pod2；這個app service只是起到一個輔助識別功能。

先決條件

在使用Ingress resource之前，必須先了解下面幾件事情。Ingress是beta版本的resource，在kubernetes1.1之前還沒有。你需要一個Ingress Controller來實現Ingress，單純的創建一個Ingress沒有任何意義。

GCE/GKE會在master節點上部署一個Ingress Controller。你可以在一個Pod中部署任意個自定義的Ingress Controller。你必須正確的annotate每個Ingress，比如運行多個Ingress Controller和關閉glbc。

Ingress清單文件幾個字段說明

Ingress資源是基於HTTP虛擬主機或URL的轉發規則，spec字段中嵌套了rules、backend、tls等字段進行定義。下面這個示例中，它包含了一個轉發規則，把發往www.ilinux.io的請求代理給名為myapp-svc的Service資源。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-demo
  namespace: default
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: www.ilinux.io
    http:
      paths:
      - backend:
          serviceName: myapp-svc
          servicePort: 80

#說明：上面資源清單文件中的annotations用於識別其所屬的Ingress控制器的類別，這一點在集群上部署多個Ingress控制器時尤為重要。

Ingress Spec（# kubectl explain ingress.spec）中的字段是定義Ingress資源的核心組成部分，主要嵌套如下三個字段：

• rules <[]Object>：用於定義當前Ingress資源的轉發規則列表；未由rules定義規則，或者沒有匹配到任何規則時，所有流量都會轉發到由backend定義的默認后端。

• backend <Object>：默認的后端用於服務那些沒有匹配到任何規則的請求；定義Ingress資源時，至少應該定義backend或rules兩者之一；此字段用於讓負載均衡器指定一個全局默認的后端。

• tls <[]Object>：TLS配置，目前僅支持通過默認端口443提供服務；如果要配置指定的列表成員指向了不同的主機，則必須通過SNI TLS擴展機制來支持此功能。

ingress.spec.rules.http.paths.backend對象的定義由兩個必須的內嵌字段組成：serviceName和servicePort，分別用於指定流量轉發的后端目標Service資源的名稱和端口。

部署Ingress Controller（Nginx）

描述

Ingress 控制器自身是運行於Pod中的容器應用，一般是Nginx或Envoy一類的具有代理及負載均衡功能的守護進程，它監視着來自API Server的Ingress對象狀態，並根據規則生成相應的應用程序專有格式的配置文件並通過重載或重啟守護進程而使新配置生效。

Ingress控制器其實就是托管於Kubernetes系統之上的用於實現在應用層發布服務的Pod資源，跟蹤Ingress資源並實時生成配置規則。

運行為Pod資源的Ingress控制器進程通過下面兩種方式接入外部請求流量：

1、以Deployment控制器管理Ingress控制器的Pod資源，通過NodePort或LoadBalancer類型的Service對象為其接入集群外部的請求流量，這就意味着，定義一個Ingress控制器時，必須在其前端定義一個專用的Service資源。

2、借助於DaemonSet控制器，將Ingress控制器的Pod資源各自以單一實例的方式運行於集群的所有或部分工作節點之上，並配置這類Pod對象以HostPort（如下圖中的a）或HostNetwork（如下圖中的b）的方式在當前節點接入外部流量。

部署

Ingress-nginx官網

Ingress-nginx GitHub倉庫地址

Ingress安裝文檔

1）在github上下載配置清單yaml文件，並創建部署

[root@k8s-master ~]# mkdir ingress-nginx   #這里創建一個目錄專門用於ingress-nginx(可省略)
[root@k8s-master ~]# cd ingress-nginx/
[root@k8s-master ingress-nginx]# wget  https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml    #下載配置清單yaml文件
[root@k8s-master ingress-nginx]# ls    #查看下載的文件
mandatory.yaml

[root@k8s-master ingress-nginx]# kubectl apply -f mandatory.yaml    #創建Ingress
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created

2）驗證

[root@k8s-master ingress-nginx]# kubectl get pods -n ingress-nginx    #查看生成的pod，注意這里在ingress-nginx名稱空間
NAME                                        READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-79f6884cf6-5fb6v   1/1     Running   0          18m
[root@k8s-master ingress-nginx]# kubectl describe pod nginx-ingress-controller-79f6884cf6-5fb6v -n ingress-nginx    查看該pod的詳細信息
Name:           nginx-ingress-controller-79f6884cf6-5fb6v
Namespace:      ingress-nginx
Priority:       0
Node:           k8s-node2/192.168.1.33
Start Time:     Fri, 27 Sep 2019 17:53:07 +0800
Labels:         app.kubernetes.io/name=ingress-nginx
                app.kubernetes.io/part-of=ingress-nginx
                pod-template-hash=79f6884cf6
Annotations:    prometheus.io/port: 10254
                prometheus.io/scrape: true
Status:         Running
IP:             10.244.2.73
......

3）如果是裸機部署，還需要安裝service。（比如VMware虛擬機、硬件服務器等）

---同樣去官網下載配置清單文件，也可以自定義創建。
[root@k8s-master ingress-nginx]# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml
[root@k8s-master ingress-nginx]# kubectl apply -f service-nodeport.yaml    #創建service資源
service/ingress-nginx created
[root@k8s-master ingress-nginx]# kubectl get svc -n ingress-nginx    #查看service資源
NAME            TYPE       CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.107.40.182   <none>        80:32699/TCP,443:30842/TCP   9s
[root@k8s-master ingress-nginx]# kubectl describe svc/ingress-nginx -n ingress-nginx    #查看該service的詳細信息
Name:                     ingress-nginx
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/part-of=ingress-nginx
Annotations:              kubectl.kubernetes.io/last-applied-configuration:
                            {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app.kubernetes.io/name":"ingress-nginx","app.kubernetes.io/par...
Selector:                 app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type:                     NodePort
IP:                       10.107.40.182
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  32699/TCP
Endpoints:                10.244.2.73:80
Port:                     https  443/TCP
TargetPort:               443/TCP
NodePort:                 https  30842/TCP
Endpoints:                10.244.2.73:443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

通過上面創建的service資源對象可以看出，隨機分配的http的NodePort為32668，https的NodePort的為30606。該端口也可以自定義，在前面的service章節說過。單一般不建議自定義。

示例1：使用Ingress發布Nginx

該示例中創建的所有資源都位於新建的testing名稱空間中。與其他的資源在邏輯上進行隔離，以方便管理。

首先創建一個單獨的目錄為了方便管理

[root@k8s-master ~]# mkdir ingress-nginx/ingress
[root@k8s-master ~]# cd ingress-nginx/ingress/

（1）、創建testing名稱空間(也可以使用命令直接創建# kubectl create namespace my-namespace，不過這里使用資源清單格式創建)

[root@k8s-master ingress]# vim namespace-testing.yaml    #編寫namespace清單文件
apiVersion: v1
kind: Namespace
metadata:
  name: testing
  labels:
    env: testing
[root@k8s-master ingress]#
[root@k8s-master ingress]# kubectl apply -f namespace-testing.yaml    #創建namespace
namespace/testing created
[root@k8s-master ingress]#
[root@k8s-master ingress]# kubectl get namespace testing    #驗證
NAME      STATUS   AGE
testing   Active   12s

（2）、部署nginx實例，這里使用Deployment控制器於testing中部署nginx相關的Pod對象。

[root@k8s-master ingress]# vim deployment-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: deploy-nginx
  namespace: testing
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.12
        ports:
        - name: http
          containerPort: 80
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl apply -f deployment-nginx.yaml 
deployment.apps/deploy-nginx created
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl get deploy -n testing
NAME           READY   UP-TO-DATE   AVAILABLE   AGE
deploy-nginx   3/3     3            3           5s
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl get pods -n testing
NAME                            READY   STATUS    RESTARTS   AGE
deploy-nginx-686bddcb56-9g7pq   1/1     Running   0          6s
deploy-nginx-686bddcb56-gqpm2   1/1     Running   0          6s
deploy-nginx-686bddcb56-vtwkq   1/1     Running   0          6s

（3）、創建Service資源，關聯后端的Pod資源。這里通過service資源svc-nginx的80端口去暴露容器的80端口。

[root@k8s-master ingress]# vim service-nginx.yaml
apiVersion: v1
kind: Service
metadata:
  name: svc-nginx
  namespace: testing
  labels:
    app: svc-nginx
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl apply -f service-nginx.yaml 
service/svc-nginx created
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl get svc -n testing
NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
svc-nginx   ClusterIP   10.99.233.90   <none>        80/TCP           6s
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl describe svc/svc-nginx -n testing
Name:              svc-nginx
Namespace:         testing
Labels:            app=svc-nginx
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"svc-nginx"},"name":"svc-nginx","namespace":"testing"},"s...
Selector:          app=nginx
Type:              ClusterIP
IP:                10.99.233.90
Port:              http  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.76:80,10.244.1.77:80,10.244.2.74:80
Session Affinity:  None
Events:            <none>

（4）、創建Ingress資源，匹配Service資源svc-nginx，並將svc-nginx的80端口暴露。

[root@k8s-master ingress]# vim ingress-nginx.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  namespace: testing
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: nginx.ilinux.io
    http:
      paths:
      - path:
        backend:
          serviceName: svc-nginx
          servicePort: 80
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl apply -f ingress-nginx.yaml 
ingress.extensions/nginx created
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl get ingress -n testing
NAME    HOSTS              ADDRESS   PORTS   AGE
nginx   nginx.ilinux.io             80      16s
[root@k8s-master ingress]# 
[root@k8s-master ingress]# kubectl describe ingress -n testing
Name:             nginx
Namespace:        testing
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host              Path  Backends
  ----              ----  --------
  tomcat.ilinux.io  
                       svc-nginx:80 (10.244.1.76:80,10.244.1.77:80,10.244.2.74:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"nginx","namespace":"testing"},"spec":{"rules":[{"host":"nginx.ilinux.io","http":{"paths":[{"backend":{"serviceName":"svc-nginx","servicePort":80},"path":null}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:                         <none>

（5）、測試，通過Ingress控制器的前端的Service資源的NodePort來訪問此服務，

#首先查看前面部署Ingress控制器的前端的Service資源的映射端口
[root@k8s-master ingress-nginx]# kubectl get svc -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.107.40.182   <none>        80:32699/TCP,443:30842/TCP   3m59s

#終端測試，添加hosts
[root@k8s-master ~]# cat /etc/hosts
192.168.1.31    k8s-master nginx.ilinux.io
192.168.1.32    k8s-node1 nginx.ilinux.io
192.168.1.33    k8s-node2 nginx.ilinux.io
#訪問測試
[root@k8s-master ~]# curl nginx.ilinux.io:32699
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
......

驗證是否調度到后端的Pod資源，查看日志

[root@k8s-master ~]# kubectl get pods -n testing
NAME                            READY   STATUS    RESTARTS   AGE
deploy-nginx-686bddcb56-9g7pq   1/1     Running   0          56m
deploy-nginx-686bddcb56-gqpm2   1/1     Running   0          56m
deploy-nginx-686bddcb56-vtwkq   1/1     Running   0          56m
[root@k8s-master ~]# kubectl logs deploy-nginx-686bddcb56-9g7pq -n testing
10.244.2.75 - - [28/Sep/2019:02:33:45 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "10.244.0.0"
10.244.2.75 - - [28/Sep/2019:02:44:02 +0000] "GET / HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36" "10.244.0.0"

（6）、配置TLS Ingress資源（這里使用自簽證書）

1)生成key
[root@k8s-master ingress]# openssl genrsa -out tls.key 2048
2)生成證書
[root@k8s-master ingress]# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=ShenZhen/L=ShenZhen/O=DevOps/CN=nginx.ilinux.io -days 3650

3)創建secret資源
[root@k8s-master ingress]# kubectl create secret tls nginx-ingress-secret --cert=tls.crt --key=tls.key -n testing
secret/nginx-ingress-secret created
[root@k8s-master ingress]# kubectl get secret -n testing
NAME                   TYPE                                  DATA   AGE
default-token-lfzrt    kubernetes.io/service-account-token   3      116m
nginx-ingress-secret   kubernetes.io/tls                     2      16s

4)編寫Ingress資源清單文件
[root@k8s-master ingress]# vim ingress-nginx-https.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-ingress-tls
  namespace: testing
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - nginx.ilinux.io
    secretName: nginx-ingress-secret
  rules:
  - host: nginx.ilinux.io
    http:
      paths:
      - path: /
        backend:
          serviceName: svc-nginx
          servicePort: 80

5)查看Ingress資源信息
[root@k8s-master ingress]# kubectl get ingress -n testing
NAME                HOSTS             ADDRESS   PORTS     AGE
nginx               nginx.ilinux.io             80        66m
nginx-ingress-tls   nginx.ilinux.io             80, 443   15s
[root@k8s-master ingress]# kubectl describe ingress/nginx-ingress-tls -n testing
Name:             nginx-ingress-tls
Namespace:        testing
Address:          
Default backend:  default-http-backend:80 (<none>)
TLS:
  nginx-ingress-secret terminates nginx.ilinux.io
Rules:
  Host             Path  Backends
  ----             ----  --------
  nginx.ilinux.io  
                   /   svc-nginx:80 (10.244.1.76:80,10.244.1.77:80,10.244.2.74:80)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"nginx-ingress-tls","namespace":"testing"},"spec":{"rules":[{"host":"nginx.ilinux.io","http":{"paths":[{"backend":{"serviceName":"svc-nginx","servicePort":80},"path":"/"}]}}],"tls":[{"hosts":["nginx.ilinux.io"],"secretName":"nginx-ingress-secret"}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  64s   nginx-ingress-controller  Ingress testing/nginx-ingress-tls

（7）、測試https（這里由於是自簽，所以上面提示不安全）

#首先查看前面部署Ingress控制器的前端的Service資源的映射端口
[root@k8s-master ingress-nginx]# kubectl get svc -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.107.40.182   <none>        80:32699/TCP,443:30842/TCP   3m59s

示例2：使用Ingress發布多個服務

將不同的服務映射不同的主機上

准備工作：這里創建一個目錄保存本示例的所有資源配置清單

[root@k8s-master ~]# mkdir ingress-nginx/multi_svc
[root@k8s-master ~]# cd !$

創建名稱空間

創建一個名稱空間保存本示例的所有對象（方便管理）

[root@k8s-master multi_svc]# vim namespace-ms.yaml    #編寫配置清單文件
apiVersion: v1
kind: Namespace
metadata:
  name: multisvc 
  labels:
    env: multisvc

[root@k8s-master multi_svc]# kubectl apply -f namespace-ms.yaml     #創建上面定義的名稱空間
namespace/multisvc created

[root@k8s-master multi_svc]# kubectl get namespace multisvc    #查看名稱空間
NAME       STATUS   AGE
multisvc   Active   9s

創建后端應用和Service

這里后端應用創建為一組nginx應用和一組tomcat應用

1）編寫資源清單文件，這里將service資源對象和deployment控制器寫在這一個文件里

[root@k8s-master multi_svc]# vim deploy_service-ms.yaml
#tomcat應用的Deployment控制器
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deploy
  namespace: multisvc
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat
  template:
    metadata:
      labels: 
        app: tomcat
    spec:
      containers:
      - name: tomcat
        image: tomcat:jdk8
        imagePullPolicy: IfNotPresent
        ports:
        - name: httpport 
          containerPort: 8080
        - name: ajpport
          containerPort: 8009
---
#tomcat應用的Service資源
apiVersion: v1
kind: Service
metadata:
  name: tomcat-svc
  namespace: multisvc
  labels:
    app: tomcat-svc
spec:
  selector:
    app: tomcat
  ports:
  - name: httpport
    port: 8080
    targetPort: 8080
    protocol: TCP
  - name: ajpport
    port: 8009
    targetPort: 8009
    protocol: TCP

---
#nginx應用的Deployment控制器
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deploy
  namespace: multisvc
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels: 
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.12
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 80
---
#nginx應用的Service資源
apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
  namespace: multisvc
  labels:
    app: nginx-svc
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
    protocol: TCP

2）創建上面定義資源對象並查看驗證

[root@k8s-master multi_svc]# kubectl apply -f deploy_service-ms.yaml 
deployment.apps/tomcat-deploy created
service/tomcat-svc created
deployment.apps/nginx-deploy created
service/nginx-svc created
[root@k8s-master multi_svc]# kubectl get pods -n multisvc -o wide    #查看pod資源
NAME                             READY   STATUS    RESTARTS   AGE   IP            NODE        NOMINATED NODE   READINESS GATES
nginx-deploy-86c667ff66-hl6rx    1/1     Running   0          13s   10.244.2.78   k8s-node2   <none>           <none>
nginx-deploy-86c667ff66-hx4j8    1/1     Running   0          13s   10.244.2.77   k8s-node2   <none>           <none>
nginx-deploy-86c667ff66-tl9mm    1/1     Running   0          13s   10.244.1.79   k8s-node1   <none>           <none>
tomcat-deploy-6484688ddc-n25hn   1/1     Running   0          13s   10.244.1.78   k8s-node1   <none>           <none>
tomcat-deploy-6484688ddc-s8dts   1/1     Running   0          13s   10.244.1.80   k8s-node1   <none>           <none>
tomcat-deploy-6484688ddc-snszk   1/1     Running   0          13s   10.244.2.76   k8s-node2   <none>           <none>
[root@k8s-master multi_svc]# kubectl get svc -n multisvc    #查看service資源對象
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
nginx-svc    ClusterIP   10.104.213.237   <none>        80/TCP              26s
tomcat-svc   ClusterIP   10.103.75.161    <none>        8080/TCP,8009/TCP   26s

[root@k8s-master multi_svc]# kubectl describe svc/nginx-svc -n multisvc    #查看service對象nginx-svc的詳細信息
Name:              nginx-svc
Namespace:         multisvc
Labels:            app=nginx-svc
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"nginx-svc"},"name":"nginx-svc","namespace":"multisvc"},"...
Selector:          app=nginx
Type:              ClusterIP
IP:                10.104.213.237
Port:              http  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.79:80,10.244.2.77:80,10.244.2.78:80
Session Affinity:  None
Events:            <none>

[root@k8s-master multi_svc]# kubectl describe svc/tomcat-svc -n multisvc    #查看service對象tomcat-svc的詳細信息
Name:              tomcat-svc
Namespace:         multisvc
Labels:            app=tomcat-svc
Annotations:       kubectl.kubernetes.io/last-applied-configuration:
                     {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"labels":{"app":"tomcat-svc"},"name":"tomcat-svc","namespace":"multisvc"}...
Selector:          app=tomcat
Type:              ClusterIP
IP:                10.103.75.161
Port:              httpport  8080/TCP
TargetPort:        8080/TCP
Endpoints:         10.244.1.78:8080,10.244.1.80:8080,10.244.2.76:8080
Port:              ajpport  8009/TCP
TargetPort:        8009/TCP
Endpoints:         10.244.1.78:8009,10.244.1.80:8009,10.244.2.76:8009
Session Affinity:  None
Events:            <none>

創建Ingress資源對象

1）編寫資源清單文件

[root@k8s-master multi_svc]# vim ingress_host-ms.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: multi-ingress
  namespace: multisvc
spec:
  rules:
  - host: nginx.imyapp.com
    http:
      paths: 
      - path: /
        backend:
          serviceName: nginx-svc
          servicePort: 80
  - host: tomcat.imyapp.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat-svc
          servicePort: 8080

2）創建上面定義資源對象並查看驗證

[root@k8s-master multi_svc]# kubectl apply -f ingress_host-ms.yaml 
ingress.extensions/multi-ingress created
[root@k8s-master multi_svc]# kubectl get ingress -n multisvc    #查看ingress資源對象
NAME            HOSTS                                ADDRESS   PORTS   AGE
multi-ingress   nginx.imyapp.com,tomcat.imyapp.com             80      18s

[root@k8s-master multi_svc]# kubectl describe ingress/multi-ingress -n multisvc    #查看ingress資源multi-ingrsss的詳細信息
Name:             multi-ingress
Namespace:        multisvc
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host               Path  Backends
  ----               ----  --------
  nginx.imyapp.com   
                     /   nginx-svc:80 (10.244.1.79:80,10.244.2.77:80,10.244.2.78:80)
  tomcat.imyapp.com  
                     /   tomcat-svc:8080 (10.244.1.78:8080,10.244.1.80:8080,10.244.2.76:8080)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{},"name":"multi-ingress","namespace":"multisvc"},"spec":{"rules":[{"host":"nginx.imyapp.com","http":{"paths":[{"backend":{"serviceName":"nginx-svc","servicePort":80},"path":"/"}]}},{"host":"tomcat.imyapp.com","http":{"paths":[{"backend":{"serviceName":"tomcat-svc","servicePort":8080},"path":"/"}]}}]}}

Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  39s   nginx-ingress-controller  Ingress multisvc/multi-ingress

測試訪問

這是測試自定義的域名，故需要配置host

192.168.1.31     nginx.imyapp.com tomcat.imyapp.com
192.168.1.32     nginx.imyapp.com tomcat.imyapp.com
192.168.1.33     nginx.imyapp.com tomcat.imyapp.com

查看部署的Ingress的Service對象的端口

[root@k8s-master multi_svc]# kubectl get svc -n ingress-nginx
NAME            TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.107.40.182   <none>        80:32699/TCP,443:30842/TCP   6h39m

訪問nginx.imyapp.com:32699

訪問tomcat.imyapp.com:32699

配置Ingress處理TLS傳輸

這里使用自簽證書，通過OpenSSL進行創建

1）創建證書

#創建nginx.imyapp.com域名的證書
[root@k8s-master multi_svc]# openssl genrsa -out nginx.imyapp.com.key 2048
[root@k8s-master multi_svc]# openssl req -new -x509 -key nginx.imyapp.com.key -out nginx.imyapp.com.crt -subj /C=CN/ST=ShenZhen/L=ShenZhen/O=DevOps/CN=nginx.imyapp.com -days 3650

#創建tomcat.imyapp.com域名的證書
[root@k8s-master multi_svc]# openssl genrsa -out tomcat.imyapp.com.key 2048
[root@k8s-master multi_svc]# openssl req -new -x509 -key tomcat.imyapp.com.key -out tomcat.imyapp.com.crt -subj /C=CN/ST=ShenZhen/L=ShenZhen/O=DevOps/CN=tomcat.imyapp.com -days 3650

#查看生成的證書
[root@k8s-master multi_svc]# ll *.com.*
-rw-r--r-- 1 root root 1298 9月  28 17:23 nginx.imyapp.com.crt
-rw-r--r-- 1 root root 1675 9月  28 17:22 nginx.imyapp.com.key
-rw-r--r-- 1 root root 1302 9月  28 17:24 tomcat.imyapp.com.crt
-rw-r--r-- 1 root root 1679 9月  28 17:24 tomcat.imyapp.com.key

2）創建secrte

#創建nginx域名的secret
[root@k8s-master multi_svc]# kubectl create secret tls nginx-ingress-secret --cert=nginx.imyapp.com.crt --key=nginx.imyapp.com.key -n multisvc
secret/nginx-ingress-secret created

#創建tomcat域名的secret
[root@k8s-master multi_svc]# kubectl create secret tls tomcat-ingress-secret --cert=tomcat.imyapp.com.crt --key=tomcat.imyapp.com.key -n multisvc
secret/tomcat-ingress-secret created

#查看secret
[root@k8s-master multi_svc]# kubectl get secret -n multisvc
NAME                    TYPE                                  DATA   AGE
default-token-mf5wd     kubernetes.io/service-account-token   3      5h12m
nginx-ingress-secret    kubernetes.io/tls                     2      53s
tomcat-ingress-secret   kubernetes.io/tls                     2      27s

3）編寫帶TLS的Ingress資源清單（這里通過復制，沒有刪除上面創建的ingress）

[root@k8s-master multi_svc]# cp ingress_host-ms.yaml ingress_host_https-ms.yaml
[root@k8s-master multi_svc]# vim ingress_host_https-ms.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: multi-ingress-https
  namespace: multisvc
  annotations:
    kubernetes.io/ingress.class: "nginx"
spec:
  tls:
  - hosts:
    - nginx.imyapp.com
    secretName: nginx-ingress-secret
  - hosts: 
    - tomcat.imyapp.com
    secretName: tomcat-ingress-secret
  rules:
  - host: nginx.imyapp.com
    http:
      paths: 
      - path: /
        backend:
          serviceName: nginx-svc
          servicePort: 80
  - host: tomcat.imyapp.com
    http:
      paths:
      - path: /
        backend:
          serviceName: tomcat-svc
          servicePort: 8080

4）創建ingress資源

[root@k8s-master multi_svc]# kubectl apply -f ingress_host_https-ms.yaml
ingress.extensions/multi-ingress-https created
[root@k8s-master multi_svc]# kubectl get ingress -n multisvc
NAME                  HOSTS                                ADDRESS   PORTS     AGE
multi-ingress         nginx.imyapp.com,tomcat.imyapp.com             80        44m
multi-ingress-https   nginx.imyapp.com,tomcat.imyapp.com             80, 443   3s

5）測試，通過Ingress控制器的前端的Service資源的NodePort來訪問此服務，上面看到ingress控制器的service資源的443端口對應的節點的30842端口。

訪問nginx

訪問tomcat

將不同的服務映射到相同主機的不同路徑

在這種情況下，根據請求的URL中的路徑，請求將發送到兩個不同的服務。因此，客戶端可以通過一個IP地址（Ingress 控制器的IP地址）訪問兩種不同的服務。

注意：這里Ingress中path的定義，需要與后端真實Service提供的Path一致，否則將被轉發到一個不存在的path上，引發錯誤。

Ingress定義示例

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: tomcat-ingress
  namespace: multisvc
spec:
  rules:
  - host: www.imyapp.com
    http:
      paths: 
      - path: /nginx
        backend:
          serviceName: nginx-svc
          servicePort: 80
      - path: /tomcat
        backend:
          serviceName: tomcat-svc
          servicePort: 8080