Kubernetes 服務入口管理 Traefik Ingress Controller

前面部署了 kubernetes/ingress-nginx 作為 Ingress Controller，使用 Nginx 反向代理與負載，通過 Ingress Controller 不斷的跟 Kubernetes API 交互，實時獲取后端 Service、Pod 等的變化，然后動態更新 Nginx 配置，並刷新使配置生效。Traefik 是一個用 Golang 開發的輕量級的 Http 反向代理和負載均衡器軟件，由於可以自動化配置和刷新 backend 節點，目前可以被絕大部分容器平台與組件支持，例如 Docker, Swarm mode, Kubernetes,, Consul, Etcd, Rancher, Eureka 等。Traefik 設計的就能夠實時跟 Kubernetes API 交互，感知后端 Service、Pod 等的變化，自動更新配置並熱重載，使用上大體上差不多，但是 Traefik 更快速更方便，同時支持更多的特性，使反向代理、負載均衡更直接，更高效。

Traefik 特性

• 自動化動態配置無需服務重啟
• 支持多個負載平衡算法
• 支持 Let’s Encrypt (通配符支持) 向您的微服務提供 HTTPS
• 支持熔斷，重試
• 集群模式的高可用性
• 提供簡潔的 UI 界面
• 支持 Websocket, HTTP/2, GRPC 協議
• 提供監控的服務(Rest、Prometheus、Datadog、Statsd、InfluxDB)
• 保留訪問日志（JSON，CLF）
• 快速
• 支持 Rest API
• 使用二進制文件打包，並作為一個 docker 鏡像提供

部署 Traefik

所有的配置文件可以在官方的 github 倉庫中找到,按照官方文檔來即可。

Role Based Access Control configuration (Kubernetes 1.6+ only)

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

$ kubectl apply -f https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-rbac.yaml

Deploy Traefik using a Deployment or DaemonSet

DaemonSet 會在每台 Node 節點上都創建 Pod 而 Deployment  是人為控制的副本數量（根據實際需求來取決），這里使用 DaemonSet 類型來部署 Traefik。

部署 Traefik（修改 hostNetwork: true）

#https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/traefik-ds.yamlvi traefik-ds.yaml

vi traefik-ds.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: DaemonSet
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      hostNetwork: true
      restartPolicy: Always
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: admin
          containerPort: 8080
          hostPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin

部署查看
  
  
  
          
 
           
$ kubectl apply -f traefik-ds.yaml
serviceaccount/traefik-ingress-controller unchanged
daemonset.extensions/traefik-ingress-controller configured
service/traefik-ingress-service unchanged
$ kubectl apply -f traefik-ds.yaml
serviceaccount/traefik-ingress-controller unchanged
daemonset.extensions/traefik-ingress-controller unchanged
service/traefik-ingress-service unchanged
[root@kubernetes-master k8s]# kubectl -n kube-system get pod -o wide
NAME                                        READY   STATUS    RESTARTS   AGE   IP              NODE                NOMINATED NODE
traefik-ingress-controller-6fk9n            1/1     Running   0          41m   10.38.0.0       kubernetes-node-2   <none>
traefik-ingress-controller-f7kmc            1/1     Running   0          41m   10.40.0.1       kubernetes-node-1   <none>
 
          

備注：上述由於修改 hostNetwork: true ，其實已經在每個 Node 節點開放了 80 與 8080 端口，80 提供正常服務，8080 是其自帶的 UI 界面。

Node 節點查看開放的端口

$  netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name      
tcp6       0      0 :::8080                 :::*                    LISTEN      10253/traefik       
tcp6       0      0 :::80                   :::*                    LISTEN      10253/traefik

Ingress 方式暴露 Traefik Web UI

#  https://raw.githubusercontent.com/containous/traefik/master/examples/k8s/ui.yaml
vi traefik-web-ui.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik-ui.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80

部署查看

$ kubectl apply -f traefik-web-ui.yaml
service/traefik-web-ui created
ingress.extensions/traefik-web-ui created
$ kubectl get ingress -o wide --all-namespaces   
NAMESPACE     NAME             HOSTS            ADDRESS   PORTS   AGE
kube-system   traefik-web-ui   traefik-ui.com             80      18s

配置Host文件

172.23.216.49 k8s.dashboard.com
172.23.216.49 traefik-ui.com

訪問 http://traefik-ui.com/dashboard/ 通過 80 端口轉發。

模擬部署一個程序

下面模擬部署一個程序，已 Nginx  為例：

vi nginx-deployment.yaml

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc
spec:
  template:
    metadata:
      labels:
        name: nginx-svc
        namespace: default
spec:
  selector:
    run: nginx-pod
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
---
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  name: nginx-pod
spec:
  replicas: 4
  template:
    metadata:
      labels:
        run: nginx-pod
    spec:
      containers:
      - name: nginx
        image: nginx:1.15.5
        ports:
        - containerPort: 80
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ngx-ing
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: k8s.nginx.com
    http:
      paths:
      - backend:
          serviceName: nginx-svc
          servicePort: 80

部署查看

$ kubectl apply -f nginx-deployment.yam
$  kubectl get pods  --all-namespaces
NAMESPACE     NAME                                        READY   STATUS    RESTARTS   AGE
default       nginx-pod-5b5bc94455-ndcl6                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-nptm5                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-ptvzp                  1/1     Running   0          18m
default       nginx-pod-5b5bc94455-vw667                  1/1     Running   0          18m

修改 Host 文件

172.23.216.49 k8s.dashboard.com
172.23.216.49 traefik-ui.com
172.23.216.49 k8s.nginx.com

訪問 k8s.nginx.com 即可，查看 traefik-ui(對應 4個 Pod)。

HTTPS 證書配置

生成自簽名證書

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=traefik-ui.com"
kubectl -n kube-system create secret tls traefik-ui-tls-cert --key=tls.key --cert=tls.crt

配置

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik-ui.com
    http:
      paths:
      - backend:
          serviceName: traefik-web-ui
          servicePort: 80
  tls:
   - secretName: traefik-ui-tls-cert

其他問題：

Basic Authentication

如果想讓 traefik-web-ui 需要驗證才能訪問，可以配置 Basic Authentication，具體看官方文檔。

https://docs.traefik.io/user-guide/kubernetes/#basic-authentication

GRpc 配置

https://docs.traefik.io/user-guide/grpc/

服務權重配置

https://docs.traefik.io/user-guide/kubernetes/#traffic-splitting

證書配置

https://docs.traefik.io/user-guide/examples/#http-redirect-on-https

REFER:
https://docs.traefik.io/user-guide/kubernetes/
https://github.com/containous/traefik/tree/master/examples/k8s
http://blog.leanote.com/post/criss_sun/K8S%E4%B9%8Btraefik
http://blog.51cto.com/ylw6006/2119784