k8s之服務暴露

k8s的服務暴露

k8s的dns實現了服務在集群“內”被自動發現，那如何是的服務在k8s集群”外“被使用和訪問呢？

• 使用nodePort型的Service

  • 注意：無法使用kube-proxy的ipvs模型，只能使用iptables模型

• 使用ingress資源

  • 注意：ingress只能調度並暴露7層應用，特指http和https協議

    • 不建議https用ingress

    那https協議來了怎么辦呢？

    在L7層負載均衡上做證書卸載

    

ingress是k8s API的標准資源類型之一，也是一種核心資源，它其實就是一組基於域名和URL路徑，把用戶的請求轉發至指定Service資源的規則

可以將集群外部的請求流量，轉發至集群內部，從而實現“服務暴露”

ingress控制器是能夠為ingress資源監聽某套接字，然后根據ingress規則匹配機制路由調度流量的一個組件。

說白了，ingress沒啥神秘的，就是個簡化版的nginx+一段go腳本而已

常用的ingress控制器的實現軟件

• Ingress-nginx
• HAProxy
• Traefik

部署traefik

在hdss7-200操作

[root@hdss7-200 coredns]# pwd
/data/k8s-yaml/coredns
[root@hdss7-200 coredns]# cd ..
[root@hdss7-200 k8s-yaml]# mkdir traefik
[root@hdss7-200 k8s-yaml]# cd traefik/

[traefik地址][https://github.com/containous/traefik]

建議使用1.7.2版本就可以了

[root@hdss7-200 traefik]# docker pull traefik:v1.7.2-alpine
[root@hdss7-200 traefik]# docker images | grep traefik
traefik         v1.7.2-alpine              add5fac61ae5        22 months ago       72.4MB

[root@hdss7-200 traefik]# docker tag add5fac61ae5 harbor.od.com/public/traefik:v1.7.2
[root@hdss7-200 traefik]# docker push !$
docker push harbor.od.com/public/traefik:v1.7.2

[root@hdss7-200 traefik]# vim rbac.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

svc.yaml

kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress
  ports:
    - protocol: TCP
      port: 80
      name: controller
    - protocol: TCP
      port: 8080
      name: admin-web

ds.yaml

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: traefik-ingress
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress
spec:
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress
        name: traefik-ingress
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: harbor.od.com/public/traefik:v1.7.2
        name: traefik-ingress
        ports:
        - name: controller
          containerPort: 80
          hostPort: 81
        - name: admin-web
          containerPort: 8080
        securityContext:
          capabilities:
            drop:
            - ALL
            add:
            - NET_BIND_SERVICE
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
        - --insecureskipverify=true
        - --kubernetes.endpoint=https://10.4.7.10:7443
        - --accesslog
        - --accesslog.filepath=/var/log/traefik_access.log
        - --traefiklog
        - --traefiklog.filepath=/var/log/traefik.log
        - --metrics.prometheus

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik.od.com
    http:
      paths:
	  - path: /
        backend:
          serviceName: traefik-ingress-service
          servicePort: 8080

    - name: controller
      containerPort: 80
      hostPort: 81

把docker的80端口映射到宿主機的81端口

ingress控制器是能夠為ingress資源監聽某套接字(暴露的端口），然后根據ingress規則匹配機制路由調度流量的一個組件。

創建對應的資源

[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/rbac.yaml
serviceaccount/traefik-ingress-controller created
clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created

[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/ds.yaml
daemonset.extensions/traefik-ingress created

[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/svc.yaml
service/traefik-ingress-service created

[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/ingress.yaml

[root@hdss7-22 ~]# kubectl apply -f http://k8s-yaml.od.com/traefik/ingress.yaml
ingress.extensions/traefik-web-ui created

[root@hdss7-22 ~]# kubectl get pods -n kube-system
NAME                       READY   STATUS    RESTARTS   AGE
coredns-6b6c4f9648-ttfg8   1/1     Running   0          11h
traefik-ingress-fnrbg      0/1     Running   0          58s
traefik-ingress-kxm8t      0/1     Running   0          33s

如果一直啟動不起來，嘗試重啟docker服務

[root@hdss7-22 ~]# systemctl restart docker.service
[root@hdss7-21 ~]# systemctl restart docker.service

再次查看

[root@hdss7-22 ~]# kubectl get pods -n kube-system
NAME                       READY   STATUS    RESTARTS   AGE
coredns-6b6c4f9648-ttfg8   1/1     Running   0          11h
traefik-ingress-fnrbg      1/1     Running   0          58s
traefik-ingress-kxm8t      1/1     Running   0          33s

netstat -luntp | grep 81

在hdss7-11和hdss7-12上操作，配置反向代理

注意：nginx的優先級是詳細的優先於泛的。

[root@hdss7-11 ~]# vim /etc/nginx/conf.d/od.com.conf

upstream default_backend_traefik {
    server 10.4.7.21:81    max_fails=3 fail_timeout=10s;
    server 10.4.7.22:81    max_fails=3 fail_timeout=10s;
}
server {
    server_name *.od.com;
  
    location / {
        proxy_pass http://default_backend_traefik;
        proxy_set_header Host       $http_host;
        proxy_set_header x-forwarded-for $proxy_add_x_forwarded_for;
    }
}

81是ingress controller端口

server_name ，泛域名匹配，泛域名流量調度

上面這個配置的意思就是，凡是走*.od.com業務域的流量，沒差別的提交給server，拋給了ingress controller，然后交給了ingress service，最終，ingress service交給了traefik ingress service，它又是怎么找到pod的呢？通過node-selector找到k8s-app: traefik-ingress，然后找到對應的k8s-app的標簽

svc.yaml

  selector:
    k8s-app: traefik-ingress

ds.yaml

      labels:
        k8s-app: traefik-ingress

再次參考圖

kube service就是由kube-proxy這個組件承載的。

回到正題，經過這樣配置之后，nginx可以永遠不用再動了，nginx七層代理就干這件事。如果還需要調度七層流量規則怎么辦？你只需要聲明ingress配置文件，七層調度全部交給ingress來干了。

修改dns解析

               2020080104; serial
traefik          A 10.4.7.10

traefik這里要指向vip的地址

重啟相關服務

[root@hdss7-12 ~]# systemctl restart named
[root@hdss7-12 ~]# nginx -s reload

訪問頁面