一、创建Traefik CRD资源
traefik v2.0 版本后，开始使用CRD（Custom Resource Definition）来完成路由配置
# mkdir -p /opt/kubernetes/traefik/yaml # cd /opt/kubernetes/traefik/yaml # vi crd.yaml ## IngressRoute apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutes.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRoute plural: ingressroutes singular: ingressroute scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: ingressroutetcps.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: IngressRouteTCP plural: ingressroutetcps singular: ingressroutetcp scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: middlewares.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: Middleware plural: middlewares singular: middleware scope: Namespaced --- apiVersion: apiextensions.k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: tlsoptions.traefik.containo.us spec: group: traefik.containo.us version: v1alpha1 names: kind: TLSOption plural: tlsoptions singular: tlsoption scope: Namespaced
创建Rraefik CRD资源
# kubectl create -f crd.yaml customresourcedefinition.apiextensions.k8s.io/ingressroutes.traefik.containo.us created customresourcedefinition.apiextensions.k8s.io/ingressroutetcps.traefik.containo.us created customresourcedefinition.apiextensions.k8s.io/middlewares.traefik.containo.us created customresourcedefinition.apiextensions.k8s.io/tlsoptions.traefik.containo.us created
二、创建Traefik RBAC权限
# vi rbac.yaml apiVersion: v1 kind: ServiceAccount metadata: name: traefik-ingress-controller namespace: kube-system --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller rules: - apiGroups: - "" resources: - services - endpoints - secrets verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses verbs: - get - list - watch - apiGroups: - extensions resources: - ingresses/status verbs: - update - apiGroups: - traefik.containo.us resources: - middlewares verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - ingressroutes verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - ingressroutetcps verbs: - get - list - watch - apiGroups: - traefik.containo.us resources: - tlsoptions verbs: - get - list - watch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: traefik-ingress-controller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: traefik-ingress-controller subjects: - kind: ServiceAccount name: traefik-ingress-controller namespace: kube-system
创建Traefik RBAC资源
# kubectl create -f rbac.yaml serviceaccount/traefik-ingress-controller created clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created
三、创建Traefik配置文件
用DaemonSet方式部署，便于在多服务器间扩展
# vi traefik.yaml apiVersion: apps/v1 kind: DaemonSet metadata: name: traefik namespace: kube-system labels: k8s-app: traefik-ingress-lb spec: selector: matchLabels: k8s-app: traefik-ingress-lb template: metadata: labels: k8s-app: traefik-ingress-lb name: traefik-ingress-lb spec: serviceAccountName: traefik-ingress-controller terminationGracePeriodSeconds: 60 restartPolicy: Always tolerations: - operator: "Exists" containers: - image: traefik:v2.0.7 name: traefik-ingress-lb resources: limits: cpu: 2000m memory: 1024Mi requests: cpu: 1000m memory: 1024Mi ports: - name: web containerPort: 80 hostPort: 80 - name: websecure containerPort: 443 hostPort: 443 - name: mysql containerPort: 3306 hostPort: 3306 - name: redis containerPort: 6379 hostPort: 6379 - name: admin containerPort: 8080 hostPort: 9999 securityContext: capabilities: drop: - ALL add: - NET_BIND_SERVICE args: - --entrypoints.web.Address=:80 - --entrypoints.websecure.Address=:443 - --entrypoints.mysql.Address=:3306 - --entrypoints.redis.Address=:6379 - --providers.kubernetescrd - --api - --api.dashboard=true - --api.insecure=true - --metrics.prometheus=true - --tracing.zipkin=true - --accesslog - --accesslog.filepath=/var/log/access.log nodeSelector: edgenode: "true" --- kind: Service apiVersion: v1 metadata: name: traefik namespace: kube-system spec: selector: k8s-app: traefik-ingress-lb ports: - name: admin port: 9999 protocol: TCP
创建Traefik资源
# kubectl create -f traefik.yaml daemonset.apps/traefik created service/traefik created
四、设置节点Label标签
由于是使用Kubernetes DeamonSet这种方式部署traefik，所以需要提前给节点设置label，这样当程序部署时pod会自动调度到设置label的点上
# kubectl get nodes --show-labels NAME STATUS ROLES AGE VERSION LABELS 192.168.168.3 Ready <none> 4d11h v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.168.3,kubernetes.io/os=linux 192.168.168.4 Ready <none> 4d11h v1.16.2 beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=192.168.168.4,kubernetes.io/os=linux # kubectl label nodes 192.168.168.3 edgenode=true node/192.168.168.3 labeled
注：如想删除label标签使用如下命令
# kubectl label nodes 192.168.168.3 edgenode-
五、配置Traefik路由规则
想让外部访问Kubernetes内部服务，需要配置路由规则，这里配置Traefik Dashboard的路由规则，使外部能够访问Traefik Dashboard
# vi IngressRoute.yaml apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: traefik-webui namespace: kube-system spec: entryPoints: - web routes: - match: Host(`traefik.k8s.local`) kind: Rule services: - name: traefik port: 9999
注:Host(` `)中的内容也可为自定义域名，如配置为traefik.
创建Traefik Dashboard https协议路由规则对象
# kubectl create -f IngressRoute.yaml
ingressroute.traefik.containo.us/traefik-webui created
注：卸载Traefik时先卸载IngressRoute.yaml再卸载其它资源
在管理机hosts里配置映射192.168.168.3 traefik.k8s.local(或搭建内网DNS服务器)，然后在浏览器中输入http://traefik.k8s.local:9999
六、配置多边缘节点高可用
1）在work-node01/02上安装keepalived｛安装过程此文略过｝
设置一个VIP IP指向自定义域名traefik.k8s.local，本文示例为192.168.168.100，这样集群外部就可以通过service的DNS映射名称来访问服务。
2）设置节点Label标签
# kubectl label nodes 192.168.168.3 edgenode=true # kubectl label nodes 192.168.168.4 edgenode=true
3）查看DaemonSet启动情况
# kubectl -n kube-system get ds NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE traefik 2 2 2 2 2 edgenode=true 16m
4）配置keepalived
# vi $KEEPALIVED_HOME/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from k8s_admin@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 100 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 6666 } virtual_ipaddress { 192.168.168.100 } } virtual_server 192.168.168.100 9999{ delay_loop 6 lb_algo loadbalance lb_kind DR nat_mask 255.255.255.0 persistence_timeout 0 protocol TCP real_server 192.168.168.3 9999{ weight 1 TCP_CHECK { connect_timeout 3 } } real_server 192.168.168.4 9999{ weight 1 TCP_CHECK { connect_timeout 3 } } }
注：real_server的IP和端口即traefik供外网访问的IP和端口
在管理机hosts里配置映射192.168.168.100 traefik.k8s.local，然后在在浏览器中输入http://traefik.k8s.local