一、Service網絡簡介：
Why：pod重啟或者重建ip會發生改變，pod之間訪問會有問題;
What：解耦了服務和應用。（集群內部服務之間調用填寫service域名/IP即可;
How：聲明一個service對象
一般常用的有兩種：
k8s集群內部的service：selector指定pod，自動創建Endpoints
k8s集群外的service：手動創建Endpoints，指定外部服務的ip、端口和協議。
1.1 k8s 三種網絡
node network
pod network
cluster network，也稱作virtual IP虛擬網絡—service
kube-proxy監聽k8s-apiserver,一旦service資源發生變化，kube-proxy就會生成對應的負載調度的調整，這樣就保證service的最新狀態。
1.2 service類型
ExternalName
Cluster IP
Node Port
Load Balancer
1.3 資源記錄
SVC_NAME.NS_NAME.DOMAIN.LTD.
DOMAIN.LTD 集群域名后綴默認為svc.cluster.local.
1.4 service 三種工作模式
userspace
iptabels
ipvs 最新版本
1.4.1 iptables網絡模型
1.4.2 ipvs模型
二、實踐
2.1 使用清單文件創建cluster IP service資源
apiVersion: v1 kind: Service metadata: name: nginx namespace: default spec: selector: app: nginx role: logstor clusterIP: 10.97.97.97 //默認為clusterIP，不指定會自動分配 type: ClusterIP ports: - port: 80 //service端口 targetPort: 80 //容器端口
master yaml]# kubectl get svc -o wide //查看創建的SVC NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28d <none> nginx ClusterIP 10.97.97.97 <none> 80/TCP 13m app=nginx,role=logstor
2.1.1 資源記錄
SVC_NAME.NS_NAME.DOMAIN.LTD.
集群默認后綴DOMAIN.LTD.：svc.cluster.local.
所以上面創建的服務記錄為：nginx.default.svc.cluster.local
2.2 使用資源清單創建類型為nodeport的svc
apiVersion: v1 kind: Service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary clusterIP: 10.99.99.99 type: NodePort ports: - port: 80 //service端口 targetPort: 80 //pod端口 nodePort: 30080 //映射節點端口，后續可以直接訪問該節點端口，DNAT到service端口，再到pod端口 master yaml]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28d myapp NodePort 10.99.99.99 <none> 80:30080/TCP 38m //網絡類型為節點網絡 nginx ClusterIP 10.97.97.97 <none> 80/TCP 76m
2.2.1 訪問servic
]# curl http://172.18.0.68:30080 //在集群外部機器訪問該集群內IP加端口，servie后端須關聯pod資源才可以被訪問
注：此時可以在節點前面做代理服務器來訪問service地址
2.3 Load Balancer（負載均衡器） 訪問示意圖，在Nodeport前面搭建一套負載均衡
參考華為雲文檔：https://support.huaweicloud.com/usermanual-cce/cce_01_0014.html
apiVersion: v1 kind: Service metadata: annotations: kubernetes.io/elb.id: 3c7caa5a-a641-4bff-801a-feace27424b6 # ELB實例ID，替換為實際值 name: nginx spec: ports: - name: service0 port: 80 protocol: TCP targetPort: 80 selector: app: nginx type: LoadBalancer
2.4 如何讓集群內資源訪問到互聯網資源
思考：namespace是作為資源隔離，不同服務放在不同namespace下，如果希望服務之間可以互相訪問，也就是跨namespace的服務訪問，應該怎么處理呢？
ExternalName（外部名稱 ）模式
參考文檔：https://www.jianshu.com/p/deaa98210827
2.4.1、使用service代理外部ip和端口
apiVersion: v1 kind: Endpoints apiVersion: v1 metadata: name: my-mysql-endpoint #此名字需與 my-mysql-service.yaml文件中的 metadata.name 的值一致 namespace: my-first-app #在固定的命名空間下 subsets: - addresses: - ip: 39.156.69.79 # 公網mysql地址 ports: - name： http
port: 3306
protocol: TCP
apiVersion: v1 kind: Service metadata: name: my-mysql-endpoint #此名字需與 my-mysql-endpoints.yaml文件中的 metadata.name 的值一致 namespace: my-first-app #在固定的命名空間下 spec: ports:
- name： http
port: 3306
protocol: TCP
查看ep
測試
如果外部IP變更，只需要更改endpoint配置文件即可。
此種方式只適合Ip+端口訪問，對於像阿里雲rds等數據庫的。需要用域名。則需要用ExternalName方式不而不是Endpoints方式
2.4.2 、EXternalName反代外部域名
創建連接外部MySQL數據庫的svc，EXternalName需要和需要訪問外網的pod在同一個namespace，pod通過訪問service，cname到externalName指定的地址。
kind: Service
apiVersion: v1
metadata:
name: my-service-1
namespace: default spec: type: ExternalName externalName: www.mysql-cluster.com //當訪問my-service-1.default.svc.cluster.local時，會由coredns cname到www.baidu.com
參考文檔：https://www.cnblogs.com/shetao/articles/14340124.html //聯系我侵刪
]# cat svc_ExternalName_visit.yaml
5 # 實現 myns 名稱空間的pod，訪問 mytest 名稱空間的Service：myapp-clusterip2 6 apiVersion: v1 7 kind: Service 8 metadata: 9 name: myapp-clusterip1-externalname 10 namespace: myns 11 spec: 12 type: ExternalName # 指定類型 13 externalName: myapp-clusterip2.mytest.svc.cluster.local 14 ports: 15 - name: http 16 port: 80 17 targetPort: 80 18 --- 19 # 實現 mytest 名稱空間的Pod，訪問 myns 名稱空間的Service：myapp-clusterip1 20 apiVersion: v1 21 kind: Service 22 metadata: 23 name: myapp-clusterip2-externalname 24 namespace: mytest 25 spec: 26 type: ExternalName 27 externalName: myapp-clusterip1.myns.svc.cluster.local //填寫對端完整域名 28 ports: 29 - name: http 30 port: 80 31 targetPort: 80
測試是否可以跨namespace訪問
如下說明通過Service externalname類型，實現了Pod跨namespace名稱空間與Service訪問
18 ~ # ping myapp-clusterip2-externalname //ExternalName名稱實現了cname轉發 19 PING myapp-clusterip2-externalname (10.100.61.11): 56 data bytes 20 64 bytes from 10.100.61.11: seq=0 ttl=64 time=0.089 ms 21 64 bytes from 10.100.61.11: seq=1 ttl=64 time=0.071 ms 24 ~ # wget myapp-clusterip2-externalname -O myns.html 25 Connecting to myapp-clusterip2-externalname (10.100.61.11:80) 26 myns.html 100% 28 ~ # cat myns.html 29 Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
2.5 創建一個hostnetwork類型的pod
2.5.1 不指定DNS策略
apiVersion: v1 kind: Pod metadata: name: nginx-hostnet spec: containers: - name: nginx image: tomcat-app1:v1 hostNetwork: true # 將pod網絡和宿主機網絡打通
查看網絡
]# ifconfig cali0c474fa1137: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 1110071 bytes 566024261 (539.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 1110071 bytes 566024261 (539.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 cali4478953ae54: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:5f:9c:3d:62 txqueuelen 0 (Ethernet) RX packets 1915824 bytes 33663989873 (31.3 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 11088604 bytes 626077036 (597.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 ens32: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 192.168.64.113 netmask 255.255.255.0 broadcast 192.168.64.255 inet6 fe80::4b4:9202:aaca:81ec prefixlen 64 scopeid 0x20<link> ether 00:0c:29:59:02:06 txqueuelen 1000 (Ethernet) RX packets 26655124 bytes 18020033636 (16.7 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 29230552 bytes 58639418242 (54.6 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
]# cat /etc/hosts # Kubernetes-managed hosts file (host network). 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.64.110 master-1 192.168.64.111 master-2 192.168.64.112 master-3 192.168.64.113 node1 192.168.64.114 node2 10.103.97.2 apiserver.cluster.local [root@node1 /]# cat /etc/resolv.conf nameserver 223.5.5.5 nameserver 223.6.6.6
/]# ping www.baidu.com PING www.a.shifen.com (182.61.200.6) 56(84) bytes of data. 64 bytes from 182.61.200.6 (182.61.200.6): icmp_seq=1 ttl=128 time=47.8 ms 64 bytes from 182.61.200.6 (182.61.200.6): icmp_seq=2 ttl=128 time=45.7 ms ^C --- www.a.shifen.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1002ms rtt min/avg/max/mdev = 45.786/46.821/47.857/1.057 ms
2.5.2 指定dns策略
apiVersion: v1 kind: Pod metadata: name: nginx-hostnet spec: containers: - name: nginx image: tomcat-app1:v1 hostNetwork: true # 將pod網絡和宿主機網絡打通 dnsPolicy: ClusterFirstWithHostNet # DNS策略，使用集群內的dns
查看dns
]# kubectl get svc -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3d6h kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 3d6h # 集群默認dns
查看網絡
]# cat /etc/hosts # Kubernetes-managed hosts file (host network). 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.64.110 master-1 192.168.64.111 master-2 192.168.64.112 master-3 192.168.64.113 node1 192.168.64.114 node2 10.103.97.2 apiserver.cluster.local
[root@node2 /]# cat /etc/resolv.conf nameserver 10.96.0.10 # 可以看到使用的集群默認dns search default.svc.cluster.local svc.cluster.local cluster.local options ndots:5
[root@node2 /]# ifconfig cali56c64d85ccf: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 calidb7d83c1f13: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1440 ether ee:ee:ee:ee:ee:ee txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255 ether 02:42:28:df:af:fc txqueuelen 0 (Ethernet) RX packets 645516 bytes 2516757196 (2.3 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 621504 bytes 45130104 (43.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
三、標簽管理
3.1 模擬匹配pod作為后端資源
master yaml]# kubectl apply -f test.yaml master yaml]# kubectl label pods nginx app=nginx master yaml]# kubectl describe svc nginx Name: nginx Namespace: default Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"nginx","namespace":"default"},"spec":{"clusterIP":"10.97.97.97","... Selector: app=nginx,role=logstor Type: ClusterIP IP: 10.97.97.97 Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: <none> //可以看到當svc標簽與pod標簽只匹配一個時，service未關聯到pod資源 Session Affinity: None Events: <none> master yaml]# kubectl label pods nginx role=logstor //手動匹配所有標簽 master yaml]# kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 28d <none> nginx ClusterIP 10.97.97.97 <none> 80/TCP 13m app=nginx,role=logstor master yaml]# kubectl get pods --show-labels NAME READY STATUS RESTARTS AGE LABELS nginx 1/1 Running 0 10m app=nginx,role=logstor //此時標簽已全匹配 master yaml]# kubectl describe svc nginx //再次查看發現已匹配后端資源 Name: nginx Namespace: default Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"nginx","namespace":"default"},"spec":{"clusterIP":"10.97.97.97","... Selector: app=nginx,role=logstor //svc標簽 Type: ClusterIP IP: 10.97.97.97 Port: <unset> 80/TCP TargetPort: 80/TCP Endpoints: 10.244.1.84:80 //標簽一致，匹配到的后端pod資源 Session Affinity: None Events: <none>
結論：servie根據Selector標簽選擇器關聯pod資源，service的Selector不需要完全匹配pod標簽，而pod標簽必須完全匹配service標簽才可以被關聯。
3.2 查看svc關聯的pod
]# kubectl get svc -A NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 236d kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 236d kube-system metrics-server ClusterIP 10.105.89.199 <none> 443/TCP 15d kubernetes-dashboard dashboard-metrics-scraper ClusterIP 10.109.95.129 <none> 8000/TCP 15d kubernetes-dashboard kubernetes-dashboard NodePort 10.98.147.122 <none> 443:30002/TCP 15d linux40 ng-deploy-80 NodePort 10.97.239.161 <none> 81:30019/TCP 20d ]# kubectl get pods -n linux40 -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-deployment-55fb8c9d77-9qlkp 1/1 Running 9 20d 10.244.1.98 node1 <none> <none> nginx-deployment-55fb8c9d77-sj8bs 1/1 Running 3 15d 10.244.2.29 node2 <none> <none> # 查看endpoint，svc關聯的pod資源 ]# kubectl get ep -n linux40 NAME ENDPOINTS AGE ng-deploy-80 10.244.1.98:80,10.244.2.29:80 20d
四、把來自於同一個客戶端的請求始終固定訪問同一個pod資源
master yaml]# kubectl patch svc myapp -p ‘{“spec”:{“sessionAffinity”:“ClientIP”}}’
master yaml]# kubectl describe svc myapp //查看service詳情 Name: myapp Namespace: default Labels: <none> Annotations: kubectl.kubernetes.io/last-applied-configuration: {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"myapp","namespace":"default"},"spec":{"clusterIP":"10.99.99.99","... Selector: app=myapp,release=canary Type: NodePort IP: 10.99.99.99 Port: <unset> 80/TCP TargetPort: 80/TCP NodePort: <unset> 30080/TCP Endpoints: 10.244.2.84:80 //后端pod資源 Session Affinity: ClientIP //此參數說明相同客戶端訪問相同pod資源；若為“none”則為負載均衡調度。 External Traffic Policy: Cluster Events: <none>
無頭service：將service名稱解析到后端pod ip地址來實現訪問
apiVersion: v1 kind: Service metadata: name: myapp-svc namespace: default spec: selector: app: myapp release: canary clusterIP: "None" //指定service IP地址為空 ports: - port: 80 targetPort: 80
master yaml]# kubectl get svc -n kube-system NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 29d master yaml]# dig -t A myapp-svc.default.svc.cluster.local. @10.96.0.10 ...... ;; ANSWER SECTION: myapp-svc.default.svc.cluster.local. 30 IN A 10.244.1.97 //可以看到解析SVC資源記錄地址為pod地址，當關聯到時 myapp-svc.default.svc.cluster.local. 30 IN A 10.244.2.101 myapp-svc.default.svc.cluster.local. 30 IN A 10.244.2.100 myapp-svc.default.svc.cluster.local. 30 IN A 10.244.1.96 myapp-svc.default.svc.cluster.local. 30 IN A 10.244.2.104
......
master yaml]# kubectl get pods -o wide --show-labels //查看pod標簽，與無頭service 標簽完全匹配
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES LABELS
myapp-deploy-f476f4fcf-jltjs 1/1 Running 3 6d1h 10.244.1.85 k8s-node1 <none> <none> app=myapp,pod-template-hash=f476f4fcf,releas=canary
myapp-deploy-f476f4fcf-nbqwv 1/1 Running 3 6d1h 10.244.2.86 k8s-node2 <none> <none> app=myapp,pod-template-hash=f476f4fcf,releas=canary
nginx 1/1 Running 1 7h26m 10.244.2.85 k8s-node2 <none> <none> app=myapp,release=canary
master yaml]# kubectl get svc -o wide //查看service標簽，與pod 標簽完全匹配時，可解析service name到pod地址
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 29d <none>
myapp NodePort 10.99.99.99 <none> 80:30080/TCP 7h49m app=myapp,release=canary
myapp-svc ClusterIP None <none> 80/TCP 3m23s app=myapp,release=canary //SVC地址為空
nginx ClusterIP 10.97.97.97 <none> 80/TCP 8h app=nginx,role=logstor