1、kubernetes1.13之后kubeadm開始GA，由於1.13的kube-proxy有bug，刪除ipvs的地方總是導致kube-proxy掛掉，所以建議直接用1.13.2，這個版本解決了bug。
！Fix race condition introduced by graceful termination which can lead to a deadlock in kube-proxy
！Support graceful termination with IPVS when deleting a service
！kube-proxy in IPVS mode will stop initiating connections to terminating pods for services with sessionAffinity set
kubeadm的配置文件較之前有了一些改變，版本變成了v1beta1,雖然目前老的版本在1.13里也是可用的，但是1.14就會廢棄掉，所以建議直接使用新的config進行初始化
下面給出我的1.13.2版本使用的kubeadm init config:
cat kubeadm-13.yaml
--- apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration mode: "ipvs"
--- apiVersion: kubeadm.k8s.io/v1beta1 certificatesDir: /etc/kubernetes/pki clusterName: kubernetes controlPlaneEndpoint: "10.4.9.210" dns: type: CoreDNS imageRepository: k8s.gcr.io kind: ClusterConfiguration kubernetesVersion: v1.13.2 networking: dnsDomain: cluster.local podSubnet: 192.244.0.0/16 serviceSubnet: 192.96.0.0/16 apiServer: timeoutForControlPlane: 4m0s certSANs: - 10.4.73.110
- 10.4.68.77
- 10.4.83.248
- ku13-1
- ku13-2
- ku13-3
- 127.0.0.1
- localhost - 10.4.9.210 controllerManager: extraArgs: address: 0.0.0.0 scheduler: extraArgs: address: 0.0.0.0 etcd: external: endpoints: - https://10.4.73.110:2379
- https://10.4.68.77:2379
- https://10.4.83.248:2379
caFile: /etc/kubernetes/ssl/ca.pem certFile: /etc/etcd/ssl/etcd.pem keyFile: /etc/etcd/ssl/etcd-key.pem
kubeadm1.13+ 支持的configuration types：
apiVersion: kubeadm.k8s.io/v1beta1 kind: InitConfiguration apiVersion: kubeadm.k8s.io/v1beta1 kind: ClusterConfiguration apiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration apiVersion: kubeproxy.config.k8s.io/v1alpha1 kind: KubeProxyConfiguration apiVersion: kubeadm.k8s.io/v1beta1 kind: JoinConfiguration
具體變動參考：
https://godoc.org/k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm/v1beta1
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.13.md#changelog-since-v1131