k8s中安裝redis6集群

注意:第二步不用操作了,yaml文件僅供參考

1.創建Namespace

kubectl apply -f Namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: cluster-redis

2.創建PersistentVolumeClaim #這一步不用操作了

kubectl apply -f PersistentVolumeClaim.yaml

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc1
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---  
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc2
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc3
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc4
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc5
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: redis-pvc6
  namespace: cluster-redis
spec:
  accessModes: 
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  storageClassName: "storage"

3.創建statefulset

kubectl apply -f ConfigMap.yaml
kubectl apply -f StatefulSet.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-cluster-configmap
  namespace: cluster-redis
data:
  update-node.sh: |
    #!/bin/sh
    REDIS_NODES="/data/nodes.conf"
    sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
    exec "$@"
  redis.conf: |+
    cluster-enabled yes
    cluster-require-full-coverage no
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    cluster-migration-barrier 1
    appendonly yes
    protected-mode no

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-pod
  namespace: cluster-redis
spec:
  serviceName: redis-cluster-service
  replicas: 6
  selector:
    matchLabels:
      app: redis-pod
  template:
    metadata:
      labels:
        app: redis-pod
    spec:
      containers:
      - name: redis
        image: redis:6.2.1
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster-configmap
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteMany" ]
      resources:
        requests:
          storage: 1Gi
      storageClassName: "storage" # 注意這個,事先創建好存儲卷

4.創建service

kubectl apply -f Service.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis-cluster-service
  namespace: cluster-redis
spec:
  type: ClusterIP
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-pod

5.初始化 Redis Cluster
注意如下這一步,分開來看:
(1) 命令: kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'
這個命令是獲取到StatefulSet.yaml文件中生成的redis pod ip,最后那個awk是過濾到不需要的信息
比如執行kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '得到的結果如下:
10.0.2.122:6379 10.0.0.65:6379 10.0.2.123:6379 10.0.0.66:6379 10.0.0.69:6379 10.0.2.125:6379 :6379
可以看到最后有個多余的:6379是沒用的,awk的作用就是去掉這個
(2) 命令:kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli --cluster create (pod ip) --cluster-replicas 1
這個命令是登錄到其中一個pod中創建集群使用的

kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli --cluster create `kubectl get pods -n cluster-redis -l app=redis-pod -o jsonpath='{range.items[*]}{.status.podIP}:6379 '|awk '{print $1" "$2" "$3" "$4" "$5" "$6" "}'` --cluster-replicas 1

6.驗證集群部署

kubectl exec -it redis-pod-0 -n cluster-redis -- redis-cli cluster info
for x in $(seq 0 5); do echo "redis-pod-$x"; kubectl exec redis-pod-$x -n cluster-redis -- redis-cli role; echo; done

7.注意事項
(1)StatefulSet類型的資源不能通過修改'replicas', 'template', and 'updateStrategy'等字段來進行更新,否則就會報錯,具體如下:

The StatefulSet "redis-pod" is invalid: spec: Forbidden: updates to statefulset spec for fields other than 'replicas', 'template', and 'updateStrategy' are forbidden

(2)生成的pod編號是從0開始的,依次是 redis-pod-0, redis-pod-1, redis-pod-2, redis-pod-3, redis-pod-4, redis-pod-5.
當刪除一個pod時則會自動生成一個同名的pod.當然了,新生成的pod ip會發生變化

(3)查看pod對應的DNS域名

# for x in $(seq 0 5); do kubectl exec redis-pod-$x -n cluster-redis -- hostname -f; done
redis-pod-0.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-1.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-2.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-3.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-4.redis-cluster-service.cluster-redis.svc.cluster.local
redis-pod-5.redis-cluster-service.cluster-redis.svc.cluster.local

8.使用
項目中代碼連接redis集群的話,使用的參數是: ClusterIP:6379