碼上快樂

相關內容    簡體    繁體

k8s 上部署 Redis 三主三從 集群

本文轉載自   查看原文   2019-11-18 14:25   3647    kubernetes

目錄

介紹

Redis代表REmote DIctionary Server是一種開源的內存中數據存儲，通常用作數據庫，緩存或消息代理。它可以存儲和操作高級數據類型，例如列表，地圖，集合和排序集合。
由於Redis接受多種格式的密鑰，因此可以在服務器上執行操作，從而減少了客戶端的工作量。
它僅將磁盤用於持久性，而將數據庫完全保存在內存中。
Redis是一種流行的數據存儲解決方案，並被GitHub，Pinterest，Snapchat，Twitter，StackOverflow，Flickr等技術巨頭所使用。

為什么要使用Redis？

  • 它的速度非常快。它是用ANSI C編寫的，並且可以在POSIX系統上運行，例如Linux，Mac OS X和Solaris。
  • Redis通常被排名為最流行的鍵/值數據庫和最流行的與容器一起使用的NoSQL數據庫。
  • 其緩存解決方案減少了對雲數據庫后端的調用次數。
  • 應用程序可以通過其客戶端API庫對其進行訪問。
  • 所有流行的編程語言都支持Redis。
  • 它是開源且穩定的。

什么是Redis群集？

Redis Cluster是一組Redis實例，旨在通過對數據庫進行分區來擴展數據庫，從而使其更具彈性。
群集中的每個成員（無論是主副本還是輔助副本）都管理哈希槽的子集。如果主機無法訪問，則其從機將升級為主機。在由三個主節點組成的最小Redis群集中，每個主節點都有一個從節點（以實現最小的故障轉移），每個主節點都分配有一個介於0到16,383之間的哈希槽范圍。節點A包含從0到5000的哈希槽，節點B從5001到10000，節點C從10001到16383。
群集內部的通信是通過內部總線進行的，使用八卦協議傳播有關群集的信息或發現新節點。

在Kubernetes中部署Redis集群

在Kubernetes中部署Redis集群面臨挑戰，因為每個Redis實例都依賴於一個配置文件，該文件可以跟蹤其他集群實例及其角色。為此，我們需要結合使用Kubernetes StatefulSets和PersistentVolumes。

從 GitHub 上下載:

git clone https://github.com/llmgo/redis-sts.git

創建pv

[root@node01 redis-sts]# cat redis-pv.yml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv1
spec:
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv1
    server: 192.168.1.91
---  
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv2
spec:
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv2
    server: 192.168.1.91
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv3
spec:
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv3
    server: 192.168.1.91
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv4
spec:
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv4
    server: 192.168.1.91
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv5
spec:
  capacity:
    storage: 5Gi
  accessModes: 
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv5
    server: 192.168.1.91
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: redis-pv6
spec:
  capacity:
    storage: 5Gi
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: "redis-cluster"
  nfs:
    path: /data/redis-cluster/pv6
    server: 192.168.1.91

$ kubectl apply -f redis-pv.yml
persistentvolume/redis-pv1 created
persistentvolume/redis-pv2 created
persistentvolume/redis-pv3 created
persistentvolume/redis-pv4 created
persistentvolume/redis-pv5 created
persistentvolume/redis-pv6 created

$ kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS    REASON   AGE
redis-pv1   5Gi        RWO            Recycle          Available           redis-cluster            71s
redis-pv2   5Gi        RWO            Recycle          Available           redis-cluster            71s
redis-pv3   5Gi        RWO            Recycle          Available           redis-cluster            71s
redis-pv4   5Gi        RWO            Recycle          Available           redis-cluster            71s
redis-pv5   5Gi        RWO            Recycle          Available           redis-cluster            71s
redis-pv6   5Gi        RWO            Recycle          Available           redis-cluster            71s

創建statefulset

[root@node01 redis-sts]# cat redis-sts.yml 
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-cluster
data:
  update-node.sh: |
    #!/bin/sh
    REDIS_NODES="/data/nodes.conf"
    sed -i -e "/myself/ s/[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}/${POD_IP}/" ${REDIS_NODES}
    exec "$@"
  redis.conf: |+
    cluster-enabled yes
    cluster-require-full-coverage no
    cluster-node-timeout 15000
    cluster-config-file /data/nodes.conf
    cluster-migration-barrier 1
    appendonly yes
    protected-mode no
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: redis-cluster
spec:
  serviceName: redis-cluster
  replicas: 6
  selector:
    matchLabels:
      app: redis-cluster
  template:
    metadata:
      labels:
        app: redis-cluster
    spec:
      containers:
      - name: redis
        image: redis:5.0.5-alpine
        ports:
        - containerPort: 6379
          name: client
        - containerPort: 16379
          name: gossip
        command: ["/conf/update-node.sh", "redis-server", "/conf/redis.conf"]
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        volumeMounts:
        - name: conf
          mountPath: /conf
          readOnly: false
        - name: data
          mountPath: /data
          readOnly: false
      volumes:
      - name: conf
        configMap:
          name: redis-cluster
          defaultMode: 0755
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi
      storageClassName: redis-cluster

$ kubectl apply -f redis-sts.yml
configmap/redis-cluster created
statefulset.apps/redis-cluster created

$ kubectl get pods -l app=redis-cluster
NAME              READY   STATUS    RESTARTS   AGE
redis-cluster-0   1/1     Running   0          53s
redis-cluster-1   1/1     Running   0          49s
redis-cluster-2   1/1     Running   0          46s
redis-cluster-3   1/1     Running   0          42s
redis-cluster-4   1/1     Running   0          38s
redis-cluster-5   1/1     Running   0          34s

創建service

[root@node01 redis-sts]# cat redis-svc.yml 
---
apiVersion: v1
kind: Service
metadata:
  name: redis-cluster
spec:
  type: ClusterIP
  clusterIP: 10.1.0.106
  ports:
  - port: 6379
    targetPort: 6379
    name: client
  - port: 16379
    targetPort: 16379
    name: gossip
  selector:
    app: redis-cluster

$ kubectl apply -f redis-svc.yml
service/redis-cluster created
$ kubectl get svc redis-cluster
NAME            TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)              AGE
redis-cluster   ClusterIP   10.1.0.106   <none>        6379/TCP,16379/TCP   35s

初始化 Redis Cluster

下一步是形成Redis集群。為此，我們運行以下命令並鍵入yes以接受配置。前三個節點成為主節點，后三個節點成為從節點。

$ kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')

kubectl exec -it redis-cluster-0 -- redis-cli --cluster create --cluster-replicas 1 $(kubectl get pods -l app=redis-cluster -o jsonpath='{range.items[*]}{.status.podIP}:6379 ')
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 10.244.2.11:6379 to 10.244.9.19:6379
Adding replica 10.244.9.20:6379 to 10.244.6.10:6379
Adding replica 10.244.8.15:6379 to 10.244.7.8:6379
M: 00721c43db194c8f2cacbafd01fd2be6a2fede28 10.244.9.19:6379
   slots:[0-5460] (5461 slots) master
M: 9c36053912dec8cb20a599bda202a654f241484f 10.244.6.10:6379
   slots:[5461-10922] (5462 slots) master
M: 2850f24ea6367de58fb50e632fc56fe4ba5ef016 10.244.7.8:6379
   slots:[10923-16383] (5461 slots) master
S: 554a58762e3dce23ca5a75886d0ccebd2d582502 10.244.8.15:6379
   replicates 2850f24ea6367de58fb50e632fc56fe4ba5ef016
S: 20028fd0b79045489824eda71fac9898f17af896 10.244.2.11:6379
   replicates 00721c43db194c8f2cacbafd01fd2be6a2fede28
S: 87e8987e314e4e5d4736e5818651abc1ed6ddcd9 10.244.9.20:6379
   replicates 9c36053912dec8cb20a599bda202a654f241484f
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
...
>>> Performing Cluster Check (using node 10.244.9.19:6379)
M: 00721c43db194c8f2cacbafd01fd2be6a2fede28 10.244.9.19:6379
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
M: 9c36053912dec8cb20a599bda202a654f241484f 10.244.6.10:6379
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 87e8987e314e4e5d4736e5818651abc1ed6ddcd9 10.244.9.20:6379
   slots: (0 slots) slave
   replicates 9c36053912dec8cb20a599bda202a654f241484f
S: 554a58762e3dce23ca5a75886d0ccebd2d582502 10.244.8.15:6379
   slots: (0 slots) slave
   replicates 2850f24ea6367de58fb50e632fc56fe4ba5ef016
S: 20028fd0b79045489824eda71fac9898f17af896 10.244.2.11:6379
   slots: (0 slots) slave
   replicates 00721c43db194c8f2cacbafd01fd2be6a2fede28
M: 2850f24ea6367de58fb50e632fc56fe4ba5ef016 10.244.7.8:6379
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

驗證集群部署

[root@node01 redis-sts]# kubectl exec -it redis-cluster-0 -- redis-cli cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:16
cluster_stats_messages_pong_sent:22
cluster_stats_messages_sent:38
cluster_stats_messages_ping_received:17
cluster_stats_messages_pong_received:16
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:38

[root@node01 redis-sts]# for x in $(seq 0 5); do echo "redis-cluster-$x"; kubectl exec redis-cluster-$x -- redis-cli role; echo; done
redis-cluster-0
master
14
10.244.2.11
6379
14

redis-cluster-1
master
28
10.244.9.20
6379
28

redis-cluster-2
master
28
10.244.8.15
6379
28

redis-cluster-3
slave
10.244.7.8
6379
connected
28

redis-cluster-4
slave
10.244.9.19
6379
connected
14

redis-cluster-5
slave
10.244.6.10
6379
connected
28

測試Redis集群

我們想使用集群，然后模擬節點的故障。對於前一項任務，我們將部署一個簡單的Python應用程序，而對於后者，我們將刪除一個節點並觀察集群行為。

部署點擊計數器應用

我們將一個簡單的應用程序部署到集群中，並在其前面放置一個負載平衡器。此應用程序的目的是在將計數器值作為HTTP響應返回之前，增加計數器並將其存儲在Redis集群中。

$ kubectl apply -f app-deployment-service.yml
service/hit-counter-lb created
deployment.apps/hit-counter-app created

在此過程中，如果我們繼續加載頁面，計數器將繼續增加，並且在刪除Pod之后，我們看到沒有數據丟失。

$  curl `kubectl get svc hit-counter-lb -o json|jq -r .spec.clusterIP`
I have been hit 20 times since deployment.
$  curl `kubectl get svc hit-counter-lb -o json|jq -r .spec.clusterIP`
I have been hit 21 times since deployment.
$ curl `kubectl get svc hit-counter-lb -o json|jq -r .spec.clusterIP`
I have been hit 22 times since deployment.
$ kubectl delete pods redis-cluster-0
pod "redis-cluster-0" deleted
$ kubectl delete pods redis-cluster-1
pod "redis-cluster-1" deleted
$  curl `kubectl get svc hit-counter-lb -o json|jq -r .spec.clusterIP`
I have been hit 23 times since deployment.

轉自: http://llmgo.cn/post/redis-003/


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Ubuntu 14.04主機上部署k8s集群 優化：在k8s上部署的gitlab Kubernetes(k8s)部署Redis cluster集群 mac搭建redis集群（三主三從） Redis集群（cluster模式）搭建（三主三從） 國內Ubuntu 14.04主機上部署k8s集群 通過mongodb-kubernetes-operator在k8s集群上部署mongodb Flink源碼閱讀 - K8s上部署session模式的集群(入口配置) k8s部署rocketmq 雙主 K8S（二）——K8S中部署tomcat集群
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM