kubernetes部署elasticsearch-6.6.2

事先環境准備：
1.k8s環境
2.集群存儲，本文使用的是ceph

 

以下用到的配置文件：
es_configmap.yaml
es_ing.yaml
es_statefulset.yaml
es_svc.yaml

 

1. 配置文件 es_svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: hotes-cluster-9300
spec:
  clusterIP: None
  selector:
    app: es-cluster
  ports:
    - port: 9300
      name: inner

---
apiVersion: v1
kind: Service
metadata:
  name: hotes-cluster-9200
spec:
  selector:
    app: es-cluster
  ports:
    - name: http
      port: 9200
      targetPort: 9200
      nodePort: 8831
  type: NodePort

 

把hotes-cluster-9300的clusterIP設置為:None,被稱作headless service（可以參考https://www.jianshu.com/p/a6d8b28c88a2）
hotes-cluster-9200設置了9200，用於外部訪問es

2. 配置文件 es_ing.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: hotes-cluster
spec:
  rules:
    - host: www.hotes.com
      http:
        paths:
          - backend:
              serviceName: hotes-cluster-9200
              servicePort: 9200
            path: /

 

設置一個域名，用於外部訪問es

3. 配置文件 es_configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: hotes-config
data:
  elasticsearch.yml: |
    node.name: ${HOSTNAME}  
    cluster.name: hotes-backup 
    network.host: "0.0.0.0"
    bootstrap.memory_lock: false
    discovery.zen.ping.unicast.hosts: esnode-0.hotes-cluster-9300.default.svc.cluster.local.,esnode-1.hotes-cluster-9300.default.svc.cluster.local.,esnode-2.hotes-cluster-9300.default.svc.cluster.local.
    discovery.zen.minimum_master_nodes: 1

node.name 節點名

cluster.name 集群名字
discovery.zen.ping.unicast.hosts 集群各節點的名字，像這樣使用headless service方式，解析出來的IP進行通信，不需要走kube-proxy，默認走的是9300端口，如果輸入的是主機名被解析成了多個地址，節點將會嘗試連接所有地址，同時只選擇其中一個可用的
pod的dns規則：{stateful-set-name}-{0…N}.{service-name}.{namespace}.svc.cluster.local.（注意后面還有一個.）
這里配置的service-name是headless service name

discovery.zen.minimum_master_nodes 為了防止過多的數據丟失，配置這個參數很重要。

如果沒有這個配置，當集群遇到網絡問題時，有可能會分離成兩個獨立的集群（腦裂現象），這樣會導致集群節點丟失，從而丟失數據，因為即使網絡好了以后，已經分成的兩個集群並不會再合並，為避免腦裂現象，需要設置master候選節點的數量：（master_eligible_nodes/2）+1

 

 

4. 配置文件 es_statefulset.yaml

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: esnode
  labels:
    app: es-cluster
spec:
  serviceName: hotes-cluster-9300
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: es-cluster
    spec:
      #securityContext:
      #  fsGroup: 1000
      initContainers:
        - name: fix-permissions
          image: video-harbor.ks-live.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sh", "-c", "chown -R 1000:1000 /usr/share/elasticsearch/data"]
          volumeMounts:
            - name: es-data
              mountPath: /usr/share/elasticsearch/data
        - name: init-ulimit
          image: xxx.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sh", "-c", "ulimit -n 655350"]          
        - name: init-sysctl
          image: xxx.com/public/busybox:latest
          imagePullPolicy: IfNotPresent
          securityContext:
            privileged: true
          command: ["sysctl", "-w", "vm.max_map_count=262144"]
      nodeSelector: 
        zone: "xxx-indexer-yuanzhan_json"
      containers:
        - name: elasticsearch
          resources:
            requests:
              memory: 300Mi
              cpu: 0.01
            limits:
              memory: 60.5Gi
              cpu: 15
          securityContext:
            privileged: true
            runAsUser: 0
            capabilities:
              add:
                - IPC_LOCK
                - SYS_RESOURCE
          image: xxx.com/st/elasticsearch-oss:6.6.2
          imagePullPolicy: IfNotPresent
          env:
            - name: ES_JAVA_OPTS
              value: "-Xms9800m -Xmx9800m"
            - name: HOSTNAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
          #command: ["/bin/sleep","9000"]
          readinessProbe:
            httpGet:
              scheme: HTTP
              path: /_cluster/health?local=true
              port: 9200
            initialDelaySeconds: 5
          ports:
            - containerPort: 9200
              name: es-http
            - containerPort: 9300
              name: es-transport
          volumeMounts:
            - name: es-data
              mountPath: /usr/share/elasticsearch/data
            - name: elasticsearch-config
              mountPath: /usr/share/elasticsearch/config/elasticsearch.yml
              subPath: elasticsearch.yml
      volumes:
        - name: elasticsearch-config
          configMap:
            name: hotes-config
            items:
              - key: elasticsearch.yml
                path: elasticsearch.yml
  volumeClaimTemplates:
    - metadata:
        name: es-data
      spec:
        accessModes: [ "ReadWriteOnce" ]
        resources:
          requests:
            storage: 1Gi
        storageClassName: rbd

使用elasticsearch-oss:6.6.2的鏡像，鏡像內elasticsearch用戶的id是1000，將configmap掛載到 /usr/share/elasticsearch/config/elasticsearch.yml，並賦給elasticsearch權限，使用ceph搭建的分布式存儲，動態創建1G的空間用於存儲數據

 

5. 使用域名 http://www.hotes.com/_cluster/state/nodes?pretty 查看集群狀態

或者通過nodePort的方式 http://host-ipaddress:8831/_cluster/state/nodes?pretty 查看集群狀態

 

參考：https://github.com/13428282016/elasticsearch-CN/wiki/es-setup--elasticsearch