helm部署gitlab

1.storageclass（存儲類）概念

storageclass是一個存儲類，k8s集群管理員通過創建storageclass可以動態生成一個存儲卷供k8s用戶使用。

2.storageclass資源定義

每個StorageClass都包含字段provisioner，parameters和reclaimPolicy，當需要動態配置屬於該類的PersistentVolume時使用這些字段。

StorageClass對象的名稱很重要，是用戶可以請求特定類的方式。管理員在首次創建StorageClass對象時設置類的名稱和其他參數，並且在創建對象后無法更新這些對象。

管理員可以為不請求任何特定類綁定的PVC指定默認的StorageClass

（1）yaml文件示例說明：

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: standard
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Retain
mountOptions:
  - debug
volumeBindingMode: Immediate

（2）Provisioner

storageclass需要有一個供應者，用來確定我們使用什么樣的存儲來創建pv

常見的provisioner供應者如下：

provisioner既可以是內部供應程序，也可以由外部供應商提供，如果是外部供應商可以參考https://github.com/kubernetes-incubator/external-storage/下提供的方法創建storageclass的provisioner，例如，NFS不提供內部配置程序，但可以使用外部配置程序。一些外部供應商列在存儲庫https://github.com/kubernetes-incubator/external-storage下。

nfs的provisioner：

https://github.com/kubernetes-incubator/external-storage/tree/master/nfs/deploy/kubernetes

Reclaim Policy

由存儲類動態創建持久化存儲卷（pv）時可以指定reclaimPolicy字段，這個字段中指定的回收策略可以是Delete或Retain。如果在創建StorageClass對象時未指定reclaimPolicy，則默認為Delete。

Mount Options（掛載選項）

如果Volume Plugin不支持這個掛載選項，但是指定了，就會使provisioner創建失敗

Volume Binding Mode

這個字段用來說明什么時候進行卷綁定和動態配置；

默認情況下，立即模式表示一旦創建了PersistentVolumeClaim，就會發生卷綁定和動態配置。對於受拓撲約束且無法從群集中的所有節點全局訪問的存儲后端，將在不知道Pod的調度要求的情況下綁定或配置PersistentVolumes。這可能導致不可調度的Pod。

集群管理員可以通過指定WaitForFirstConsumer模式來解決此問題，該模式將延遲綁定和配置PersistentVolume，直到創建使用PersistentVolumeClaim的Pod。將根據Pod的調度約束指定的拓撲選擇或配置PersistentVolumes。這些包括但不限於資源需求，節點選擇器，pod親和力和反親和力，以及污點和容忍度。

通過nfs實現存儲類的動態供給

1.安裝nfs服務

選擇自己的任意一台機器，我選擇k8s的master1節點，對應的機器ip是192.168.0.6，在192.168.0.6上執行如下步驟，大家在自己環境找一台k8s節點的機器安裝就可以了~

（1）yum安裝nfs

yum install nfs-utils -y
systemctl start nfs
chkconfig nfs on

（2）在master1上創建一個nfs共享目錄

mkdir  /data/nfs_pro   -p

#修改/etc/exports文件，把剛才生成的共享目錄添加到這個文件里

cat  /etc/exports

 /data/nfs_pro    192.168.0.0/24(rw,no_root_squash)

讓剛才配置文件生效

exportfs -arv  

systemctl restart nfs

（3）k8s的各個node節點也需要安裝nfs

yum install nfs-utils -y
systemctl start nfs
chkconfig nfs on

2.創建運行nfs-provisioner的sa賬號

cat  serviceaccount.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-provisioner

 通過kubectl更新yaml文件

kubectl apply -f serviceaccount.yaml

3.對sa賬號做rbac授權

cat service-rbac.yaml

kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
  - apiGroups: [""]
    resources: ["services", "endpoints"]
    verbs: ["get"]
  - apiGroups: ["extensions"]
    resources: ["podsecuritypolicies"]
    resourceNames: ["nfs-provisioner"]
    verbs: ["use"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-provisioner
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-provisioner
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-provisioner
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-provisioner
  apiGroup: rbac.authorization.k8s.io

 通過kubectl更新yaml文件

kubectl apply -f service-rbac.yaml

4.通過deployment創建pod用來運行nfs-provisioner

cat  deployment.yaml

kind: Deployment
apiVersion: apps/v1
metadata:
  name: nfs-provisioner
spec:
  selector:
    matchLabels:
      app: nfs-provisioner
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-provisioner
    spec:
      serviceAccount: nfs-provisioner
      containers:
        - name: nfs-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner:latest
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: example.com/nfs
            - name: NFS_SERVER
              value: 192.168.0.6
            - name: NFS_PATH
              value: /data/storage
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.0.6
            path: /data/storage

 通過kubectl更新yaml文件

kubectl apply -f deployment.yaml

查看nfs provisioner的pod是否創建成功

kubectl get pods

顯示如下，說明創建成功

NAME                               READY   STATUS    RESTARTS   AGE
nfs-provisioner-7d6859cc59-pzr7n   1/1     Running   0          72s

 創建storageclass

cat  class.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: nfs-storage
provisioner: example.com/nfs

通過kubectl更新yaml文件

kubectl apply -f class.yaml

查看storageclass是否創建成功

kubectl get storageclass

顯示如下，說明創建成功

NAME          PROVISIONER       RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
nfs-storage   example.com/nfs   Delete          Immediate           false                  11h

 通過helm部署gitlab

1、解壓gitlab-ce.tar.gz

tar zxvf gitlab-ce.tar.gz

2、進入到解壓路徑，通過helm安裝gitlab、postgresql、redis

cd gitlab-ce

通過helm安裝部署
helm install .

查看是否部署成功

kubectl get pods

顯示如下，看到STATUS都是running狀態，說明部署成功

NAME                                             READY   STATUS    RESTARTS   AGE
alliterating-quail-gitlab-ce-756ddcb6d5-rwnzj    1/1     Running   0          3m56s
alliterating-quail-postgresql-68bc89cd7f-slt6g   1/1     Running   0          3m56s
alliterating-quail-redis-7954fbbfbc-5jqfx        1/1     Running   0          3m56s

 查看gitlab在宿主機暴露的端口

kubectl get svc | grep gitlab

alliterating-quail-gitlab-ce    NodePort    10.105.171.114   <none>        22:32503/TCP,80:31572/TCP,443:32

通過上面可以看到容器里的80端口在宿主機映射的端口是31572，所以瀏覽器訪問k8s的master1節點ip:31572即可，我在瀏覽器訪問如下地址：

192.168.0.6:31572

默認的用戶名和密碼分別是：

root

gitlab0920

輸入用戶名和密碼之后即可登陸到gitlab的web界面

 

gitlab-ce.tar.gz里涉及的Chart文件說明講解

 1.上面的壓縮包解壓之后，在gitlab-ce目錄下，有個values.yaml文件，內容及解釋如下

## GitLab CE image
## ref: https://hub.docker.com/r/gitlab/gitlab-ce/tags/
##
image: gitlab/gitlab-ce:10.6.2-ce.0
#gitlab使用的鏡像，我們安裝的是10版本

## Specify a imagePullPolicy
## 'Always' if imageTag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
# imagePullPolicy:
imagePullPolicy: IfNotPresent
#鏡像拉取策略，IfNotPresent表示本地不存在，就從官方拉取

## The URL (with protocol) that your users will use to reach the install.
## ref: https://docs.gitlab.com/omnibus/settings/configuration.html#configuring-the-external-url-for-gitlab
##
externalUrl: http://gitlab.lucky.com/
#可訪問的域名，這里我隨便寫了有一個，暫時用不到
## Change the initial default admin password if set. If not set, you'll be
## able to set it when you first visit your install.
##
gitlabRootPassword: "gitlab0920"
#登陸gitlab web界面的密碼，可以隨便寫，符合密碼復雜度即可
#登陸gitlab web界面的默認用戶是root

## For minikube, set this to NodePort, elsewhere use LoadBalancer
## ref: http://kubernetes.io/docs/user-guide/services/#publishing-services---service-types
##
serviceType: NodePort
#service的類型，NodePort表示暴露出來的服務可以在k8s集群外部訪問

## Ingress configuration options
##
ingress:
  annotations:
      # kubernetes.io/ingress.class: nginx
      # kubernetes.io/tls-acme: "true"
  enabled: false
  tls:
      # - secretName: gitlab.cluster.local
      #   hosts:
      #     - gitlab.cluster.local
  url: gitlab.cluster.local

#上面是ingress相關的，暫時我們用不到

## Configure external service ports
## ref: http://kubernetes.io/docs/user-guide/services/
sshPort: 22
#容器里暴露的22端口
httpPort: 80
#容器里暴露的80端口
httpsPort: 443
#容器里暴露的443端口
## livenessPort Port of liveness probe endpoint
livenessPort: http
## readinessPort Port of readiness probe endpoint
readinessPort: http

## Configure resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
resources:
  ## GitLab requires a good deal of resources. We have split out Postgres and
  ## redis, which helps some. Refer to the guidelines for larger installs.
  ## ref: https://docs.gitlab.com/ce/install/requirements.html#hardware-requirements
  requests:
    memory: 1Gi
    cpu: 500m
  limits:
    memory: 2Gi
    cpu: 1

## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
## ref: https://docs.gitlab.com/ce/install/requirements.html#storage
##
persistence:
#下面配置的是跟持久化存儲相關的內容
  ## This volume persists generated configuration files, keys, and certs.
  ##
  gitlabEtc:
    enabled: true
    size: 1Gi
    ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass>
    ## Default: volume.alpha.kubernetes.io/storage-class: default
    ##
    storageClass: "nfs-storage"
#開啟了storageclass，這里的storageclass是nfs-storage，
#就是我們上面創建的那個nfs-storage
    accessMode: ReadWriteOnce
#訪問模式，單路獨寫
  ## This volume is used to store git data and other project files.
  ## ref: https://docs.gitlab.com/omnibus/settings/configuration.html#storing-git-data-in-an-alternative-directory
  ##
  gitlabData:
    enabled: true
    size: 10Gi
    ## If defined, volume.beta.kubernetes.io/storage-class: <storageClass>
    ## Default: volume.alpha.kubernetes.io/storage-class: default
    ##
    storageClass: "nfs-storage"
#開啟了storageclass，這里的storageclass是nfs-storage，
#就是我們上面創建的那個nfs-storage
    accessMode: ReadWriteOnce
#訪問模式，單路獨寫
## Configuration values for the postgresql dependency.
## ref: https://github.com/kubernetes/charts/blob/master/stable/postgresql/README.md
##
postgresql:
  # 9.6 is the newest supported version for the GitLab container
  imageTag: "9.6.5"
  cpu: 1000m
  memory: 1Gi

  postgresUser: gitlab
  postgresPassword: gitlab
  postgresDatabase: gitlab

  persistence:
    size: 10Gi
    storageClass: "nfs-storage"
## Configuration values for the redis dependency.
## ref: https://github.com/kubernetes/charts/blob/master/stable/redis/README.md
##
redis:
  redisPassword: "gitlab"

  resources:
    requests:
      memory: 1Gi

  persistence:
    size: 10Gi
    storageClass: "nfs-storage"
#開啟了storageclass，這里的storageclass是nfs-storage，
#就是我們上面創建的那個nfs-storage

 原文：https://mp.weixin.qq.com/s/vOI1o7iH6gz4-aqwzZQyGQ