kuberneets 1.17 設置 kube-reserved, system-reserved

主要修改 kubelet的啟動命令 增加reserved相關參數

 

[Unit]
   Description=Kubernetes Kubelet Server
   After=docker.service
   Requires=docker.service
[Service]
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/pids/system.slice/kubelet.service
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpu/system.slice/kubelet.service
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpuacct/system.slice/kubelet.service
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/cpuset/system.slice/kubelet.service
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/memory/system.slice/kubelet.service
   ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/systemd/system.slice/kubelet.service
   WorkingDirectory=/var/lib/kubelet
   EnvironmentFile=-/etc/kubernetes/config
   EnvironmentFile=-/etc/kubernetes/kubelet
   ExecStart=/root/k8s/cmd/kubelet \
               $KUBE_LOGTOSTDERR \
               $KUBE_LOG_LEVEL \
               $KUBELET_API_SERVER \
               $KUBELET_PORT \
               $KUBELET_HOSTNAME \
               $KUBELET_POD_INFRA_CONTAINER \
               $KUBELET_ARGS \
               --network-plugin=cni \
               --feature-gates=LocalStorageCapacityIsolationFSQuotaMonitoring=true \
               --kube-reserved=cpu=6,memory=8Gi,ephemeral-storage=4Gi \
               --enforce-node-allocatable=pods,kube-reserved \
               --kube-reserved-cgroup=/system.slice/kubelet.service \
               --root-dir=/home/kubelet \
               --v=2
Restart=on-failure
[Install]
WantedBy=multi-user.target

　　

在實際使用中沒有增加system-reserved參數，由於有些場景預留資源給少了會早知 device busy的錯誤 可以參考 https://github.com/rootsongjc/qa/issues/3

上邊那些 ExecStartPre  還創建的，具體參考上邊的issue

添加完成后 重啟kubelet 查看資源信息

Capacity:
  cpu:                48
  ephemeral-storage:  1892365528Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             131142712Ki
  nvidia.com/gpu:     2
  pods:               110
Allocatable:
  cpu:                42
  ephemeral-storage:  1888171224Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  memory:             122242104Ki
  nvidia.com/gpu:     2
  pods:               110

　　

 

如果 

grep CGROUP_HUGETLB /boot/config-xxxxx

CONFIG_CGROUP_HUGETLB=y

需要在kubelet.service中加入

ExecStartPre=/usr/bin/mkdir -p /sys/fs/cgroup/hugetlb/system.slice/kubelet.service

來解決 
hugetlb.1GB.limit_in_bytes: no such file or directory