1 - minikube
Minikube is a tool that makes it easy to run Kubernetes locally.
Minikube runs a single-node Kubernetes cluster inside a Virtual Machine (VM) on your laptop for users looking to try out Kubernetes or develop with it day-to-day.
Minikube 是一個易於在本地運行 Kubernetes 的工具，
可在筆記本電腦上的虛擬機內輕松創建單機版 Kubernetes 集群，便於嘗試 Kubernetes 或使用 Kubernetes 日常開發。
- HomePage：https://minikube.sigs.k8s.io/
- Documentation：https://minikube.sigs.k8s.io/docs/
- Getting Started：https://minikube.sigs.k8s.io/docs/start/
- Installation：https://minikube.sigs.k8s.io/docs/start/linux/
- GitHub：https://github.com/kubernetes/minikube
- Releases：https://github.com/kubernetes/minikube/releases
kubernetes.io
- Installing Minikube：https://kubernetes.io/docs/tasks/tools/install-minikube/
- Installing Kubernetes with Minikube：https://kubernetes.io/docs/setup/learning-environment/minikube/
Hello Minikube
- Hello Minikube：https://kubernetes.io/docs/tutorials/hello-minikube/
This tutorial shows you how to run a simple Hello World Node.js app on Kubernetes using Minikube and Katacoda.
注意：
- Minikube是由Kubernetes開源社區開發，專為本地部署而設計的一個版本，但Minikube只能局限於包含一個節點的本地集群，因此對於生產級別的多節點Kubernetes集群來說，很難在本地進行模擬測試。。
- 從1.5.0版本開始，Minikube缺省使用本地最好的驅動來創建Kubernetes本地環境，因此需先確保安裝了類似VirtualBox、Hyper-V、KVM、Docker等。
2 - 環境准備
虛擬機中已安裝docker
[anliven@anliven ~]$ uname -a
Linux anliven 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[anliven@anliven ~]$
[anliven@anliven ~]$ cat /etc/system-release
CentOS Linux release 7.8.2003 (Core)
[anliven@anliven ~]$
[anliven@anliven ~]$ docker version
Client: Docker Engine - Community
Version: 19.03.11
API version: 1.40
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:13:48 2020
OS/Arch: linux/amd64
Experimental: false
Server: Docker Engine - Community
Engine:
Version: 19.03.11
API version: 1.40 (minimum version 1.12)
Go version: go1.13.10
Git commit: 42e35e61f3
Built: Mon Jun 1 09:12:26 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.2.13
GitCommit: 7ad184331fa3e55e52b890ea95e65ba581ae3429
runc:
Version: 1.0.0-rc10
GitCommit: dc9208a3303feef5b3839f4323d9beb36df0a9dd
docker-init:
Version: 0.18.0
GitCommit: fec3683
[anliven@anliven ~]$
3 - 用戶權限設置
[root@anliven ~]# visudo
[root@anliven ~]#
[root@anliven ~]# cat /etc/sudoers |grep anliven
anliven ALL=(ALL) NOPASSWD: ALL
[root@anliven ~]#
[root@anliven ~]# exit
[anliven@anliven ~]# sudo usermod -aG docker $USER && newgrp docker
4 - 安裝kubectl
使用 kubectl 與集群進行交互。
kubectl是Kubernetes命令行工具，可以部署和管理應用， 查看各種資源， 創建、 刪除和更新各種組件。
- Overview of kubectl：https://kubernetes.io/docs/reference/kubectl/overview/
- Install and Set Up kubectl：https://kubernetes.io/docs/tasks/tools/install-kubectl/
命令
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
chmod +x kubectl
sudo mv ./kubectl /usr/local/bin/kubectl
kubectl version --client
示例
[anliven@anliven K8s]$ pwd
/home/anliven/K8s
[anliven@anliven K8s]$
[anliven@anliven K8s]$ curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 41.0M 100 41.0M 0 0 2819k 0 0:00:14 0:00:14 --:--:-- 2832k
[anliven@anliven K8s]$
[anliven@anliven K8s]$ chmod +x kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ sudo mv ./kubectl /usr/local/bin/kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ ls -l /usr/local/bin/kubectl
-rwxrwxr-x 1 anliven anliven 43003904 Oct 14 22:43 /usr/local/bin/kubectl
[anliven@anliven K8s]$
[anliven@anliven K8s]$ kubectl version --client
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.3", GitCommit:"1e11e4a2108024935ecfcb2912226cedeafd99df", GitTreeState:"clean", BuildDate:"2020-10-14T12:50:19Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
[anliven@anliven K8s]$
kubectl命令幫助
kubectl -h # 查看子命令列表
kubectl options # 查看全局選項
Use "kubectl <command> --help" for more information about a given command.
kubectl <command> --help # 查看子命令的幫助
Use "kubectl options" for a list of global command-line options (applies to all commands).
kubectl [command] [PARAMS] -o=<format> # 設置輸出格式（如 json、yaml、jsonpath 等）
kubectl explain RESOURCE [options] # 查看資源的定義
kubectl explain -h
kubectl get [flags] [options] # 顯示一個或多個資源
kubectl get -h
5 - 安裝minikube
命令
curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
sudo install minikube-linux-amd64 /usr/local/bin/minikube
minikube version
示例
[anliven@anliven K8s]$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 56.0M 100 56.0M 0 0 2510k 0 0:00:22 0:00:22 --:--:-- 2234k
[anliven@anliven K8s]$
[anliven@anliven K8s]$ sudo install minikube-linux-amd64 /usr/local/bin/minikube
[anliven@anliven K8s]$
[anliven@anliven K8s]$ ls -l /usr/local/bin/minikube
-rwxr-xr-x 1 root root 58733392 Oct 14 22:46 /usr/local/bin/minikube
[anliven@anliven K8s]$
[anliven@anliven K8s]$ minikube version
minikube version: v1.14.0
commit: b09ee50ec047410326a85435f4d99026f9c4f5c4
6 - 從github下載並安裝kubectl和minikube
如果無法訪問google站點，也可以從github下載並安裝kubectl和minikube。
kubectl
- 訪問 https://github.com/kubernetes/kubernetes/releases
- 點擊類似
CHANGELOG-*.*.md的鏈接，在“Client Binaries”區域找到匹配的二進制包， - 下載並解壓，然后放入
/usr/local/bin目錄
minikube
找到匹配的二進制包並下載： https://github.com/kubernetes/minikube/releases/
curl -Lo minikube https://github.com/kubernetes/minikube/releases/download/v1.14.0/minikube-linux-arm64
chmod +x minikube
sudo mv minikube /usr/local/bin
也可以使用阿里雲的minikube地址： https://github.com/AliyunContainerService/minikube
curl -Lo minikube https://kubernetes.oss-cn-hangzhou.aliyuncs.com/minikube/releases/v1.13.0/minikube-linux-amd64
chmod +x minikube
sudo mv minikube /usr/local/bin/
7 - 啟動與運行minikube
示例
[anliven@anliven ~]$ minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Automatically selected the docker driver # 本例中自動選擇了docker驅動來創建Kubernetes本地環境
* Starting control plane node minikube in cluster minikube
* Pulling base image ... # 第一次會拉取基礎鏡像，用時較久
* Creating docker container (CPUs=2, Memory=2200MB) ...
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
> kubelet.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubectl.sha256: 65 B / 65 B [--------------------------] 100.00% ? p/s 0s
> kubeadm: 37.30 MiB / 37.30 MiB [-------------] 100.00% 822.08 KiB p/s 47s
> kubectl: 41.01 MiB / 41.01 MiB [-------------] 100.00% 881.98 KiB p/s 48s
> kubelet: 104.88 MiB / 104.88 MiB [------------] 100.00% 1.58 MiB p/s 1m6s
* Verifying Kubernetes components...
* Enabled addons: storage-provisioner, default-storageclass
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
[anliven@anliven ~]$ docker images |grep k8s
gcr.io/k8s-minikube/kicbase v0.0.13 90f1294ff9ac 2 weeks ago 800MB
[anliven@anliven ~]$
[anliven@anliven ~]$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5d520392622e gcr.io/k8s-minikube/kicbase:v0.0.13 "/usr/local/bin/entr…" 4 hours ago Up 4 hours 127.0.0.1:32775->22/tcp, 127.0.0.1:32774->2376/tcp, 127.0.0.1:32773->5000/tcp, 127.0.0.1:32772->8443/tcp minikube
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ip # 獲取minikube的ip地址
192.168.49.2
[anliven@anliven ~]$
可以根據需要配置啟動參數，例如minikube start --cpus=2 --memory=2048 -- --kubernetes-version=v1.19.3
--driver 從1.5.0版本開始，Minikube缺省使用本地最好的驅動來創建Kubernetes本地環境
--image-mirror-country cn 將缺省利用 registry.cn-hangzhou.aliyuncs.com/google_containers 作為安裝Kubernetes的容器鏡像倉庫
--iso-url 指定鏡像地址下載相應的 .iso 文件
--registry-mirror 為了拉取Docker Hub鏡像，需要為 Docker daemon 配置鏡像加速
--cpus=2 為minikube虛擬機分配CPU核數
--memory=2048mb 為minikube虛擬機分配內存數
--kubernetes-version minikube 虛擬機將使用的 kubernetes 版本
此時，使用kubectl get nodes檢查集群所包含節點的命令可以返回minikube的主機名和狀態。
[anliven@anliven ~]$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
minikube Ready master 130d v1.19.2
通過minikube start命令可以重啟現有的minikube
[anliven@anliven ~]$ minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Using the docker driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing docker container for "minikube" ... # 重啟現有的minikube
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
如果minikube突然出現不穩定的狀況，可以使用minikube stop和minikube delete命令刪除當前的minikube，然后通過 minikube start命令重新安裝。
一個部署示例：
[anliven@anliven ~]$ kubectl get all -A # 查看所有的資源
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system pod/coredns-f9fd979d6-qjj7q 1/1 Running 0 4m
kube-system pod/etcd-minikube 1/1 Running 0 4m2s
kube-system pod/kube-apiserver-minikube 1/1 Running 0 4m2s
kube-system pod/kube-controller-manager-minikube 1/1 Running 0 4m1s
kube-system pod/kube-proxy-ldbqb 1/1 Running 0 4m
kube-system pod/kube-scheduler-minikube 1/1 Running 0 4m1s
kube-system pod/storage-provisioner 1/1 Running 0 3m58s
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m8s
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 4m5s
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 4m5s
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system deployment.apps/coredns 1/1 1 1 4m5s
NAMESPACE NAME DESIRED CURRENT READY AGE
kube-system replicaset.apps/coredns-f9fd979d6 1 1 1 4m
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m12s
[anliven@anliven ~]$
[anliven@anliven ~]$ docker images|grep nginx # 查看鏡像
nginx latest 7e4d58f0e5f3 5 months ago 133MB
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl create deployment nginx --image=nginx:latest # 部署應用
deployment.apps/nginx created
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-55649fd747-lg4d6 0/1 ContainerCreating 0 17s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4m51s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 0/1 1 0 17s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-55649fd747 1 1 0 17s
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl expose deployment nginx --type=NodePort --port=8888 # 暴露8080端口，類型為NodePort（在每個Node上分配一個端口作為外部訪問入口）
service/nginx exposed
[anliven@anliven ~]$
[anliven@anliven ~]$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-55649fd747-lg4d6 1/1 Running 0 9m26s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10m
service/nginx NodePort 10.102.183.1 <none> 8888:31765/TCP 8m33s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 9m26s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-55649fd747 1 1 1 9m26s
[anliven@anliven ~]$
8 - Minikube 命令
命令幫助
Use "minikube <command> --help" for more information about a given command.
Use "minikube options" for a list of global command-line options (applies to all commands).
命令選項
[anliven@anliven ~]$ minikube options
The following options can be passed to any command:
--alsologtostderr=false: log to standard error as well as files
-b, --bootstrapper='kubeadm': The name of the cluster bootstrapper that will set up the Kubernetes cluster.
--log_backtrace_at=:0: when logging hits line file:N, emit a stack trace
--log_dir='': If non-empty, write log files in this directory
--logtostderr=false: log to standard error instead of files
-p, --profile='minikube': The name of the minikube VM being used. This can be set to allow having multiple instances of minikube independently.
--stderrthreshold=2: logs at or above this threshold go to stderr
-v, --v=0: log level for V logs
--vmodule=: comma-separated list of pattern=N settings for file-filtered logging
[anliven@anliven ~]$
一些命令
Minikube 集群
minikube version # 查看版本
minikube status # 查看狀態
minikube profile list # 查看屬性
minikube addons list # 查看當前支持的插件
minikube service list # 查看服務列表
minikube node list # 參看添加的node
minikube ssh # 登錄
minikube ip # 獲取IP地址
minikube start # 啟動
minikube stop # 停止
minikube delete # 刪除
minikube dashboard # 在默認瀏覽器中啟動儀表盤
rm-rf ~/.minikube # 重置(清理所有緩存的鏡像，重頭開始)
一些示例
[anliven@anliven ~]$ minikube status # 查看集群狀態
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube profile list # 查看集群屬性
|----------|-----------|---------|--------------|------|---------|---------|
| Profile | VM Driver | Runtime | IP | Port | Version | Status |
|----------|-----------|---------|--------------|------|---------|---------|
| minikube | docker | docker | 192.168.49.2 | 8443 | v1.19.2 | Running |
|----------|-----------|---------|--------------|------|---------|---------|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube addons list # 查看集群當前支持的插件
|-----------------------------|----------|--------------|
| ADDON NAME | PROFILE | STATUS |
|-----------------------------|----------|--------------|
| ambassador | minikube | disabled |
| csi-hostpath-driver | minikube | disabled |
| dashboard | minikube | enabled ✅ |
| default-storageclass | minikube | enabled ✅ |
| efk | minikube | disabled |
| freshpod | minikube | disabled |
| gcp-auth | minikube | disabled |
| gvisor | minikube | disabled |
| helm-tiller | minikube | disabled |
| ingress | minikube | disabled |
| ingress-dns | minikube | disabled |
| istio | minikube | disabled |
| istio-provisioner | minikube | disabled |
| kubevirt | minikube | disabled |
| logviewer | minikube | disabled |
| metallb | minikube | disabled |
| metrics-server | minikube | disabled |
| nvidia-driver-installer | minikube | disabled |
| nvidia-gpu-device-plugin | minikube | disabled |
| olm | minikube | disabled |
| pod-security-policy | minikube | disabled |
| registry | minikube | disabled |
| registry-aliases | minikube | disabled |
| registry-creds | minikube | disabled |
| storage-provisioner | minikube | enabled ✅ |
| storage-provisioner-gluster | minikube | disabled |
| volumesnapshots | minikube | disabled |
|-----------------------------|----------|--------------|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube service list # 查看集群服務列表
|----------------------|---------------------------|--------------|-----|
| NAMESPACE | NAME | TARGET PORT | URL |
|----------------------|---------------------------|--------------|-----|
| default | kubernetes | No node port |
| kube-system | kube-dns | No node port |
| kubernetes-dashboard | dashboard-metrics-scraper | No node port |
| kubernetes-dashboard | kubernetes-dashboard | No node port |
|----------------------|---------------------------|--------------|-----|
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube node list # node列表
minikube 192.168.49.2
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ip # ip地址
192.168.49.2
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube ssh # 登錄
docker@minikube:~$ whoami
docker
docker@minikube:~$ exit
logout
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube stop # 停止
* Stopping node "minikube" ...
* Powering off "minikube" via SSH ...
* 1 nodes stopped.
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube status # 查看狀態
minikube
type: Control Plane
host: Stopped
kubelet: Stopped
apiserver: Stopped
kubeconfig: Stopped
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube start # 重新啟動，不會“Pulling base image ...”
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Using the docker driver based on existing profile
* Starting control plane node minikube in cluster minikube
* Restarting existing docker container for "minikube" ...
* Preparing Kubernetes v1.19.2 on Docker 19.03.8 ...
* Verifying Kubernetes components...
* Enabled addons: default-storageclass, storage-provisioner, dashboard
* Done! kubectl is now configured to use "minikube" by default
[anliven@anliven ~]$
[anliven@anliven ~]$ minikube status # 查看狀態
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
[anliven@anliven ~]$
啟動Dashboard
minikube dashboard # 在默認瀏覽器中啟動儀表盤
9 - 問題處理
1 - minikube啟動失敗
[root@anliven K8s]# minikube start
* minikube v1.14.0 on Centos 7.8.2003 (vbox/amd64)
* Automatically selected the docker driver
* The "docker" driver should not be used with root privileges.
* If you are running minikube within a VM, consider using --driver=none:
* https://minikube.sigs.k8s.io/docs/reference/drivers/none/
X Exiting due to DRV_AS_ROOT: The "docker" driver should not be used with root privileges.
處理方法： 使用非root用戶，並將此用戶添加到docker組： sudo usermod -aG docker $USER && newgrp docker
2 - dashboard啟動失敗
[anliven@anliven ~]$ minikube dashboard
* Enabling dashboard ...
* Verifying dashboard health ...
* Launching proxy ...
* Verifying proxy health ...
* Opening http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/ in your default browser...
START /usr/bin/firefox "http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/"
Failed to open connection to "session" message bus: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
Running without a11y support!
Error: no DISPLAY environment variable specified
xdg-open: no method available for opening 'http://127.0.0.1:37006/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/'
X Exiting due to HOST_BROWSER: failed to open browser: exit status 3
[anliven@anliven ~]$
處理方法： 需要圖形界面下的命令行中執行minikube dashboard，以便啟動瀏覽器