1、概述
我們都知道，當我們在k8s中創建了一個ingress對象之后，ingress-nginx-controller都會根據我們在ingress中配置的規則呢，重新生成一個nginx.conf的配置文件，然后通過重新加載配置的方法，來使得的nginx.conf的配置生效。
那么問題是，當我們在k8s中進行哪些操作的時候，會觸發ingress-nginx-controller中的nginx.conf進行重新加載？
本文檔，我們就通過一些測試，來看下，哪些操作，哪些動作，會導致nginx.conf這個配置文件被重新的加載。
2、配置重新加載的一些情況
在ingress的官方文檔中，我們看到，當出現如下的情況的時候，需要重新加載配置：
- 創建新的ingress資源
- 在ingrss中增加TLS的部分
- ingress中注釋的變化，並且應西鄉不只一個upstream的配置
- ingress中增加或者減少一個path
- 刪除ingress,service,secret
- ingress中丟失的一些引用的對象，變得可用了（或者增加上了），比如說，service或者secret
- secret的更新
看到這些，其實，不是很有直觀的感受，那么我們針對上面提到的情況做一些實驗，看一下，到底nginx的配置是否有變化，是否被重新的加載，然后配置有什么樣的變化。
OK，我們開始··· ···
3、針對每種情況的實驗
3.1、創建新的ingress
創建ingress對象之前，我們先看下ingress-nginx-controller容器中nginx.conf配置文件的時間和nginx進程的啟動時間
bash-5.1$ ls -l nginx.conf ;ps -ef | grep nginx
-rw-r--r-- 1 www-data www-data 25615 Apr 18 06:45 nginx.conf
1 www-data 0:00 /usr/bin/dumb-init -- /nginx-ingress-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --configmap=ingress-nginx/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key
8 www-data 5h46 /nginx-ingress-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --configmap=ingress-nginx/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key
33 www-data 0:02 nginx: master process /usr/local/nginx/sbin/nginx -c /etc/nginx/nginx.conf
1422 www-data 0:00 vi nginx.conf
5460 www-data 0:27 nginx: worker process
5461 www-data 0:27 nginx: worker process
5462 www-data 0:27 nginx: worker process
5463 www-data 0:27 nginx: worker process
5464 www-data 0:40 nginx: worker process
5465 www-data 0:27 nginx: worker process
5466 www-data 0:26 nginx: worker process
5467 www-data 0:26 nginx: worker process
5468 www-data 0:01 nginx: cache manager process
5746 www-data 0:00 grep nginx
bash-5.1$ 
我們通過下面的命令，創建一個ingress對象
kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-new-ingress
namespace: default
spec:
ingressClassName: nginx
rules:
- host: nginx-new.k8s.com
http:
paths:
- backend:
service:
name: nginx-new
port:
number: 80
path: /
pathType: Prefix
EOF查看創建好的ingress
[root@nccztsjb-node-23 ~]# kubectl get ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
nginx-new-ingress nginx nginx-new.k8s.com 80 5s
[root@nccztsjb-node-23 ~]#進入ingress-nginx-controller容器中查看nginx的配置文件的時間及容器重新加載的時間
bash-5.1$ ls -l nginx.conf ;ps -ef | grep nginx
-rw-r--r-- 1 www-data www-data 29592 Apr 20 03:23 nginx.conf
1 www-data 0:00 /usr/bin/dumb-init -- /nginx-ingress-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --configmap=ingress-nginx/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key
8 www-data 5h46 /nginx-ingress-controller --election-id=ingress-controller-leader --controller-class=k8s.io/ingress-nginx --configmap=ingress-nginx/ingress-nginx-controller --validating-webhook=:8443 --validating-webhook-certificate=/usr/local/certificates/cert --validating-webhook-key=/usr/local/certificates/key
33 www-data 0:02 nginx: master process /usr/local/nginx/sbin/nginx -c /etc/nginx/nginx.conf
1422 www-data 0:00 vi nginx.conf
5795 www-data 0:00 nginx: worker process
5796 www-data 0:00 nginx: worker process
5797 www-data 0:00 nginx: worker process
5798 www-data 0:00 nginx: worker process
5799 www-data 0:00 nginx: worker process
5800 www-data 0:00 nginx: worker process
5801 www-data 0:00 nginx: worker process
5802 www-data 0:00 nginx: worker process
5803 www-data 0:00 nginx: cache manager process
6074 www-data 0:00 grep nginx
我們發現，nginx.conf配置文件的時間已經發生了變化，並且，各個worker進程的運行時間也發生了變化。說明配置進行了重新的加載。
3.2、ingress中增加或減少path
對上面的那個yaml配置文件中，增加一個path路徑，項下面這樣
kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-new-ingress
namespace: default
spec:
ingressClassName: nginx
rules:
- host: nginx-new.k8s.com
http:
paths:
- backend:
service:
name: nginx-new
port:
number: 80
path: /
pathType: Prefix
- backend:
service:
name: nginx-new01
port:
number: 81
path: /new01
pathType: Prefix
EOF更新了之后，查看ingress-nginx-controller配置的變化
配置文件的時間發生了變化，同樣的，通過ingress-nginx-controller pod的日志，也可以看到是否進行了加載
kubectl logs -f -n ingress-nginx ingress-nginx-controller-z6bhs
...省略...
W0420 06:00:27.398445 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new01": no object matching key "default/nginx-new01" in local store
I0420 06:00:27.398665 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:00:27.536060 8 controller.go:172] "Backend successfully reloaded"
I0420 06:00:27.537326 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration從這里的日志我們可以看到，配置發生了變化，需要進行后端的reload.(Configuration changes detected, backend reload required)，然后是pod由於配置的變化而觸發的nginx的reload的操作。。
type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
OK，通過這個實驗，我們也發現了，當ingress規則中path增加的時候，會觸發重新的加載。
那我們再次，變更，通過下面的減少path，查看效果（這個是在上面的基礎上的配置變更）
kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-new-ingress
namespace: default
spec:
ingressClassName: nginx
rules:
- host: nginx-new.k8s.com
http:
paths:
- backend:
service:
name: nginx-new
port:
number: 80
path: /
pathType: Prefix
EOFingress-nginx-controller的pod的日志
W0420 06:09:47.154332 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:09:47.154444 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:09:47.154466 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new": no object matching key "default/nginx-new" in local store
I0420 06:09:47.264935 8 admission.go:149] processed ingress via admission controller {testedIngressLength:2 testedIngressTime:0.111s renderingIngressLength:2 renderingIngressTime:0s admissionTime:29.6kBs testedConfigurationSize:0.111}
I0420 06:09:47.265017 8 main.go:101] "successfully validated configuration, accepting" ingress="default/nginx-new-ingress"
I0420 06:09:47.270238 8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-new-ingress", UID:"40040233-dd6e-4309-a62e-70ed34d0403e", APIVersion:"networking.k8s.io/v1", ResourceVersion:"11464712", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
W0420 06:09:47.270311 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:09:47.270362 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:09:47.270502 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new": no object matching key "default/nginx-new" in local store
I0420 06:09:47.270757 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:09:47.416746 8 controller.go:172] "Backend successfully reloaded"
I0420 06:09:47.417550 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration從日志發現，也是觸發了nginx的reload的操作。
3.3、刪除ingress對象
在這個步驟里面呢，我們刪除一下，剛剛實驗用的ingress對象，查看下是否會發生reload的操作
kubectl delete ingress nginx-new-ingress執行完上面的命令之后，我們查看ingress-nginx-controller的pod日志
W0420 06:12:16.618607 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:12:16.618696 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
I0420 06:12:16.618899 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:12:16.820738 8 controller.go:172] "Backend successfully reloaded"
I0420 06:12:16.821728 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration同樣的，監測到配置文件發生了變化，然后進行了reload的操作。
這點就和實際是相符合的。
3.4、service變得可用
剛我們上面的實驗，就是創建了一個ingress的對象，但是呢，后端並沒有一個對應service和pod，這次我們創建一個ingress，同時看后端的service有效了之后，會如何，在增加個有效的pod又如何
好，首先我們還是先創建個ingress對象
kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: nginx-new-ingress
namespace: default
spec:
ingressClassName: nginx
rules:
- host: nginx-new.k8s.com
http:
paths:
- backend:
service:
name: nginx-new
port:
number: 80
path: /
pathType: Prefix
EOFpod的日志
W0420 06:22:48.569666 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:22:48.569806 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:22:48.569846 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new": no object matching key "default/nginx-new" in local store
I0420 06:22:48.687580 8 admission.go:149] processed ingress via admission controller {testedIngressLength:2 testedIngressTime:0.118s renderingIngressLength:2 renderingIngressTime:0.001s admissionTime:29.6kBs testedConfigurationSize:0.119}
I0420 06:22:48.687654 8 main.go:101] "successfully validated configuration, accepting" ingress="default/nginx-new-ingress"
I0420 06:22:48.694129 8 store.go:424] "Found valid IngressClass" ingress="default/nginx-new-ingress" ingressclass="nginx"
I0420 06:22:48.694394 8 event.go:282] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"nginx-new-ingress", UID:"297dc73c-82d8-4085-a008-b46a1a9bc775", APIVersion:"networking.k8s.io/v1", ResourceVersion:"11465915", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
W0420 06:22:48.694695 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:22:48.694725 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:22:48.694741 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new": no object matching key "default/nginx-new" in local store
I0420 06:22:48.694853 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:22:48.820448 8 controller.go:172] "Backend successfully reloaded"
I0420 06:22:48.820978 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration可以看到是無法獲得對應的endpoints的日志
Error obtaining Endpoints for Service "default/nginx-newOK，那我們再創建一個對應的service看看
kubectl apply -f - <<EOF
apiVersion: v1
kind: Service
metadata:
labels:
app: nginx-new
name: nginx-new
spec:
ports:
- name: 80-80
port: 80
protocol: TCP
targetPort: 80
selector:
app: nginx-new
type: ClusterIP
EOF查看ingress-nginx-controller的pod日志
W0420 06:26:10.403081 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:26:10.403214 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:26:10.403338 8 controller.go:1083] Service "default/nginx-new" does not have any active Endpoint.
I0420 06:26:10.404248 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:26:10.587655 8 controller.go:172] "Backend successfully reloaded"
I0420 06:26:10.588417 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration發現，還是進行了reload的操作。也就是說，當service和ingress關聯上了之后，變得有效了之后，會重新進行nginx的reload的操作。
OK，接下去呢，我們創建一個對應的pod，看看效果
kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-new
spec:
selector:
matchLabels:
app: nginx-new
replicas: 2
template:
metadata:
labels:
app: nginx-new
spec:
containers:
- image: 172.20.58.152/middleware/nginx:1.21.4
imagePullPolicy: IfNotPresent
name: nginx-new
EOF[root@nccztsjb-node-23 ~]# kubectl get endpoints nginx-new
NAME ENDPOINTS AGE
nginx-new 172.39.157.203:80,172.39.21.75:80 4m2sservice是和pod進行關聯，但是ingress-nginx-pod沒有看到reload的操作。
3.5、刪除service
我們看下，刪除了service之后，是否會發生nginx的reload
kubectl delete svc nginx-newW0420 06:31:50.095619 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test": no object matching key "test01/nginx-test" in local store
W0420 06:31:50.095691 8 controller.go:988] Error obtaining Endpoints for Service "test01/nginx-test-02": no object matching key "test01/nginx-test-02" in local store
W0420 06:31:50.095736 8 controller.go:988] Error obtaining Endpoints for Service "default/nginx-new": no object matching key "default/nginx-new" in local store
I0420 06:31:50.096049 8 controller.go:155] "Configuration changes detected, backend reload required"
I0420 06:31:50.270016 8 controller.go:172] "Backend successfully reloaded"
I0420 06:31:50.270428 8 event.go:282] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-z6bhs", UID:"402f44fb-cd6b-41e3-a233-675da8fe394c", APIVersion:"v1", ResourceVersion:"258445", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration通過日志可以看到，當將ingress中使用的service對象刪除了之后，會發生nginx的reload的操作。
OK，今天我們的實驗，就先到這里，后面關於證書的，sercret在ingress中的使用等研究好了，再進行發布。