Kubernetes部署nginx-ingress

本文转载自查看原文 2021-11-02 14:43 2026 kubernetes

一、Ingress介绍

ingress可以让互联网客户访问kubernetes集群，而ClientIP和coredns只能在集群内部访问，Ingress的架构图如下：

要使用 Ingress的步骤

先部署 Ingress Controller 实体（相当于前端 Nginx）
然后再创建 Ingress （相当于 Nginx 配置的 k8s 资源体现）
Ingress Controller 部署好后会动态检测 Ingress 的创建情况生成相应配置

Ingress Controller 的实现有很多种：

基于 Nginx 的，基于 Nginx 的 Ingress Controller 有两种，一种是 k8s 社区提供的 ingress-nginx，另一种是 Nginx 社区提供的nginx-ingress（常见，亦本文的方式），参见他们的区别
基于 HAProxy的，
基于 OpenResty 的 Kong Ingress Controller 等
更多 Controller 见：https://kubernetes.io/docs/concepts/services-networking/ingress-controllers

基于Nginx Ingress的拓扑图如下

二、安装nginx-ingress

先查找仓库

$ helm search repo nginx-ingress
NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
aliyuncs/nginx-ingress                  1.30.3          0.28.0          An nginx Ingress controller that uses ConfigMap...
aliyuncs/nginx-ingress-controller       5.3.4           0.29.0          Chart for the nginx Ingress controller
bitnami/nginx-ingress-controller 9.0.2 1.0.4 Chart for the nginx Ingress controller
aliyuncs/nginx-lego                     0.3.1                           Chart for nginx-ingress-controller and kube-lego

安装版本最新的第三个，要仔细看安装以后给的提示

$ helm install nginx-ingress bitnami/nginx-ingress-controller
NAME: nginx-ingress
LAST DEPLOYED: Fri Nov  5 15:33:30 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: nginx-ingress-controller
CHART VERSION: 9.0.3
APP VERSION: 1.0.4

** Please be patient while the chart is being deployed **

The nginx-ingress controller has been installed.

Get the application URL by running these commands:

 NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        You can watch its status by running 'kubectl get --namespace default svc -w nginx-ingress-nginx-ingress-controller'

    export SERVICE_IP=$(kubectl get svc --namespace default nginx-ingress-nginx-ingress-controller -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    echo "Visit http://${SERVICE_IP} to access your application via HTTP."
    echo "Visit https://${SERVICE_IP} to access your application via HTTPS."

An example Ingress that makes use of the controller:

  apiVersion: networking.k8s.io/v1
  kind: Ingress
  metadata:
    annotations:
      kubernetes.io/ingress.class:
    name: example
    namespace: default
  spec:
    rules:
      - host: www.example.com
        http:
          paths:
            - backend:
                service:
                  name: example-service
                  port:
                    number: 80
              path: /
              pathType: Prefix
    # This section is only required if TLS is to be enabled for the Ingress
    tls:
        - hosts:
            - www.example.com
          secretName: example-tls

If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:

  apiVersion: v1
  kind: Secret
  metadata:
    name: example-tls
    namespace: default
  data:
    tls.crt: <base64 encoded cert>
    tls.key: <base64 encoded key>
  type: kubernetes.io/tls

查看安装的release

$ helm list
NAME            NAMESPACE       REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
nginx-ingress   default         1               2021-11-02 08:22:43.135546577 +0000 UTC deployed        nginx-ingress-1.30.3    0.28.0

查看pods

$ kubectl get pods -o wide
NAME                                             READY   STATUS    RESTARTS      AGE     IP               NODE   NOMINATED NODE   READINESS GATES
gostarter-dep-df898587f-9gfmh                    1/1     Running   0             3h20m   192.168.205.50   kbsm   <none>           <none>
gostarter-dep-df898587f-p76kn                    1/1     Running   0             3h20m   192.168.205.39   kbsm   <none>           <none>
gostarter-dep-df898587f-scn8g                    1/1     Running   0             3h20m   192.168.205.37   kbsm   <none>           <none>
gostarter-dep-df898587f-wq2cz                    1/1     Running   0             3h20m   192.168.184.36   kbs2   <none>           <none>
gostarter-dep-df898587f-ztsmc                    1/1     Running   0             3h20m   192.168.205.62   kbsm   <none>           <none>
nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     Running   2 (30s ago)   2m13s   192.168.151.2    kbs1   <none>           <none>
nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running   0             2m13s   192.168.151.57   kbs1   <none>           <none>

查看Service

$ kubectl get svc
NAME                            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
gostarter-svc                   ClusterIP      10.109.68.204    <none>        8000/TCP                     5h44m
kubernetes                      ClusterIP      10.96.0.1        <none>        443/TCP                      21d
nginx-ingress-controller        LoadBalancer   10.101.177.250   <pending>     80:31534/TCP,443:32619/TCP   15m
nginx-ingress-default-backend ClusterIP 10.108.181.102 <none> 80/TCP 15m

访问一下backend

$ curl http://10.108.181.102
default backend - 404

三、nginx-ingress错误

我安装过程中出现过这种问题

$ kubectl get pods
NAME                                             READY   STATUS             RESTARTS      AGE
nginx-ingress-controller-6f4cf4656d-m7wvn        0/1     CrashLoopBackOff  9 (43s ago)   18m
nginx-ingress-default-backend-78669dcf66-md9bp   1/1     Running            0             18m

查看这个pod的日志，有大量这种错误：Failed to list *v1beta1.Ingress: the server could not find the requested resource

$ kubectl logs nginx-ingress-controller-6f4cf4656d-m7wvn
I1102 08:39:16.674917       8 flags.go:205] Watching for Ingress class: nginx ------------------------------------------------------------------------------- NGINX Ingress controller Release: 0.28.0 Build: git-1f93cb8f3 W1102 08:39:16.675175 8 flags.go:250] SSL certificate chain completion is disabled (--enable-ssl-chain-completion=false) Repository: https://github.com/kubernetes/ingress-nginx nginx version: nginx/1.17.7 ------------------------------------------------------------------------------- W1102 08:39:16.675227 8 client_config.go:543] Neither --kubeconfig nor --master was specified. Using the inClusterConfig. This might not work. I1102 08:39:16.675433 8 main.go:193] Creating API client for https://10.96.0.1:443 I1102 08:39:16.842896 8 main.go:237] Running in Kubernetes cluster version v1.22 (v1.22.2) - git (clean) commit 8b5a19147530eaac9476b0ab82980b4088bbc1b2 - platform linux/amd64 I1102 08:39:16.858079 8 main.go:91] Validated default/nginx-ingress-default-backend as the default backend. I1102 08:39:17.000127 8 main.go:102] SSL fake certificate created /etc/ingress-controller/ssl/default-fake-certificate.pem W1102 08:39:17.047593 8 store.go:636] Unexpected error reading configuration configmap: configmaps "nginx-ingress-controller" not found I1102 08:39:17.054741 8 nginx.go:263] Starting NGINX Ingress controller E1102 08:39:18.379226 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:19.449966 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:20.753084 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:22.109971 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:23.136053 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:24.458595 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource E1102 08:39:25.492555 8 reflector.go:153] k8s.io/ingress-nginx/internal/ingress/controller/store/store.go:181: Failed to list *v1beta1.Ingress: the server could not find the requested resource I1102 08:39:53.382051 8 main.go:152] Received SIGTERM, shutting down I1102 08:39:53.382078 8 nginx.go:391] Shutting down controller queues I1102 08:39:53.382091 8 status.go:117] updating status of Ingress rules (remove) E1102 08:39:53.382177 8 store.go:185] timed out waiting for caches to sync I1102 08:39:53.382249 8 nginx.go:307] Starting NGINX process I1102 08:39:53.382435 8 leaderelection.go:242] attempting to acquire leader lease default/ingress-controller-leader-nginx... E1102 08:39:53.382666 8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:initial-sync,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},} I1102 08:39:53.398779 8 leaderelection.go:252] successfully acquired lease default/ingress-controller-leader-nginx I1102 08:39:53.398841 8 status.go:86] new leader elected: nginx-ingress-controller-6f4cf4656d-m7wvn E1102 08:39:53.398884 8 queue.go:78] queue has been shutdown, failed to enqueue: &ObjectMeta{Name:sync status,GenerateName:,Namespace:,SelfLink:,UID:,ResourceVersion:,Generation:0,CreationTimestamp:0001-01-01 00:00:00 +0000 UTC,DeletionTimestamp:<nil>,DeletionGracePeriodSeconds:nil,Labels:map[string]string{},Annotations:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},} I1102 08:39:53.422921 8 status.go:136] removing address from ingress status ([192.168.0.106]) I1102 08:39:53.423027 8 nginx.go:407] Stopping NGINX process 2021/11/02 08:39:53 [notice] 36#36: signal process started I1102 08:39:56.444623 8 nginx.go:420] NGINX process has stopped I1102 08:39:56.444662 8 main.go:160] Handled quit, awaiting Pod deletion I1102 08:40:06.445065 8 main.go:163] Exiting with 0

原因：我的kubernetes:v1.22 不再支持v1beta1，所以与低版本的aliyuncs/nginx-ingress:0.28.0不匹配，要改成使用高版本的bitnami/nginx-ingress-controller:1.0.4

下面再列一下helm的nginx-ingress的chart的源的查询

$ helm search repo nginx-ingress
NAME                                    CHART VERSION   APP VERSION     DESCRIPTION
aliyuncs/nginx-ingress 1.30.3 0.28.0 An nginx Ingress controller that uses ConfigMap... aliyuncs/nginx-ingress-controller 5.3.4 0.29.0 Chart for the nginx Ingress controller bitnami/nginx-ingress-controller 9.0.2 1.0.4 Chart for the nginx Ingress controller aliyuncs/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego

五、安装Nginx

helm中查看chart

$ helm search repo nginx
NAME                                    CHART VERSION   APP VERSION             DESCRIPTION
aliyuncs/nginx                          5.1.5           1.16.1                  Chart for the nginx server aliyuncs/nginx-ingress 1.30.3 0.28.0 An nginx Ingress controller that uses ConfigMap... aliyuncs/nginx-ingress-controller 5.3.4 0.29.0 Chart for the nginx Ingress controller aliyuncs/nginx-lego 0.3.1 Chart for nginx-ingress-controller and kube-lego aliyuncs/nginx-php 1.0.0 nginx-1.10.3_php-7.0 Chart for the nginx php server bitnami/nginx 9.5.12 1.21.3 Chart for the nginx server bitnami/nginx-ingress-controller 9.0.2 1.0.4 Chart for the nginx Ingress controller bitnami/kong 4.1.7 2.6.0 Kong is a scalable, open source API layer (aka ...

执行安装

helm install nginx bitnami/nginx
NAME: nginx
LAST DEPLOYED: Fri Nov  5 16:01:35 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: nginx
CHART VERSION: 9.5.13
APP VERSION: 1.21.4

** Please be patient while the chart is being deployed **

NGINX can be accessed through the following DNS name from within your cluster:

    nginx.default.svc.cluster.local (port 80)

To access NGINX from outside the cluster, follow the steps below:

1. Get the NGINX URL by running these commands:

  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        Watch the status with: 'kubectl get svc --namespace default -w nginx'

    export SERVICE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].port}" services nginx)
    export SERVICE_IP=$(kubectl get svc --namespace default nginx -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
    echo "http://${SERVICE_IP}:${SERVICE_PORT}"
root@kbsm:~/k8s/nginx-ingress#

查看pod

$ kubectl get pods
NAME                                                              READY   STATUS    RESTARTS        AGE
nginx-588469f6d6-rd5gx 1/1 Running 0 82s
nginx-ingress-nginx-ingress-controller-7bdbcc7787-pgk82           1/1     Running   0               29m
nginx-ingress-nginx-ingress-controller-default-backend-57ftmtdx   1/1     Running   0               29m

六、Ingress配置

先配置nginx自身的ingress：vi nginx-ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-nginx
  annotations:
    # use the shared ingress-nginx
    kubernetes.io/ingress.class: "nginx"
spec:
  rules:
  - host: kbsm
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port: 
              number: 80

ingress的配置新版本的格式与老版本格式不一致，可以查看官方文档

对于安装的kubernetes集群支持的apiVersion版本，可以通过这个命令查看：kubectl api-versions

然后发布ingress

$ kubectl apply -f nginx-ingress.yaml
ingress.networking.k8s.io/ingress-nginx created

七、Ingress测试

八、删除本文配置

如果nginx-ingress配置失败，删除本文所有的配置

kubectl delete -f nginx-ingress.yaml
helm uninstall nginx
helm uninstall nginx-ingress

参考资料：

使用 Kubernetes Ingress 对外暴露服务

免责声明！

本站转载的文章为个人学习借鉴使用，本站对版权不负任何法律责任。如果侵犯了您的隐私权益，请联系本站邮箱yoyou2525@163.com删除。

猜您在找 kubernetes nginx ingress controller部署 nginx-ingress配置https Kubernetes 使用Nginx-Ingress实现蓝绿发布/金丝雀发布/AB测试 Kubernetes 使用Nginx-Ingress实现蓝绿发布/金丝雀发布/AB测试 kubernetes 1.21 部署 ingress-nginx nginx-ingress之server-snippet用法 Kubernetes部署SpringCloud(三) 使用 Ingress-nginx 暴露服务 [k8s]nginx-ingress配置4/7层测试 Kubernetes Ingress-nginx使用 kubernetes安装ingress-nginx