碼上歡樂
相關內容    簡體    繁體

Istio的流量管理(實操二)

本文轉載自   查看原文   2020-05-25 14:23   1558    istio

Istio的流量管理(實操二)

涵蓋官方文檔Traffic Management章節中的inrgess部分。

目錄

Ingress網關

在kubernetes環境中，kubernetes ingress資源用於指定暴露到集群外的服務。在istio服務網格中，使用了一種不同的配置模型，稱為istio網關。一個網關允許將istio的特性，如鏡像和路由規則應用到進入集群的流量上。

本節描述了如何使用istio網關將一個服務暴露到服務網格外。

環境准備

使用如下命令創建httpbin服務

$ kubectl apply -f samples/httpbin/httpbin.yaml

確定ingress的IP和端口

由於本環境中沒有配置對外的負載均衡，因此此處的EXTERNAL-IP為空，使用node port進行訪問

# kubectl get svc istio-ingressgateway -n istio-system
NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP  PORT(S)    AGE
istio-ingressgateway   LoadBalancer   10.84.93.45   <pending>     ...        11d

獲取ingressgateway service的http2和https對應的端口

$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].nodePort}')
$ export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].nodePort}')

下面是istio-system命名空間的istio-ingressgateway service中的一部分端口信息，可以看到http2和https的nodeport分別為31194和31785，對應上面的INGRESS_PORT和SECURE_INGRESS_PORT

{
    "name": "http2",
    "nodePort": 31194,
    "port": 80,
    "protocol": "TCP",
    "targetPort": 80
},
{
    "name": "https",
    "nodePort": 31785,
    "port": 443,
    "protocol": "TCP",
    "targetPort": 443
},

獲取istio-system命名空間中ingressgateway pod 的hostIP

$ export INGRESS_HOST=$(kubectl get po -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].status.hostIP}')

使用istio網關配置ingress

一個ingress網關描述了在網格邊緣用於接收入站HTTP/TCP連接的負載均衡，配置了暴露的端口，協議等。kubernetes ingress資源不包括任何流量路由配置。ingress 流量的路由使用istio路由規則，與內部服務請求相同：

  1. 創建istio Gateway，網關監聽地址為httpbin.example.com，端口為80(位於默認的ingressgatewaypod中)

    kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80 #gateway暴露80端口
      name: http
      protocol: HTTP
    hosts: #gateway暴露的主機名
    - "httpbin.example.com"
EOF

  2. 通過Gateway配置進入的流量路由，將URI為httpbin.example.com，且目的地為/status或/delay的請求分發到httpbin服務的8000端口，其他請求會返回404響應。

    kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts: #virtual service的hosts字段與Gateway的servers.hosts字段需要匹配
  - "httpbin.example.com" 
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

    可以看到流量被導入了httpbin service暴露的8000端口上

    $ oc get svc |grep httpbin
httpbin       ClusterIP      10.84.222.69   <none>    8000/TCP   19h

    來自網格內部其他服務的請求則不受此規則的約束，會使用默認的輪詢路由進行請求分發。為了限制內部的調用規則，可以將特定的值mesh添加到gateways列表中。由於內部服務主機名(如httpbin.default.svc.cluster.local) 可能與外部不同，因此需要將主機名添加到hosts列表中。

  3. 使用curl命令訪問httpbin服務，此時通過-H選項修改了HTTP請求首部的Host字段，使用http2的nodeport方式訪問：

    $ curl -s -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/status/200
HTTP/1.1 200 OK
server: istio-envoy
date: Thu, 21 May 2020 03:22:50 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 21

  4. 訪問其他URL路徑則返回404錯誤

    $ curl -s -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/headers
HTTP/1.1 404 Not Found
date: Thu, 21 May 2020 03:25:20 GMT
server: istio-envoy
transfer-encoding: chunked

使用瀏覽器訪問ingress服務

由於無法像使用curl一樣修改請求的Host首部字段，因此無法使用瀏覽器訪問httpbin服務。為了解決這個問題，可以在Gateway和VirtualService中的host字段使用通配符*。

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"  #指定通配符，監聽所有流量，不限制外部流量的地址
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "*"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /headers
    route:
    - destination:
        port:
          number: 8000
        host: httpbin
EOF

問題定位

  1. 檢查環境變量INGRESS_HOST和INGRESS_PORT，保證這兩個值是有效的

    $ kubectl get svc -n istio-system
$ echo INGRESS_HOST=$INGRESS_HOST, INGRESS_PORT=$INGRESS_PORT

  2. 校驗相同的端口上沒有其他istio ingress網格

    $ kubectl get gateway --all-namespaces

  3. 校驗沒有在相同的IP和端口上定義kubernetes ingress資源

    $ kubectl get ingress --all-namespaces

  4. 如果沒有負載均衡，可以參照上面步驟使用node port方式

卸載

$ kubectl delete gateway httpbin-gateway
$ kubectl delete virtualservice httpbin
$ kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

Ingress(kubernetes)

執行ingress流量控制中的Before you begin和Determining the ingress IP and ports小節的操作，部署httpbin服務。本節介紹如何通過kubernetes的Ingress(非istio的gateway)進行訪問。

下面展示如何配置一個80端口的Ingress，用於HTTP流量：

  1. 創建一個istio Gateway，將來自httpbin.example.com:80/status/*的流量分發到service httpbin的8000端口

    $ kubectl apply -f - <<EOF
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: istio
  name: ingress
spec:
  rules:
  - host: httpbin.example.com
    http:
      paths:
      - path: /status/*
        backend:
          serviceName: httpbin
          servicePort: 8000
EOF

    注意需要使用 kubernetes.io/ingress.class annotation來告訴istio網關控制器處理該ingress，否則會忽略該ingress。

  2. 使用curl命令訪問httpbin服務。Ingress的流量也需要經過istio ingressgateway。

    $ curl -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/status/200
HTTP/1.1 200 OK
server: istio-envoy
date: Fri, 22 May 2020 06:12:56 GMT
content-type: text/html; charset=utf-8
access-control-allow-origin: *
access-control-allow-credentials: true
content-length: 0
x-envoy-upstream-service-time: 20

    httpbin的服務發現是通過EDS實現的，使用如下命令查看：

    $ istioctl proxy-config cluster istio-ingressgateway-569669bb67-b6p5r|grep 8000
httpbin.default.svc.cluster.local                   8000    -    outbound      EDS
outbound_.8000_._.httpbin.default.svc.cluster.local   -     -       -          EDS

  3. 訪問其他未暴露的路徑，返回HTTP 404錯誤：

    $ curl -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/headers
HTTP/1.1 404 Not Found
date: Fri, 22 May 2020 06:24:30 GMT
server: istio-envoy
transfer-encoding: chunked

下一步

TLS

Ingress支持TLS設置。Istio也支持TLS設置，但相關的secret必須存在istio-ingressgateway deployment所在的命名空間中。可以使用cert-manager生成這些證書。

指定路徑類型

默認情況下，Istio會將路徑視為完全匹配，如果路徑使用/*或*結尾，則該路徑為前綴匹配。不支持其他正則匹配。

kubernetes 1.18中新增了一個字段pathType，允許聲明為Exact或Prefix。

指定IngressClass

kubernetes 1.18中新增了一個資源IngressClass，替換了Ingress資源的 kubernetes.io/ingress.class annotation。如果使用該資源，則需要將controller設置為istio.io/ingress-controller：

apiVersion: networking.k8s.io/v1beta1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress
spec:
  ingressClassName: istio
  ...

卸載

$ kubectl delete ingress ingress
$ kubectl delete --ignore-not-found=true -f samples/httpbin/httpbin.yaml

安全網關

本節講述如何使用simple或mutual TLS暴露安全的HTTPS服務。證書是通過SDS進行密鑰發現的。

TLS需要的私鑰，服務端證書，根證書是使用基於文件裝載的方法配置的。

執行ingress流量控制中的Before you begin和Determining the ingress IP and ports小節的操作，部署httpbin服務，並獲取 INGRESS_HOST 和SECURE_INGRESS_PORT變量。

生成服務端證書和私鑰

下面使用openssl生成需要的證書和密鑰

  1. 生成一個根證書和一個私鑰，用於簽名服務的證書

    $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt

  2. 為 httpbin.example.com生成一個證書和私鑰

    $ openssl req -out httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization"
$ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in httpbin.example.com.csr -out httpbin.example.com.crt

單主機配置TLS ingress網關

  1. 為ingree網關創建一個secret

    secret的名字不能以istio或prometheus開頭，且secret不應該包含token字段

    $ kubectl create -n istio-system secret tls httpbin-credential --key=httpbin.example.com.key --cert=httpbin.example.com.crt

  2. 在server部分定義一個443端口的Gateway，將credentialName指定為httpbin-credential，與secret的名字相同，TLS的mode為SIMPLE。

    $ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls: #對暴露的服務使用SIMPLE模式的tls，即單向tls驗證
      mode: SIMPLE
      credentialName: httpbin-credential # must be the same as secret
    hosts:
    - httpbin.example.com
EOF

  3. 配置進入Gateway的流量路由。與上一節中的VirtualService相同

    $ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin
spec:
  hosts:
  - "httpbin.example.com"
  gateways:
  - httpbin-gateway
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:
        port:
          number: 8000  #可以看到tls只是這是在gateway上的，當進入網格之后就不需要了
        host: httpbin
EOF

  4. 使用curl向SECURE_INGRESS_PORT發送HTTPS請求訪問httpbin服務，請求中攜帶了公鑰example.com.crt 。--resolve標記可以在使用curl訪問TLS的網關IP時，在SNI中支持httpbin.example.com。--cacert選擇支持使用生成的證書校驗服務。

    -HHost:httpbin.example.com 選項僅在SECURE_INGRESS_PORT不同於實際的網關端口(443)時才會需要，例如，通過映射的NodePort方式訪問服務時。

    通過將請求發送到/status/418 URL路徑時，可以看到httpbin確實被訪問了，httpbin服務會返回418 I’m a Teapot代碼。.

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
--cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"

> --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
* Added httpbin.example.com:31967:172.20.127.211 to DNS cache
* About to connect() to httpbin.example.com port 31967 (#0)
*   Trying 172.20.127.211...
* Connected to httpbin.example.com (172.20.127.211) port 31967 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
*       subject: O=httpbin organization,CN=httpbin.example.com
*       start date: May 22 09:03:01 2020 GMT
*       expire date: May 22 09:03:01 2021 GMT
*       common name: httpbin.example.com
*       issuer: CN=example.com,O=example Inc.
> GET /status/418 HTTP/1.1
> User-Agent: curl/7.29.0
> Accept: */*
> Host:httpbin.example.com
>
< HTTP/1.1 418 Unknown
< server: istio-envoy
< date: Fri, 22 May 2020 09:08:29 GMT
< x-more-info: http://tools.ietf.org/html/rfc2324
< access-control-allow-origin: *
< access-control-allow-credentials: true
< content-length: 135
< x-envoy-upstream-service-time: 2
<

    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`
* Connection #0 to host httpbin.example.com left intact

    查看curl輸出中的Server certificate中的信息，上述返回值的最后有一個茶壺的圖片，說明運行成功。

  5. 刪除老的網關secret，創建一個新的secret，並使用該secret修改ingress網關的憑據

    $ kubectl -n istio-system delete secret httpbin-credential

    $ mkdir new_certificates
$ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout new_certificates/example.com.key -out new_certificates/example.com.crt
$ openssl req -out new_certificates/httpbin.example.com.csr -newkey rsa:2048 -nodes -keyout new_certificates/httpbin.example.com.key -subj "/CN=httpbin.example.com/O=httpbin organization"
$ openssl x509 -req -days 365 -CA new_certificates/example.com.crt -CAkey new_certificates/example.com.key -set_serial 0 -in new_certificates/httpbin.example.com.csr -out new_certificates/httpbin.example.com.crt
$ kubectl create -n istio-system secret tls httpbin-credential \
--key=new_certificates/httpbin.example.com.key \
--cert=new_certificates/httpbin.example.com.crt

  6. 使用新的證書鏈訪問httpbin服務

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
--cacert new_certificates/example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"

  7. 如果使用老的證書訪問，則返回錯誤

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
> --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
* Added httpbin.example.com:31967:172.20.127.211 to DNS cache
* About to connect() to httpbin.example.com port 31967 (#0)
*   Trying 172.20.127.211...
* Connected to httpbin.example.com (172.20.127.211) port 31967 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* Server certificate:
*       subject: O=httpbin organization,CN=httpbin.example.com
*       start date: May 22 09:24:07 2020 GMT
*       expire date: May 22 09:24:07 2021 GMT
*       common name: httpbin.example.com
*       issuer: CN=example.com,O=example Inc.
* NSS error -8182 (SEC_ERROR_BAD_SIGNATURE)
* Peer's certificate has an invalid signature.
* Closing connection 0
curl: (60) Peer's certificate has an invalid signature.

多主機配置TLS ingress網關

本節會為多個主機(httpbin.example.com和helloworld-v1.example.com)配置一個ingress網關。ingress網關會在credentialName中查找唯一的憑據。

  1. 刪除之前創建的secret並為httpbin重建憑據

    $ kubectl -n istio-system delete secret httpbin-credential
$ kubectl create -n istio-system secret tls httpbin-credential \
--key=httpbin.example.com.key \
--cert=httpbin.example.com.crt

  2. 啟動helloworld-v1

    $ cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: helloworld-v1
  labels:
    app: helloworld-v1
spec:
  ports:
  - name: http
    port: 5000
  selector:
    app: helloworld-v1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld-v1
      version: v1
  template:
    metadata:
      labels:
        app: helloworld-v1
        version: v1
    spec:
      containers:
      - name: helloworld
        image: istio/examples-helloworld-v1
        resources:
          requests:
            cpu: "100m"
        imagePullPolicy: IfNotPresent #Always
        ports:
        - containerPort: 5000
EOF

  3. 為helloworld-v1.example.com創建證書和私鑰

    $ openssl req -out helloworld-v1.example.com.csr -newkey rsa:2048 -nodes -keyout helloworld-v1.example.com.key -subj "/CN=helloworld-v1.example.com/O=helloworld organization"
$ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 1 -in helloworld-v1.example.com.csr -out helloworld-v1.example.com.crt

  4. 為helloworld-credential創建secret

    $ kubectl create -n istio-system secret tls helloworld-credential --key=helloworld-v1.example.com.key --cert=helloworld-v1.example.com.crt

  5. 定義一個網關，網關端口為443。在credentialName字段分別設置httpbin-credential和helloworld-credential，TLS模式為SIMPLE。

    $ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https-httpbin #httpbin的gateway配置
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: httpbin-credential
    hosts:
    - httpbin.example.com
  - port:
      number: 443
      name: https-helloworld #https-helloword的gateway配置
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: helloworld-credential
    hosts:
    - helloworld-v1.example.com
EOF

  6. 配置gateway的流量路由，為新應用添加對應的virtual service

    $ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: helloworld-v1
spec:
  hosts:
  - helloworld-v1.example.com
  gateways:
  - mygateway
  http:
  - match:
    - uri:
        exact: /hello
    route:
    - destination:
        host: helloworld-v1
        port:
          number: 5000
EOF

  7. 向helloworld-v1.example.com發送HTTPS請求

    $ curl -v -HHost:helloworld-v1.example.com --resolve "helloworld-v1.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
--cacert example.com.crt "https://helloworld-v1.example.com:$SECURE_INGRESS_PORT/hello"

...
< HTTP/1.1 200 OK
< content-type: text/html; charset=utf-8
< content-length: 60
< server: istio-envoy
< date: Sat, 23 May 2020 07:38:40 GMT
< x-envoy-upstream-service-time: 143
<
Hello version: v1, instance: helloworld-v1-5dfcf5d5cd-2l44c
* Connection #0 to host helloworld-v1.example.com left intact

  8. 向 httpbin.example.com發送請求

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
--cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"

 ...
    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`
* Connection #0 to host httpbin.example.com left intact

配置一個mutual TLS ingress網關

刪除之前的secreting創建一個新的secret，server會使用該CA證書校驗client，使用cacert保存CA證書。

$ kubectl -n istio-system delete secret httpbin-credential
$ kubectl create -n istio-system secret generic httpbin-credential --from-file=tls.key=httpbin.example.com.key \
--from-file=tls.crt=httpbin.example.com.crt --from-file=ca.crt=example.com.crt

  1. 將gateway的TLS模式設置為MUTUAL

    $ cat <<EOF | kubectl apply -f -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
 name: mygateway
spec:
 selector:
   istio: ingressgateway # use istio default ingress gateway
 servers:
 - port:
     number: 443
     name: https
     protocol: HTTPS
   tls:
     mode: MUTUAL #對網關暴露的服務httpbin.example.com啟用雙向認證
     credentialName: httpbin-credential # must be the same as secret
   hosts:
   - httpbin.example.com
EOF

  2. 使用先前的方式發送HTTPS請求，可以看到訪問失敗

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" \
> --cacert example.com.crt "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
* Added httpbin.example.com:31967:172.20.127.211 to DNS cache
* About to connect() to httpbin.example.com port 31967 (#0)
*   Trying 172.20.127.211...
* Connected to httpbin.example.com (172.20.127.211) port 31967 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: example.com.crt
  CApath: none
* NSS: client certificate not found (nickname not specified)
* NSS error -12227 (SSL_ERROR_HANDSHAKE_FAILURE_ALERT)
* SSL peer was unable to negotiate an acceptable set of security parameters.
* Closing connection 0
curl: (35) NSS: client certificate not found (nickname not specified)

  3. 使用公鑰example.com.crt生成client的證書和私鑰。在curl中傳入客戶端的證書和私鑰，使用--cert傳入客戶端證書，使用--key傳入私鑰

    $ openssl req -out client.example.com.csr -newkey rsa:2048 -nodes -keyout client.example.com.key -subj "/CN=client.example.com/O=client organization"
$ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 1 -in client.example.com.csr -out client.example.com.crt

    $ curl -v -HHost:httpbin.example.com --resolve "httpbin.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST" --cacert example.com.crt --cert ./client.example.com.crt --key ./client.example.com.key "https://httpbin.example.com:$SECURE_INGRESS_PORT/status/418"
...
    -=[ teapot ]=-

       _...._
     .'  _ _ `.
    | ."` ^ `". _,
    \_;`"---"`|//
      |       ;/
      \_     _/
        `"""`
* Connection #0 to host httpbin.example.com left intact

istio支持幾種不同的Secret格式，來支持與多種工具的集成，如cert-manager:

  • 一個TLS Secret使用tls.key和tls.crt；對於mutual TLS，會用到ca.crt
  • 一個generic Secret會用到key和cert；對於mutual TLS，會用到cacert
  • 一個generic Secret會用到key和cert；對於mutual TLS，會用到一個單獨的名為 <secret>-cacert的generic Secret，以及一個cacert key。例如httpbin-credential 包含key 和cert，httpbin-credential-cacert 包含cacert

問題定位

  • 檢查INGRESS_HOST和SECURE_INGRESS_PORT環境變量

    $ kubectl get svc -n istio-system
$ echo INGRESS_HOST=$INGRESS_HOST, SECURE_INGRESS_PORT=$SECURE_INGRESS_PORT

  • 檢查istio-ingressgateway控制器的錯誤日志

    $ kubectl logs -n istio-system "$(kubectl get pod -l istio=ingressgateway \
-n istio-system -o jsonpath='{.items[0].metadata.name}')"

  • 校驗istio-system命名空間中成功創建了secret。上例中應該存在httpbin-credential和helloworld-credential

    $ kubectl -n istio-system get secrets

  • 校驗ingress網關agent將密鑰/證書對上傳到了ingress網關

    $ kubectl logs -n istio-system "$(kubectl get pod -l istio=ingressgateway \
-n istio-system -o jsonpath='{.items[0].metadata.name}')"

定位mutul TLS問題

  • 校驗CA加載到了 istio-ingressgateway pod中，查看是否存在example.com.crt

    $ kubectl exec -it -n istio-system $(kubectl -n istio-system get pods -l istio=ingressgateway -o jsonpath='{.items[0].metadata.name}') -- ls -al /etc/istio/ingressgateway-ca-certs

  • 如果創建了istio-ingressgateway-ca-certs secret，但沒有加載CA證書，刪除ingress網關pod，強制加載該證書

    $ kubectl delete pod -n istio-system -l istio=ingressgateway

  • 校驗CA證書的Subject字段是否正確

    $ kubectl exec -i -n istio-system $(kubectl get pod -l istio=ingressgateway -n istio-system -o jsonpath='{.items[0].metadata.name}')  -- cat /etc/istio/ingressgateway-ca-certs/example.com.crt | openssl x509 -text -noout | grep 'Subject:'
        Subject: O=example Inc., CN=example.com

    log中可以看到添加了httpbin-credential secret。如果使用mutual TLS，那么也會出現 httpbin-credential-cacert secret。校驗log中顯示了網關agent從ingress網關接收到了SDS請求，資源的名稱為 httpbin-credential，且ingress網關獲取到了密鑰/證書對。如果使用了mutual TLS，日志應該顯示將密鑰/證書發送到ingress網關，網關agent接收到了帶 httpbin-credential-cacert 資源名稱的SDS請求，並回去到了根證書。

卸載

  1. 刪除Gateway配置，VirtualService和secret

    $ kubectl delete gateway mygateway
$ kubectl delete virtualservice httpbin
$ kubectl delete --ignore-not-found=true -n istio-system secret httpbin-credential \
helloworld-credential
$ kubectl delete --ignore-not-found=true virtualservice helloworld-v1

  2. 刪除證書目錄

    $ rm -rf example.com.crt example.com.key httpbin.example.com.crt httpbin.example.com.key httpbin.example.com.csr helloworld-v1.example.com.crt helloworld-v1.example.com.key helloworld-v1.example.com.csr client.example.com.crt client.example.com.csr client.example.com.key ./new_certificates

  3. 停止httpbin和helloworld-v1 服務：

    $ kubectl delete deployment --ignore-not-found=true httpbin helloworld-v1
$ kubectl delete service --ignore-not-found=true httpbin helloworld-v1

不終止TLS的ingress網關

上一節中描述了如何配置HTTPS ingree來訪問一個HTTP服務。本節中描述如何配置HTTPS ingrss來訪問HTTPS服務等。通過配置ingress網關來執行SNI方式的訪問，而不會在請求進入ingress時終止TLS。

本例中使用一個NGINX服務器作為HTTPS服務。

生成客戶端和服務端的證書和密鑰

  1. 生成一個根證書和私鑰，用於簽名服務

    $ openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -subj '/O=example Inc./CN=example.com' -keyout example.com.key -out example.com.crt

  2. 為 nginx.example.com創建證書和私鑰

    $ openssl req -out nginx.example.com.csr -newkey rsa:2048 -nodes -keyout nginx.example.com.key -subj "/CN=nginx.example.com/O=some organization"
$ openssl x509 -req -days 365 -CA example.com.crt -CAkey example.com.key -set_serial 0 -in nginx.example.com.csr -out nginx.example.com.crt

部署NGINX服務

  1. 創建kubernetes Secret保存服務的證書

    $ kubectl create secret tls nginx-server-certs --key nginx.example.com.key --cert nginx.example.com.crt

  2. 為NGINX服務創建配置文件

    $ cat <<EOF > ./nginx.conf
events {
}

http {
  log_format main '$remote_addr - $remote_user [$time_local]  $status '
  '"$request" $body_bytes_sent "$http_referer" '
  '"$http_user_agent" "$http_x_forwarded_for"';
  access_log /var/log/nginx/access.log main;
  error_log  /var/log/nginx/error.log;

  server {
    listen 443 ssl;

    root /usr/share/nginx/html;
    index index.html;

    server_name nginx.example.com;
    ssl_certificate /etc/nginx-server-certs/tls.crt;
    ssl_certificate_key /etc/nginx-server-certs/tls.key;
  }
}
EOF

  3. 為NGINX服務創建kubernetes configmap

    $ kubectl create configmap nginx-configmap --from-file=nginx.conf=./nginx.conf

  4. 部署NGINX服務

    $ cat <<EOF | istioctl kube-inject -f - | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    run: my-nginx
spec:
  ports:
  - port: 443
    protocol: TCP
  selector:
    run: my-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 1
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx
        ports:
        - containerPort: 443
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx
          readOnly: true
        - name: nginx-server-certs
          mountPath: /etc/nginx-server-certs
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: nginx-configmap
      - name: nginx-server-certs
        secret:
          secretName: nginx-server-certs #保存了NGINX服務的證書和私鑰
EOF

  5. 為了測試NGINX服務部署成功，向服務發送不使用證書的方式請求，並校驗打印信息是否正確：

    $ kubectl exec -it $(kubectl get pod  -l run=my-nginx -o jsonpath={.items..metadata.name}) -c istio-proxy -- curl -v -k --resolve 
...
* Server certificate:
*  subject: CN=nginx.example.com; O=some organization
*  start date: May 25 02:09:02 2020 GMT
*  expire date: May 25 02:09:02 2021 GMT
*  issuer: O=example Inc.; CN=example.com
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
...

配置一個ingress gateway

  1. 定義一個gateway，端口為443.注意TLS的模式為PASSTHROUGH，表示gateway會放行ingress流量，不終止TLS

    $ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: mygateway
spec:
  selector:
    istio: ingressgateway # use istio default ingress gateway
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: PASSTHROUGH #不終止TLS
    hosts:
    - nginx.example.com
EOF

  2. 配置經過Gateway的流量路由

    $ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
spec:
  hosts:
  - nginx.example.com
  gateways:
  - mygateway
  tls:
  - match:
    - port: 443 #將gateway的流量導入kubernetes的my-nginx service
      sniHosts:
      - nginx.example.com
    route:
    - destination:
        host: my-nginx
        port:
          number: 443
EOF

  3. 根據指導配置SECURE_INGRESS_PORT和INGRESS_HOST環境變量

  4. 通過ingress訪問nginx，可以看到訪問成功

    $ curl -v --resolve nginx.example.com:$SECURE_INGRESS_PORT:$INGRESS_HOST --cacert example.com.crt https://nginx.example.com:$SECURE_INGRESS_PORT
...
* SSL connection using TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
* Server certificate:
*       subject: O=some organization,CN=nginx.example.com
*       start date: May 25 02:09:02 2020 GMT
*       expire date: May 25 02:09:02 2021 GMT
*       common name: nginx.example.com
*       issuer: CN=example.com,O=example Inc.
...
<title>Welcome to nginx!</title>
...

卸載

  1. 移除kubernetes資源

    $ kubectl delete secret nginx-server-certs
$ kubectl delete configmap nginx-configmap
$ kubectl delete service my-nginx
$ kubectl delete deployment my-nginx
$ kubectl delete gateway mygateway
$ kubectl delete virtualservice nginx

  2. 刪除證書和密鑰

    $ rm example.com.crt example.com.key nginx.example.com.crt nginx.example.com.key nginx.example.com.csr

  3. 刪除生成的配置文件

    $ rm ./nginx.conf

TIPS

Gateway支持的TLS模式如下：

Name Description
PASSTHROUGH 客戶端提供的SNI字符串將用作VirtualService TLS路由中的匹配條件，以根據服務注冊表確定目標服務
SIMPLE 使用標准TLS語義的安全連接
MUTUAL 通過提供服務器證書進行身份驗證，使用雙邊TLS來保護與下游的連接
AUTO_PASSTHROUGH 與直通模式相似，不同之處在於具有此TLS模式的服務器不需要關聯的VirtualService即可從SNI值映射到注冊表中的服務。目標詳細信息（例如服務/子集/端口）被編碼在SNI值中。代理將轉發到SNI值指定的上游（Envoy）群集（一組端點）。
ISTIO_MUTUAL 通過提供用於身份驗證的服務器證書，使用相互TLS使用來自下游的安全連接


免責聲明！

本站轉載的文章為個人學習借鑒使用，本站對版權不負任何法律責任。如果侵犯了您的隱私權益，請聯系本站郵箱yoyou2525@163.com刪除。



猜您在找 Istio的流量管理(實操一) Istio的流量管理(實操三) istio流量管理 Istio的流量管理(概念) Istio流量管理能力介紹 istio流量管理yaml示例 kubernetes + istio進行流量管理 istio流量管理：非侵入式流量治理 Istio安全-證書管理(實操一) Istio流量管理實現機制深度解析
 
粵ICP備18138465號   © 2018-2025 CODEPRJ.COM