k8s平台集成kong ingress 布署konga集成ui
各ingress的對比，其他博客有詳細對比
簡單談點體驗
-
istio很強大，但過於復雜，個人用過一段時期，勉強算是熟悉了基本使用，但復雜是相對團隊的，每個人都掌握不可能，團隊大部分只是想簡單的上線一個api,然后能快速cicd部署上線到k8s 公開訪問罷了，istio的流程做的全透明化需要額外的工作，個人來不及搞，團隊后續為了圖方便，都不喜歡用，算是半廢棄狀態，目前服務治理的方案是consul
-
ambassador 也是很強大的方案，未深入使用，只用過一些rewrite規則，方案比較強大
-
nginx/traefik 這兩個用過，只是很簡單的使用，未使用過復雜功能
-
kong 本身相比其他ingress並沒有太大的優點，支持一些常用的插件,ssl托管，賬號認證token,ip白名單等，不過這些也不只是kong有，真正決定使用kong，主要是因為konga的存在
官方向導見
https://docs.konghq.com/kubernetes-ingress-controller/1.1.x/deployment/minikube/
個人的環境並不是minikube 而是線上的生產集群，只不過喜歡，先以minikube類的精簡的方案為基准，再逐組件替換為線上的ha方案
相比deploy，個人目前傾向deploy,改為sts
-
改動 變更postgres類型
生產環境，建議使用postgres HA方案https://github.com/sorintlab/stolon 官方deploy為postgres,個人會替換為StatefulSet
apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres namespace: kong spec: replicas: 1 selector: matchLabels: app: postgres serviceName: postgres template: metadata: labels: app: postgres spec: containers: - env: - name: POSTGRES_USER value: kong - name: POSTGRES_PASSWORD value: kong - name: POSTGRES_DB value: kong - name: PGDATA value: /var/lib/postgresql/data/pgdata image: postgres:9.5 name: postgres ports: - containerPort: 5432 volumeMounts: - mountPath: /var/lib/postgresql/data name: datadir subPath: pgdata terminationGracePeriodSeconds: 60 volumeClaimTemplates: - metadata: name: datadir spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi -
官方只有kong ingress是不夠的，需要布署konga,之所以用kong就是因為konga的存在
apiVersion: apps/v1 kind: Deployment metadata: labels: app: konga name: konga namespace: kong spec: replicas: 1 selector: matchLabels: app: konga template: metadata: labels: app: konga spec: containers: - env: - name: HOST value: 0.0.0.0 - name: PORT value: '80' - name: NODE_ENV value: production - name: DB_ADAPTER value: postgres - name: DB_HOST value: kong - name: DB_PORT value: '5432' - name: DB_USER value: kong - name: DB_PASSWORD value: kong - name: DB_DATABASE value: kong - name: DB_PG_SCHEMA value: kong - name: NO_AUTH value: 'true' image: pantsel/konga:0.14.9 name: konga ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: konga namespace: kong spec: externalTrafficPolicy: Cluster ports: - name: konga port: 80 protocol: TCP targetPort: 80 selector: app: konga
初始化konga，默認konga不可執行，進入konga執行初始化db
./bin/konga.js -c prepare -a postgres -u postgresql://kong:kong@postgres:5432/konga
$ kb -n kong get pod
NAME READY STATUS RESTARTS AGE
ingress-kong-6b9544969-2pxwl 2/2 Running 0 125m
kong-migrations-6rshd 0/1 Completed 0 170m
postgres-767c99c648-fgd97 1/1 Running 0 20m
$ kb -n kong get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kong-proxy LoadBalancer 10.99.12.233 <pending> 80:30193/TCP,443:31473/TCP 170m
kong-validation-webhook ClusterIP 10.102.217.8 <none> 443/TCP 170m
postgres ClusterIP 10.105.201.5 <none> 5432/TCP 170m
暫時把 kong-proxy LoadBalancer 改為 NodePort
$ kb -n kong get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kong-proxy NodePort 10.99.12.233 <none> 80:30193/TCP,443:31473/TCP 171m
kong-validation-webhook ClusterIP 10.102.217.8 <none> 443/TCP 171m
postgres ClusterIP 10.105.201.5 <none> 5432/TCP 171m
驗證訪問
curl -i $PROXY_IP
HTTP/1.1 404 Not Found
Date: Tue, 30 Jun 2020 09:34:23 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Length: 48
X-Kong-Response-Latency: 1
Server: kong/2.0.4
{"message":"no Route matched with those values"}
驗證真實地址,確認kong 集成成功
$ curl -i $PROXY_IP/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Tue, 30 Jun 2020 09:34:40 GMT
Server: echoserver
X-Kong-Upstream-Latency: 2
X-Kong-Proxy-Latency: 4
Via: kong/2.0.4
Hostname: echo-599d77c5c7-jv8jl
Pod Information:
pod name: echo-599d77c5c7-jv8jl
pod namespace: default
pod IP: 192.168.63.51
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=192.168.111.254
method=GET
real path=/foo
query=
request_version=1.1
request_scheme=http
Request Headers:
accept=*/*
connection=keep-alive
user-agent=curl/7.29.0
x-forwarded-for=192.168.75.0
x-forwarded-port=8000
x-forwarded-proto=http
x-real-ip=192.168.75.0
Request Body:
-no body in request-
通過konga注冊綁定k8s的kong
試了 nodeport ingress host,api 都失敗 查看konga日志得
KongProxyController request error undefined
Sending 500 ("Server Error") response:
{
error: Error: self signed certificate
at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
at TLSSocket.emit (events.js:310:20)
at TLSSocket.EventEmitter.emit (domain.js:482:12)
at TLSSocket._finishInit (_tls_wrap.js:917:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12) {
code: 'DEPTH_ZERO_SELF_SIGNED_CERT'
}
}
error: unexpected EOF
是konga 訪問kong-admin-api的證書認證失敗
查看到官方kong ingress 默認的配置是
spec:
containers:
- env:
- name: KONG_PROXY_LISTEN
value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
- name: KONG_ADMIN_LISTEN
value: 127.0.0.1:8444 ssl
- name: KONG_STATUS_LISTEN
value: 0.0.0.0:8100
- name: KONG_DATABASE
value: postgres
- name: KONG_PG_HOST
value: stolon-proxy-service.default
- name: KONG_PG_PASSWORD
value: bia_miaozhen
- name: KONG_NGINX_WORKER_PROCESSES
value: "1"
- name: KONG_ADMIN_ACCESS_LOG
value: /dev/stdout
- name: KONG_ADMIN_ERROR_LOG
value: /dev/stderr
- name: KONG_PROXY_ERROR_LOG
value: /dev/stderr
image: kong:2.0
imagePullPolicy: IfNotPresent
KONG_ADMIN_LISTEN: 127.0.0.1:8444 ssl
更改 KONG_ADMIN_LISTEN 為 0.0.0.0:8444 ssl ，可以外部訪問，但證書認證不通過
兩個思路
- 1加證書認證
暫內網服務，為圖簡單，先不采用證書的方式
- 2通過http訪問
先公開http服務，看kong的官方示例和文檔
$ docker run -d --name kong \
--link kong-database:kong-database \
-e "KONG_DATABASE=postgres" \
-e "KONG_PG_HOST=kong-database" \
-e "KONG_CASSANDRA_CONTACT_POINTS=kong-database" \
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
-e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl" \
-p 8000:8000 \
-p 8443:8443 \
-p 8001:8001 \
-p 8444:8444 \
kong
KONG_ADMIN_LISTEN是允許公開多個地址的，帶ssl后綴的為https，不帶ssl的 為http
我們額外公開8001做為KONG_ADMIN_LISTEN http服務，重啟生效
- name: KONG_ADMIN_LISTEN
value: 0.0.0.0:8001, 0.0.0.0:8444 ssl
外部通過ingress 可以訪問kong-admin-api.bia.com
http://ingress-kong-admin:8001/
截圖和配置略有不同，個人的db是 stolon
kong ingress+konga 配置完畢
k8s kong ingress有一些集成的功能可以通過參數配置好，例如rewrite,下篇會談
也可以k8s kong ingress 只注冊一個標准的ingress，之后konga會同步顯示，在頁面上為該ingress配置各種插件
End