一、如何實現https負載均衡 Ingress和Ingress-controller
1、存在問題
kubernetes中、service資源和pod資源的IP地址僅能用於集群網絡內部的通信
所有的網絡流量都無法穿透邊界路由器以實現集群內外通信、盡管可以為service使用NodePortand LoadBalancer類型通過節點引入外部流量
但它依然是4層部分流量轉發、可用的負載均衡器也為傳輸層負載均衡機制
2、解決方案及工作原理
Ingress是kubernetes API的標准資源類型之一、它其實就是一組基於DNS名稱或URL路徑把請求轉發至指定的service資源的規則
用於將集群外部的請求流量轉發至集群內部完成服務發布、然而Ingress資源自身並不能進行"流量穿透"、它僅是一組路由規則的集合
這些規則要想真正發揮作用還需要其他功能的輔助、如監聽套接字、然后根據這些規則的匹配機制路由請求流量、這種為夠為Ingress資源監聽套接字並轉發流量的組件成為Ingress控制器
Ingress控制器並不直接運行為kube-controller-manager的一部分、它是kubernetes集群的一個重要附件、類似於CoreDNS、需要在集群上單獨部署
3、Ingress控制器的實現
Nginx、Envoy、HAProxy、Vulcand和Traefik等
Ingress控制器自身也是運行於集群中的pod資源對象、它於被代理的運行為pod資源的應用運行於同一網絡中
4、優點和優勢
使用Ingress資源進行流量分發是、Ingress控制器可基於某Ingress資源定義的規則將客戶端請求流量直接轉發至於service對應的后端pod資源上
這種轉發機制會繞過service資源、從而省去了由kube-proxy實現的端口代理開銷
Ingress規則需要由一個service資源對象輔助識別相關的所有pod對象、但Ingress-nginx控制器可經由api.ilinux.io規則的定義直接將請求流量
調度至pod3或pod4、而無需經由service對象API的再次轉發、WAP相關規則的作用方式與此類同
二、安裝ingress-nginx
1、Ingress.spec核心資源
[root@master chapter6]# kubectl explain Ingress.spec
KIND: Ingress
VERSION: extensions/v1beta1
RESOURCE: spec <Object>
DESCRIPTION:
Spec is the desired state of the Ingress. More info:
https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#spec-and-status
IngressSpec describes the Ingress the user wishes to exist.
FIELDS:
backend <Object>
#默認的后端用於服務那些沒有匹配到任何規則的請求；定義Ingress資源時、至少應該定義backend或rules兩者之一；此字段用於讓負責均衡器指定一個全局默認的后端
A default backend capable of servicing requests that don't match any rule.
At least one of 'backend' or 'rules' must be specified. This field is
optional to allow the loadbalancer controller or defaulting logic to
specify a global default.
#backend對象的定義由兩個必選的內嵌字段組成：serviceName和servicePort、分別用於指定流量轉發的后端目標serbice資源的名稱和端口
ingressClassName <string>
IngressClassName is the name of the IngressClass cluster resource. The
associated IngressClass defines which controller will implement the
resource. This replaces the deprecated `kubernetes.io/ingress.class`
annotation. For backwards compatibility, when that annotation is set, it
must be given precedence over this field. The controller may emit a warning
if the field and annotation have different values. Implementations of this
API should ignore Ingresses without a class specified. An IngressClass
resource may be marked as default, which can be used to set a default value
for this field. For more information, refer to the IngressClass
documentation.
rules <[]Object>
#用於定義當前Ingress資源的轉發規則列表；未由rules定義規則、或者沒有匹配到任何規則時、所有流量都會轉發到由backend定義的默認后端
A list of host rules used to configure the Ingress. If unspecified, or no
rule matches, all traffic is sent to the default backend.
#對象由一系列配置Ingress資源的host規則組成、這些host規則用於將一個主機上的某個URL路徑映射至相關的后端service對象
tls <[]Object>
#TLS配置、目前僅支持通過默認端口443提供服務；如果要配置指定的列表成員指向了不同的主機、則必須通過SNI TLS擴展機制來支持
TLS configuration. Currently the Ingress only supports a single TLS port,
443. If multiple members of this list specify different hosts, they will be
multiplexed on the same port according to the hostname specified through
the SNI TLS extension, if the ingress controller fulfilling the ingress
supports SNI.
2、Ingress.spec.rules對象
[root@master chapter6]# kubectl explain Ingress.spec.rules
KIND: Ingress
VERSION: extensions/v1beta1
RESOURCE: rules <[]Object>
DESCRIPTION:
A list of host rules used to configure the Ingress. If unspecified, or no
rule matches, all traffic is sent to the default backend.
IngressRule represents the rules mapping the paths under a specified host
to the related backend services. Incoming requests are first evaluated for
a host match, then routed to the backend associated with the matching
IngressRuleValue.
FIELDS:
host <string>
#屬性值目前不支持使用IP地址、也不支持后跟"PORT" 格式的端口號、且此字段值留空表示統配所有的主機名
Host is the fully qualified domain name of a network host, as defined by
RFC 3986. Note the following deviations from the "host" part of the URI as
defined in RFC 3986: 1. IPs are not allowed. Currently an IngressRuleValue
can only apply to the IP in the Spec of the parent Ingress. 2. The `:`
delimiter is not respected because ports are not allowed. Currently the
port of an Ingress is implicitly :80 for http and :443 for https. Both
these may change in the future. Incoming requests are matched against the
host before the IngressRuleValue. If the host is unspecified, the Ingress
routes all traffic based on the specified IngressRuleValue. Host can be
"precise" which is a domain name without the terminating dot of a network
host (e.g. "foo.bar.com") or "wildcard", which is a domain name prefixed
with a single wildcard label (e.g. "*.foo.com"). The wildcard character '*'
must appear by itself as the first DNS label and matches only a single
label. You cannot have a wildcard label by itself (e.g. Host == "*").
Requests will be matched against the Host field in the following way: 1. If
Host is precise, the request matches this rule if the http host header is
equal to Host. 2. If Host is a wildcard, then the request matches this rule
if the http host header is to equal to the suffix (removing the first
label) of the wildcard rule.
http <Object>
3、Ingress.spec.tls對象
[root@master data]# kubectl explain Ingress.spec.tls
KIND: Ingress
VERSION: extensions/v1beta1
RESOURCE: tls <[]Object>
DESCRIPTION:
TLS configuration. Currently the Ingress only supports a single TLS port,
443. If multiple members of this list specify different hosts, they will be
multiplexed on the same port according to the hostname specified through
the SNI TLS extension, if the ingress controller fulfilling the ingress
supports SNI.
IngressTLS describes the transport layer security associated with an
Ingress.
FIELDS:
hosts <[]string>
#包含於使用的TLS證書之內的主機名稱字符串列表、因此、此處使用的主機名必須匹配tlsSecret中的名稱
Hosts are a list of hosts included in the TLS certificate. The values in
this list must match the name/s used in the tlsSecret. Defaults to the
wildcard host setting for the loadbalancer controller fulfilling this
Ingress, if left unspecified.
secretName <string>
#用於引用SSL會話的secret對象名稱、在基於SNI實現多主機路由的場景中、此字段為可選
SecretName is the name of the secret used to terminate SSL traffic on 443.
Field is left optional to allow SSL routing based on SNI hostname alone. If
the SNI host in a listener conflicts with the "Host" header field used by
an IngressRule, the SNI host is used for termination and value of the Host
header is used for routing.
三、Ingress資源類型
1、單service資源型Ingress
暴露單個服務的方法有很多種、如服務類型中的NodePort and LoadBalancer、不過一樣可以考慮使用IngressL來暴露服務、此時只需要為Ingress指定
"default-backend" 即可、例如下面的示例
Ingress控制器會為其分配一個IP地址接入請求流量、並將它們轉至示例中的my-svc后端
apiVersion: v1
kind: Ingress
metadata:
name: my-ingress
spec:
backend:
serviceName: test
servicePort: 80
2、基於URL路徑進行流量分發
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: simple-fanout-example
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: service1
servicePort: 4200
- path: /bar
backend:
serviceName: service2
servicePort: 8080
3、基於主機名稱的虛擬機主機
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: name-virtual-host-ingress
spec:
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: bar.foo.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
4、TLS類型的Ingress資源
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: test-ingress
spec:
backend:
serviceName: testsvc
servicePort: 80
四、部署Ingress控制器(Nginx)
部署Ingress-Nginx控制器的配置文件被切割存放在了多個不同的文件中、並集中存儲於源碼deploy子目錄下、同時為了方便用戶部署、它還將所有的資源全部集成為一個配置文件mandatory.yaml
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/deploy.yaml 修改331行鏡像為：image: siriuszg/nginx-ingress-controller:latest
1、創建運行
[root@master data]# kubectl apply -f deploy.yaml namespace/ingress-nginx unchanged serviceaccount/ingress-nginx unchanged configmap/ingress-nginx-controller configured clusterrole.rbac.authorization.k8s.io/ingress-nginx unchanged clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged role.rbac.authorization.k8s.io/ingress-nginx unchanged rolebinding.rbac.authorization.k8s.io/ingress-nginx unchanged service/ingress-nginx-controller-admission unchanged service/ingress-nginx-controller unchanged deployment.apps/ingress-nginx-controller configured validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission configured serviceaccount/ingress-nginx-admission unchanged clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission unchanged clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged role.rbac.authorization.k8s.io/ingress-nginx-admission unchanged rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission unchanged job.batch/ingress-nginx-admission-create unchanged job.batch/ingress-nginx-admission-patch unchanged
2、驗證
[root@master chapter6]# kubectl get pods -n ingress-nginx -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ingress-nginx-admission-create-6nwwj 0/1 Completed 0 15m 10.244.2.103 nodes2 <none> <none> ingress-nginx-admission-patch-qndv7 0/1 Completed 3 15m 10.244.0.91 master <none> <none> ingress-nginx-controller-5d9498494d-97blq 1/1 Running 0 38s 10.244.0.92 master <none>
在線的配置清單中采用了基於Deployment控制器部署ingress nginx的方式、因此接入外部流量之前還要手動為其創建相關的NodePort and LoadBalancer
類型的service資源對象、下面的配置清單示例中對類型定義了NodePort、並明確指定了易記的端口和IP地址
3、service資源清單
[root@master chapter6]# cat nginx-ingress-service.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx-ingress-controller
namespace: ingress-nginx
spec:
type: NodePort
clusterIP: 10.99.99.99
ports:
- port: 80
name: http
nodePort: 30080
- port: 443
name: https
nodePort: 30443
selector:
app.kubernetes.io/name: ingress-nginx
4、創建service資源
[root@master chapter6]# kubectl apply -f nginx-ingress-service.yaml service/nginx-ingress-controller created
5、驗證
[root@master chapter6]# kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx-controller NodePort 10.96.209.11 <none> 80:30724/TCP,443:32624/TCP 30m ingress-nginx-controller-admission ClusterIP 10.96.105.3 <none> 443/TCP 30m nginx-ingress-controller NodePort 10.99.99.99 <none> 80:30080/TCP,443:30443/TCP 13m
如果讀者的集群運行支持LBaaS的IaaS雲環境、則可以將其類型指定為LoadBalancer這樣直接就有了可用的external-LB
6、測試代碼和截圖
[root@master data]# curl tomcat.ikubernetes.io:30080
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>Apache Tomcat/8.0.50</title>
<link href="favicon.ico" rel="icon" type="image/x-icon" />
<link href="favicon.ico" rel="shortcut icon" type="image/x-icon" />
<link href="tomcat.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="wrapper">
<div id="navigation" class="curved container">
<span id="nav-home"><a href="http://tomcat.apache.org/">Home</a></span>
<span id="nav-hosts"><a href="/docs/">Documentation</a></span>
<span id="nav-config"><a href="/docs/config/">Configuration</a></span>
<span id="nav-examples"><a href="/examples/">Examples</a></span>
<span id="nav-wiki"><a href="http://wiki.apache.org/tomcat/FrontPage">Wiki</a></span>
<span id="nav-lists"><a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></span>
<span id="nav-help"><a href="http://tomcat.apache.org/findhelp.html">Find Help</a></span>
<br class="separator" />
</div>
<div id="asf-box">
<h1>Apache Tomcat/8.0.50</h1>
</div>
<div id="upper" class="curved container">
<div id="congrats" class="curved container">
<h2>If you're seeing this, you've successfully installed Tomcat. Congratulations!</h2>
</div>
<div id="notice">
<img src="tomcat.png" alt="[tomcat logo]" />
<div id="tasks">
<h3>Recommended Reading:</h3>
<h4><a href="/docs/security-howto.html">Security Considerations HOW-TO</a></h4>
<h4><a href="/docs/manager-howto.html">Manager Application HOW-TO</a></h4>
<h4><a href="/docs/cluster-howto.html">Clustering/Session Replication HOW-TO</a></h4>
</div>
</div>
<div id="actions">
<div class="button">
<a class="container shadow" href="/manager/status"><span>Server Status</span></a>
</div>
<div class="button">
<a class="container shadow" href="/manager/html"><span>Manager App</span></a>
</div>
<div class="button">
<a class="container shadow" href="/host-manager/html"><span>Host Manager</span></a>
</div>
</div>
<!--
<br class="separator" />
-->
<br class="separator" />
</div>
<div id="middle" class="curved container">
<h3>Developer Quick Start</h3>
<div class="col25">
<div class="container">
<p><a href="/docs/setup.html">Tomcat Setup</a></p>
<p><a href="/docs/appdev/">First Web Application</a></p>
</div>
</div>
<div class="col25">
<div class="container">
<p><a href="/docs/realm-howto.html">Realms & AAA</a></p>
<p><a href="/docs/jndi-datasource-examples-howto.html">JDBC DataSources</a></p>
</div>
</div>
<div class="col25">
<div class="container">
<p><a href="/examples/">Examples</a></p>
</div>
</div>
<div class="col25">
<div class="container">
<p><a href="http://wiki.apache.org/tomcat/Specifications">Servlet Specifications</a></p>
<p><a href="http://wiki.apache.org/tomcat/TomcatVersions">Tomcat Versions</a></p>
</div>
</div>
<br class="separator" />
</div>
<div id="lower">
<div id="low-manage" class="">
<div class="curved container">
<h3>Managing Tomcat</h3>
<p>For security, access to the <a href="/manager/html">manager webapp</a> is restricted.
Users are defined in:</p>
<pre>$CATALINA_HOME/conf/tomcat-users.xml</pre>
<p>In Tomcat 8.0 access to the manager application is split between
different users. <a href="/docs/manager-howto.html">Read more...</a></p>
<br />
<h4><a href="/docs/RELEASE-NOTES.txt">Release Notes</a></h4>
<h4><a href="/docs/changelog.html">Changelog</a></h4>
<h4><a href="http://tomcat.apache.org/migration.html">Migration Guide</a></h4>
<h4><a href="http://tomcat.apache.org/security.html">Security Notices</a></h4>
</div>
</div>
<div id="low-docs" class="">
<div class="curved container">
<h3>Documentation</h3>
<h4><a href="/docs/">Tomcat 8.0 Documentation</a></h4>
<h4><a href="/docs/config/">Tomcat 8.0 Configuration</a></h4>
<h4><a href="http://wiki.apache.org/tomcat/FrontPage">Tomcat Wiki</a></h4>
<p>Find additional important configuration information in:</p>
<pre>$CATALINA_HOME/RUNNING.txt</pre>
<p>Developers may be interested in:</p>
<ul>
<li><a href="http://tomcat.apache.org/bugreport.html">Tomcat 8.0 Bug Database</a></li>
<li><a href="/docs/api/index.html">Tomcat 8.0 JavaDocs</a></li>
<li><a href="http://svn.apache.org/repos/asf/tomcat/tc8.0.x/">Tomcat 8.0 SVN Repository</a></li>
</ul>
</div>
</div>
<div id="low-help" class="">
<div class="curved container">
<h3>Getting Help</h3>
<h4><a href="http://tomcat.apache.org/faq/">FAQ</a> and <a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></h4>
<p>The following mailing lists are available:</p>
<ul>
<li id="list-announce"><strong><a href="http://tomcat.apache.org/lists.html#tomcat-announce">tomcat-announce</a><br />
Important announcements, releases, security vulnerability notifications. (Low volume).</strong>
</li>
<li><a href="http://tomcat.apache.org/lists.html#tomcat-users">tomcat-users</a><br />
User support and discussion
</li>
<li><a href="http://tomcat.apache.org/lists.html#taglibs-user">taglibs-user</a><br />
User support and discussion for <a href="http://tomcat.apache.org/taglibs/">Apache Taglibs</a>
</li>
<li><a href="http://tomcat.apache.org/lists.html#tomcat-dev">tomcat-dev</a><br />
Development mailing list, including commit messages
</li>
</ul>
</div>
</div>
<br class="separator" />
</div>
<div id="footer" class="curved container">
<div class="col20">
<div class="container">
<h4>Other Downloads</h4>
<ul>
<li><a href="http://tomcat.apache.org/download-connectors.cgi">Tomcat Connectors</a></li>
<li><a href="http://tomcat.apache.org/download-native.cgi">Tomcat Native</a></li>
<li><a href="http://tomcat.apache.org/taglibs/">Taglibs</a></li>
<li><a href="/docs/deployer-howto.html">Deployer</a></li>
</ul>
</div>
</div>
<div class="col20">
<div class="container">
<h4>Other Documentation</h4>
<ul>
<li><a href="http://tomcat.apache.org/connectors-doc/">Tomcat Connectors</a></li>
<li><a href="http://tomcat.apache.org/connectors-doc/">mod_jk Documentation</a></li>
<li><a href="http://tomcat.apache.org/native-doc/">Tomcat Native</a></li>
<li><a href="/docs/deployer-howto.html">Deployer</a></li>
</ul>
</div>
</div>
<div class="col20">
<div class="container">
<h4>Get Involved</h4>
<ul>
<li><a href="http://tomcat.apache.org/getinvolved.html">Overview</a></li>
<li><a href="http://tomcat.apache.org/svn.html">SVN Repositories</a></li>
<li><a href="http://tomcat.apache.org/lists.html">Mailing Lists</a></li>
<li><a href="http://wiki.apache.org/tomcat/FrontPage">Wiki</a></li>
</ul>
</div>
</div>
<div class="col20">
<div class="container">
<h4>Miscellaneous</h4>
<ul>
<li><a href="http://tomcat.apache.org/contact.html">Contact</a></li>
<li><a href="http://tomcat.apache.org/legal.html">Legal</a></li>
<li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
<li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
</ul>
</div>
</div>
<div class="col20">
<div class="container">
<h4>Apache Software Foundation</h4>
<ul>
<li><a href="http://tomcat.apache.org/whoweare.html">Who We Are</a></li>
<li><a href="http://tomcat.apache.org/heritage.html">Heritage</a></li>
<li><a href="http://www.apache.org">Apache Home</a></li>
<li><a href="http://tomcat.apache.org/resources.html">Resources</a></li>
</ul>
</div>
</div>
<br class="separator" />
</div>
<p class="copyright">Copyright ©1999-2020 Apache Software Foundation. All Rights Reserved</p>
</div>
</body>
</html>