k8s学习笔记5-部署和应用ingress-nginx-controller-1.3.0
- 一.介绍和原理
-
- A.Ingress 是什么
- B.Ingress的组成
- 二.部署
-
- A.下载部署文件和镜像
- B.node节点打标签
- C.修改部署文件
- D.部署并查看
- 三.应用
- 四.参考文档
一.介绍和原理
A.Ingress 是什么
Ingress 从集群外到集群内公开服务 HTTP 和 HTTPS 路由。 流量路由由 Ingress 规则控制的资源定义。 下面是一个将所有流量发送到相同的流量 Service 的简单 Ingress 示例： 
B.Ingress的组成
ingress是一个API对象，通过yaml配置文件，ingress对象的作用是定义如何转发请求service配置模板可以理解为规则。 ingress通过http或https暴露集群内部service，给service提供外部URI、负载均衡、SSL/TLS基于域名的能力和反向代理。ingress要依靠ingress-controiler具体实现上述功能。 ingress-controller是实现反向代理和负载平衡的具体程序ingress根据配置规则则，并根据配置规则转发请求。 ingress-controller并不是k8s实际上，自带组件ingess-controller用户可以选择不同的总称ingress-controller实现，目前，由k8s维护的ingress-controller只有google云的GCE与ingress-nginx两个，还有很多其他的第三方维护ingress-controller，详见官方文件。 但不管是哪一种ingress-controller，实现的机制相似，但具体配置不同。
一般来说，ingress-controller一种形式pod，里面跑着demon程序和反向代理程序。daemon根据集群的变化，负责不断监控集群的变化ingress对象生成配置并的配置应用于反向代理，如ingress-nginx是动态生成nginx配置，动态更新upstreanm，必要时reload新配置的程序应用。为了方便，后面的例子都以k8s官方维护的ingress-nginx为例。
1.ingress-controller通过和 kubernetes APIServer互动，动态感知集群ingress规则变化； 2.然后读取它。根据自定义规则，规则是指哪个域名对应哪个域名service，生成一段nginx配置； 3.再写到nginx-ingress-controller的pod里，这个ingres-controller的pod一个在里面运行Nginx服务， 生成的控制器nginx置写入/etc/nginx.conf文件中； 4.然后reload使配置生效。为了实现域名区分配置和动态更新。
客户端通常会将请求发送到负载平衡器，从负载平衡器转发到负载平衡器ingress控制所在的node减少节点的目的Node节点的负载压力
ingress根据控制器ingress相应的资源对象配置转发请求service
service请求将根据端点转发到相关点Pod
二.部署
A.下载部署文件和镜像
1.下载ingress-nginx部署文件(如果文件无法下载，请参考链接https://blog.csdn.net/weixin_43656190/article/details/117604612)
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml 2.会导致镜像问题yaml文件中的镜像无法下载，解决方案：首先docker hub搜索并下载后，将其推送到私人harbor在镜像服务器上 下载以下两个镜像：
root@k8s-master1:~/ingress# cat deploy.yaml |grep imag image: registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5 imagePullPolicy: IfNotPresent image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660 imagePullPolicy: IfNotPresent image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660 imagePullPolicy: IfNotPresent 在docker hub上搜索
root@k8s-master1:~/ingress# docker search ingress-nginx --no-trunc NAME DESCRIPTION STARS OFFICIAL AUTOMATED willdockerhub/ingress-nginx-controller sync from k8s.gcr.io/ingress-nginx/controller 20 pollyduan/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 9 liangjw/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 7 wangshun1024/ingress-nginx-controller 3 dyrnq/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller 2 acicn/ingress-nginx-controller 2
wallarm/ingress-nginx 0
13916077927/ingress-nginx-controller sync from k8s.gcr.io/ingress-nginx/controller 0
rancher/ingress-nginx 0
v5cn/ingress-nginx-controller k8s.gcr.io/ingress-nginx/controller:v0.46.0 0
platform9/ingress-nginx 0
cangyin/ingress-nginx-controller 0
forging2012/ingress-nginx-controller 0
sundewang/ingress-nginx-controller 0
boy530/ingress-nginx-controller 0
blendlabs/ingress-nginx-catchall A blend branded nginx catchall. 0
cjk2atmb/ingress-nginx-controller 0
opsdockerimage/ingress-nginx-controller 0
acicn/ingress-nginx-kube-webhook-certgen 0
rudl/ingress-nginx Nginx ingress controller 0
opsdockerimage/ingress-nginx-nginx 0
lchdzh/ingress-nginx-controller 0
liy36/ingress-nginx-controller 0
opsdockerimage/ingress-nginx-e2e-test-runner 0
chainstack/ingress-nginx 0
root@k8s-master1:~/ingress# docker search kube-webhook-certgen --no-trunc
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
jettech/kube-webhook-certgen Kubernetes webhook certificate generator and patcher 14
liangjw/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 8
dyrnq/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 5
wangshun1024/kube-webhook-certgen 3
wonderflow/kube-webhook-certgen 1
lianyuxue1020/kube-webhook-certgen new pull lianyuxue1020/kube-webhook-certgen:v1.1.1 1
newrelic/kube-webhook-certgen 0
catalystcloud/kube-webhook-certgen 0
spdplx2021/kube-webhook-certgen 0
congruity360/kube-webhook-certgen 0
pangser/kube-webhook-certgen 0
oamdev/kube-webhook-certgen 0
sundewang/kube-webhook-certgen 0
hzde0128/kube-webhook-certgen 0
kubespheredev/kube-webhook-certgen 0
caas4/kube-webhook-certgen 0
rpkatz/kube-webhook-certgen 0
xiaoyan1121/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen 0
serenade11/kube-webhook-certgen 0
invidian/kube-webhook-certgen 0
serialt/kube-webhook-certgen 0
zhangdiandong/kube-webhook-certgen kube-webhook-certgen:v1.0 0
fnape/kube-webhook-certgen k8s.gcr.io/ingress-nginx/kube-webhook-certgen Mirror accessible from CN 0
giantswarm/kube-webhook-certgen 0
madforg/kube-webhook-certgen k8s ingress-nginx 实验所需的gcr.io镜像 ( k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 ) 0
root@k8s-master1:~/ingress#
3.下载，重命名，上传，替换镜像
#1.下载镜像
root@k8s-master1:~/ingress# docker pull dyrnq/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker pull lianyuxue1020/kube-webhook-certgen:v1.1.1
#2.将镜像重命名
root@k8s-master1:~/ingress# docker tag dyrnq/ingress-nginx-controller:v1.3.0 registry.harbor.com/library/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker tag lianyuxue1020/kube-webhook-certgen:v1.1.1 registry.harbor.com/library/kube-webhook-certgen:v1.1.1
#3.将镜像push到我们之前搭建的harbor仓库
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/ingress-nginx-controller:v1.3.0
The push refers to repository [registry.harbor.com/library/ingress-nginx-controller]
11d5b7949407: Pushed
a2ab1d98af90: Pushed
5b42063defba: Pushed
3ee663ef34d0: Pushed
bf8b0761126a: Pushed
158aa99c71d8: Pushed
05856d1bb285: Pushed
f1b0a27f62c1: Pushed
5f70bf18a086: Pushed
7f366fb88640: Pushed
12ca2606c07b: Pushed
ca8e478eeb76: Pushed
b45bedc7f782: Pushed
24302eb7d908: Pushed
v1.3.0: digest: sha256:067673df26a65ec5c2d5b30f25db869bad4d7d391fc81882250134577e581ef0 size: 3256
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/kube-webhook-certgen:v1.1.1
The push refers to repository [registry.harbor.com/library/kube-webhook-certgen]
ce7a3c1169b6: Pushed
c0d270ab7e0d: Pushed
v1.1.1: digest: sha256:78351fc9d9b5f835e0809921c029208faeb7fbb6dc2d3b0d1db0a6584195cfed size: 739
#4.将ingress-nginx部署文件中的镜像地址替换成harbor仓库的地址
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323e#registry.harbor.com/library/ingress-nginx-controller:v1.3.0#g deploy.yaml
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660#registry.harbor.com/library/kube-webhook-certgen:v1.1.1#g deploy.yaml
#5.查看镜像地址是否替换成功
root@k8s-master1:~/ingress# cat deploy.yaml |grep image
image: registry.harbor.com/library/ingress-nginx-controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
imagePullPolicy: IfNotPresent
image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
imagePullPolicy: IfNotPresent
root@k8s-master1:~/ingress#
B.node节点打标签
可以通过nodeSelector确定将ingress-nginx-controller部署在那些机器上
root@k8s-master1:~/ingress# kubectl label nodes k8s-node1 hasIngress=true
node/k8s-node1 labeled
root@k8s-master1:~/ingress# kubectl label nodes k8s-node2 hasIngress=true
node/k8s-node2 labeled
root@k8s-master1:~/ingress#
C.修改部署文件
修改三个地方 1.HostNetwork 是为了打通Cluster和node的网络，让Cluster直接监听node的端口，一般是80和443，不用再通过随机绑定的nodePort来访问集群服务
以前访问：http://www.xwl.com:30080/web1/contextpath
现在访问：http://www.xwl.com/web1/contextpath
可以看到不再需要随机绑定的nodePort了，因为hostNetwork直接监听集群的80端口。
2.nodeSelector 主要是为了能够选择将ingress-nginx-controller部署到那个机器上 3.DaemonSet 因为Deployment可能会把多个pod调度到同一个node，那就失去高可用的意义了。而DaemonSet在一个节点上只会有一个Pod，符合我们的要求。 修改deploy.yaml
......
apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet #这里把Deployment改成DaemonSet
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
app.kubernetes.io/version: 1.3.0
name: ingress-nginx-controller
namespace: ingress-nginx
spec:
minReadySeconds: 0
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
template:
metadata:
labels:
app.kubernetes.io/component: controller
app.kubernetes.io/instance: ingress-nginx
app.kubernetes.io/name: ingress-nginx
spec:
hostNetwork: true #这里加一句
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-controller-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
env:
......
nodeSelector:
kubernetes.io/os: linux
hasIngress: "true" # 这里个面具标签选择node
......
D.部署并查看
部署ingress-nginx,部署后，需要等待一会，大约1分钟这样，貌似部署成功，其实没有部署成功
root@k8s-master1:~/ingress# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
root@k8s-master1:~/ingress# kubectl get pods -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-vc554 0/1 Completed 0 3m18s 10.244.2.44 k8s-node2 <none> <none>
ingress-nginx-admission-patch-swbrw 0/1 Completed 0 3m18s 10.244.2.45 k8s-node2 <none> <none>
ingress-nginx-controller-c8bxg 1/1 Running 0 3m18s 192.168.100.243 k8s-node1 <none> <none>
ingress-nginx-controller-ph6vn 1/1 Running 0 3m18s 192.168.100.244 k8s-node2 <none> <none>
root@k8s-master1:~/ingress#
三.应用
1、创建tomcat-nginx.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: nginx-pod
template:
metadata:
labels:
app: nginx-pod
spec:
containers:
- name: nginx
image: nginx:1.17.1
ports:
- containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: tomcat-deployment
namespace: dev
spec:
replicas: 3
selector:
matchLabels:
app: tomcat-pod
template:
metadata:
labels:
app: tomcat-pod
spec:
containers:
- name: tomcat
image: tomcat:8.5-jre10-slim
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
namespace: dev
spec:
ports:
- port: 80
name: nginx
clusterIP: None
selector:
app: nginx-pod
---
apiVersion: v1
kind: Service
metadata:
name: tomcat-service
namespace: dev
spec:
ports:
- port: 8080
name: tomcat
clusterIP: None
selector:
app: tomcat-pod
2.创建ingress-http.yaml
kind: Ingress
metadata:
name: ingress-http
namespace: dev
spec:
ingressClassName: nginx
rules:
- host: nginx.xwl.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx-service
port:
number: 80
- host: tomcat.xwl.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: tomcat-service
port:
number: 8080
3.修改任意一个主机的/etc/hosts文件，添加如下4个域名解析
192.168.100.243 nginx.xwl.com
192.168.100.244 nginx.xwl.com
192.168.100.243 tomcat.xwl.com
192.168.100.244 tomcat.xwl.com
4.验证是否能够正常登录 登录nginx.xwl.com
登录tomcat.xwl.com
四.参考文档
1.ingress官方文档 https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/ 2.ingress-nginx官方文档 https://kubernetes.github.io/ingress-nginx/deploy/#quick-start 3.hostNetwork与ingress-nginx高可用 https://blog.csdn.net/weixin_41535316/article/details/121873795 4.k8s部署ingress-nginx的方法步骤 https://blog.csdn.net/a13568hki/article/details/124452407 5.K8S对外服务之Ingress （使用https的时候，可以参考这篇文档） https://blog.51cto.com/u_15382481/5205194 6.k8s ingress原理 https://blog.csdn.net/weixin_46108954/article/details/105717399