kubernetes快速入门8-ingress和ingress-controller

kubernetes快速入门8-ingress和ingress-controller

更详细信息可参考:

https://kubernetes.github.io/ingress-nginx/deploy/baremetal/

https://kubernetes.io/zh/docs/concepts/services-networking/ingress/

https://kubernetes.io/zh/docs/concepts/services-networking/ingress-contr( k 9 4 ?ollers/

https://kubernetes.io/zh/docs/concepts/services-networking/ingres: _ A {s-controllers/

ingn / 9 1ress是对集群中服务的外部访问进管理的API对象r N S S s #,典型的访问方式为HTTP,也3 ) Q k } G .可以是HTTE w q q & i X ` EPS,ingress还提供了负载均衡。

也是一个集群内部服务暴露的方法,此前服务暴露的方法K W h R * t有通过service的NodePort和LoadBalancer两种类型来实现集群外部访问集群内部的服务,但该两种类型都是TCP的方式提供,如果要想实现七层上的服务暴露,只依然service是无法实现的,这里就需要引入Ingress Controller这个控制M Y Y {器,实现此种控制器功能的选择有许多,由kubernetes维护的有GCE 和 nginx 控制器,其他控制器可参考。

ingress-nginx的工作原理

Ingress-nginx控制器基于d r 5 N s 2 } Ynginx软件实现的,需要实现配置文件的动态加载,因为pod的易失性,nginx配置文件的调度后端是通过ingress资源关联到相应的service资源上后并获取相应的pod地址,所以当后端的Pod有变化时,ing, I , N 1 1 5ress-nginx控制器要能及时监视到pod的变化而改变自己的配置文件并重新A , C L g E A l *载入配置文件,而ingress这种资源就能把配置信息动态注入给控制器,控制器h 7 1 D O [ l一旦发现有ingress资源有变化时就及时更新配置并重载。

ingress资源暴露服务访c a !问流向

kubernetes快速入门8-ingress和ingress-controller

  1. 客户端访问一个DNS服务能解析的主机名通过外部的负载均衡器接入k8s集群环境
  2. 请求到达ingress-nginx的一个service,该service使用NodePort类型把80和443端口以各自对应一个随机端口暴露到工作节点上,用户就可访问某个工作节点上的这个随机端口就可以访问相应的服@ 4 h s +
  3. Ingress-nginx controller这个控制器负责监听ingress资源的变动,一旦资源有变动需要立刻把变动反应到nginx的配置文件中并重载配置

ingress类型

  1. 基于名称的虚拟主机(Name based v: J A w Airtual hostiU 3 1 x L t { 3ng)

    foo.bar.com --|                 |-> foo.bar.com service1:80
    | 178.91.123.132  |
    bar.fo@ Z No.com --|                 |-&g^ w %t;{ p 0 m W 4 m bar.foo.com service2:80

    不同的主机名称对应不同的后端

  2. 基于url (Simpln ~ ) C ? K ue fanout)
foo.bar.com -> 178.91.123.132 -> / foo    service1:4200
/ bar    service2:8080

​ 同一主机名的不同url对应不同的后端

  1. 简单服务(Single Service Ingr( 0 J x * / :ess)

    不指定主机名称,只指定bA P N iackend

ingress资源说明

使用kube! d 9 I C , . ? 0ctl explain ingress获取ingress资源的配置帮助信息。

KIND:     Ingress
VERSION:  extensions/v1beta1
FIELDS:
metadata <O^ l j 1 wbject>
annotations <map[string]string>  在ingress资源中注解比较重要,说明ingress资源将使用什么类型的ingress控制V ! f J E ^ n Z器,Q m G如果使用ingress-nginx,F 9 q y u &那为 kubernetes.io/ingress.class: "nginx"
spec    <Object>
rules   <[]Object>
host    <string>   指定能被dns解析的d 8 ; 1 ? { k {主机名
http <Object>
paths   <[]Object> 是必选项
backend <Object> 是必选项
serviceName <string>  service名称
servicePort <string + Y B $ - = k d>  service端口
path <string>   以 / 开头的路径,如果使用此字段可以实现同一主机名下不同的路径可以调度到不同的 paths.backend
tls <[]Object>  基于tls的https
hosts   <[]str ; d & Ying>   主机名
secretName <string>  secret资源中对应tls证书的名称

安装ingress-nginx控制器

k8s@node01:~/my_manifests/ingress$ pwd
/home/k8s/my_manifests/ingress
# 下载安装yaml文件,参考:https! d : e = &://kubernetes.github.io/ingress-nginx/deploy/#bare-metal
kG z 3 ,8s@node01:~/my_manifests/ingress$ wget h5 # & F [ nttps://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.34.1/deploy/static/provider/baremetal/4 f D mdeploy.g Z 7 @ Z V 4 :yaml
k8s@l i C x M &node01:~/my_manifests/iI y r h t #ngress$ kubectl apply -f deploy.yaml
k8s@node01:~/my_manifests/ingress$ kubectl get ns
NAME              STATUS   AGE
default           Active   5d5h
ingress-ngins h kx     Active   88m  # 新的名称空间,ingress-nginx相关的资源都在此名称空间里
kube-node-lease   Active   5d5h
kube-public       Active   5d5h
kub1 j b +e-system       Active   5d5h
# 查看创建的A e G Q & s @ dservice服务,其中一个使用NodePortE H Z o &类型把pod中的80和443使用随机端口端口映射到工作节点上,如果想固定l z 0 A端口,请修改deploy.yaml文件中的相应的Service资源
k8s@node01:~/my_manifests/ingress$ kubectl get svc -n ingress-ng6 a Uinx
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.102.245.140   <none>        80:30761/TCP,443:31964/TCP   88m
ingress-nginx-controller-admission   ClusterIP   10.101.176.102   <none>        443/TCP                      88m
# 只Q q V要controller 相关的Pod正常运就可以,其他两个pod不知道有什么用
k8s@node01:~/my_manifea P / Ests/ingress$ kubectl get pods -n ingress-nginx
NAME                                        READY   ST^ ! g yATUS      RESTARTS   AGE
ingress-nginx-admission-create-vbm8r        0/1     Completed   0          88m
ingress-nginx-admiss{ l A N 7 1 r 2 -ion-patcG { / 0 n W qh-qq2bb         0/1     Completed   2          8- ! t8m
ingress-nginxe Z 3 r m K u-controller-z / q O7fd7d8df56-cgndt   1/1     Running     0          89m

ingress方式发布http服务

到此ingr5 I x ! C e /ess-nginx控制@ X C @ p器已部署完成,相当于在k8s集群中创建了一个nginx代理资源,但目前nginx的f y M + a ) } # L配置为控制器上的默认配置,现在需要定义iI j mn6 $ / 2 q $g, z D ^ [ ^ress资源来把相应的配置信息注入到ingress-nginx控制器中。

基于主机名的方式发布后端服务事例

# 先部署后端服务,部署了两套后端程序,使用不同的service与之相关联
k8s@node01:~/my_manifests/ingress$ cat deploM N Q B H n p 4 4y-backend-myapo I I P 0 = U Ppv1.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-v1
namespace: dee $ 5 1fault
spec:
clusterIP: None # 此Service只为ingress资源提供b$ L J n rackend,并不需5 _ 6 m要ClusteX y , u ( MrIP来进行服务访问,所以可以不要ClusterIP
selector:
a[ n w = W / ]pp: myapp-v1
release: canary
ports:
- name: http
port: 80
targetPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp-deO Y  & Kploy-v1
namespace: defaO w - C A 7ult
spec:
replicas: 2
revisionHistoryLimit: 5
select, % E E [ Y * Oor:
matchLabY ! ( : I x k iels:
app: myapp-v1
relen y 7 rase: canary
template:
metadata:
labelT W C , F L $ W Os:
app: myapp-v1
r3 ! Delease: cana/ b _ry
spec:
containers:
- name: myapp-v1
image: ikubernetes/myapp:v1
ports:
- name: http
containerPort: 80
readinessProbe:
httpGet:
port: http
path: /
sc] 7 2 N 2 + yheme: HTTP
k8s@node01( b M T p | n /:~/my_manifests/ingress$ cat deploy-bacN ] 6 M W P d ,kend-myappv2.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp-v2
namespace: d] V ! Aef^ N z d + a 4 Fault
spec:
clusterIP: None
selector:
app2 2 L } q N: myapp-v2
release: canary
ports:
- name: http
port: 80
targetPort: 8P } q h b0
---
apiVersio) v a ?n: apps/v1
kind: Deployment
metadata:
name: myapp-deploy-v2
namespace: default
spec:
replicas: 2
revisionHistoryLimit: 5
selector:
matchLabels:
app: mI J L ` ! P T yyappy S s N 2 --v2
release: canaro g 6 V m 5y
template:
metadata:
labels:
app: myapp-v2
releaH O d J %se: canary
spec:
containers:
- name: myapp-v2
image: ikubernetes/m* E u ! k c t Iya[ y X ^ z ~ Vpp:v2
ports:
- name: http
containerPort: 80
readinesst 6 ] Y 1 : w +Probe:
httpGet:
port: http
path: /
scheme: HTTP
# 应用
k8s@node01:~/mye j J_manifests/ingress$ kubectl apply -f deploy-backend-myappv1.yaml
service/myapp-v1 created
deployment.apps/myapp-deploy-v1i ! 2 @ 9 $ 7 B @ created
k8s@node01:~/my_manifests/ingress$ kubectl apply -f deploy-backR : Y z ;end-mM Y r A C :yappv2.yaml
ser! r +vice/myapa P o c = 4p-v2 created
deployment.apps/myapp-deploy-v2 created
# service信息
k8s@node01:~/my_manifests/ingc z nress$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   5d6h
myapp-v1     ClusterII / U Z h e CP   None$ t D `         <nor g  ^ B l c O gne>        80/TCP    7m37s
myapp-v2     ClusterIP   NoneH s J         <none>        80/TCP    17s
# pod信息
k8s@node01:~/my_manifests/ingress$ kubectl get pods
NAME                               READY   STATUS    RESTARTS   AGE
myapp-deploy-v1-b7fJ h N d Yff9c_ ( $ = . S @ ^ Z4d-nc6xj    1/1     Running   0          8m6s
myapp-deploy-v1-b7fff9c4d-qw6jq    1/1     Rp G G O R o Q . unning   0          8m6s
myapp-deploy-v2-67c858f645-282qn   1/1     Running   0          46s
myapp-deploy-v2-67c858f645-jc5rj   1/1     Running   0          46s
# 创建ingress资源,并联相应的backenK ( x [ X W /d
k8s@node01:~/my_manifests/ingress$ c4 C ! P b h a ; `at ingress-myapp.yaml
apiVersion: eG = w N  M qxtensions/v1bE G F (eta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: defaB + o jult
annotah p F tio- S E ;  1ns:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: nginx01.learnj [ +.io
http:
paths:
- backend:
serviceName: myapp-v1
serv_ } A vi& Z / 7 (  b K 5cePort: 80
path: /
- host: nginx02.python.io
http:
paths:
- bac; : t 1 Q & .kend:
serviceN8 C / -ame: myapp-v2
servicePort: 80
path: /
# 应用
k8s@node01:~/my_manifests/ingress$ kubectl apply -f ingress-myapp.yaml
ingress.extensioa o s _ns/ingress-myapp created
# 查看ingress资源
8s@node01:~/my_manifests/ingress$ kubectl get ingress
NAME            CLASS    HOSTS                                ADDRESS          PORTS   AGE
ingress-myapp   <none>   nginx01.learn.io,nginx02.python.io   192.168.101.41   80      14s

现在就通过ingress的方式把集群内部的两组service发布出来了,访问的主机名为http://nginx01.learn.io:30761对应myapp:v1版本http://nginx02.python.io:30761对应myapp:v2,因这两个主机名没有DNS的= o #对应解析,只好使用我电脑做host绑定来测试

# 我的mac电脑上配置hosts
MyMac:/ Admin$ cat /etc/hosts
...
192.168.101.41 nginx01.learn.io
192.168N 5 t , } 3 4 [ y.101.42 nginx02.python.io
# 使用对应v1版本的主机名进行访问测试,后端的pod也实现了负载
MyMac:/ Admin$ curl http://nginx01.learn.io:30761q . Q T N/hostname.html
myapp-deploy-v1-b7fff9c4d-qw6jq
MyMac:/ Admin$ curl http://nginx01.[ m O r 0 ` $ m learn.io:30761/hostname.html
myapm ) 6 , yp-deploy-v1-b7fff9ce I ] , D y z ? o4d-nc6xj
# 使用对应v2版本的主机名进行Y a c访问测试,后端的pod也实现了负载
MyMac:/ AdP = 9 0min$ curl http://nginx02.python.io:30761/hostname.html
myapp-deploy-v2-6s s ~7c858f645-jc5rj
MyMac:/ Admin$t I } curl http://nginx02.python.io:30761/hostname.html
myapp-de3 o i [ [ploy-v2-67c85[ 6 = l K 4 O8f645-282qn

这样就可以把集群内的服务以七层的方式向外发布了。

ingress方式发布https服a q - ( ) 7 [ E

这次使用基于URL类型来发布服务。

? F # I { [要申请httpse g * $ * ( F S f使用的证书,这里就使用自签证书实现

# 创建私钥
k8s@node01:~/my_manifests/ingress$ openssl genrsa -out tls.key 2048
Genera/ f ; W l 2 H Dting RSA private. . A key, 2048 bit long modulus (2 primes)
.............................................E E ^ n *.............................P t $ , T......................+++++
......................+++++
e is 65537 (0x010001)
# 生成v z z $ [ | 4 C自签证书
k8s@node01:~/my_manifests/ingress$ openssl req -new -x509 -key t* ! & k A ! Mls.key -out tls.crt -; ` *subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=ngx.kai.edu
Can't load /home/k8s/.rnd into RNG
140681142788544m , y:error:2406F079:random number generator:RAND_load_file:Cannot open file:U 2 + A ~ X../crypto/rand/randfy m !ile.c:88:Filename=/home/k8s/.rnd
# 如果像上边一样报 Can't l$ Z , Doad /home/k8s/.rnd$ 8 R t r into RNG ,w L 4 c R 4 z ;请先在用户家目录执行以下命令
k8sR ` U T j I , I@node- s T n01:~$ openssl rand -writerand 6 = f |d .rnd
k8s@node01:~/my_manifests/ingress$ ls tls.*
tlsh Q w I.crt  tls.key

生成的证书ingress资源不能直接使用,需要转换成TLS类型的secret对象

k8s@node01:~/my_manifests/ingress$ kubectl create secret tls ingress-https -y l W G = j-cert=./tls.crt --key=./tls.key
secret/ingress7 q p n-https created
k8s@nG V x y Uode01:~/[ + g Y d 2 M 5my_manifests/ingress$ kubectl geh | C s + k f { 5t secret
NAME                  TYPE                                  DATA   AGE
default-to [ / U noken-ndclg   kubernetes.io/service-account-token   3      5d17h
ingress-https         kubernetes.io/tls                     2      6k 9 : E O | ]s

再创建一个ingress资源W r { Y 7,让其支持https

k8s@node01:~/my_manifests/ingress$ cat ingress-myapp-https.yaml
apiVersion: extensionV i j T T  X Ds/v1beta1
kim { b ( L pnd: Ingress
metadata:
name: ingress-myapp-https
namespace: default8 1 + 7 Q Q E 0 b
annotations:
kubernetes. X d ~ ^io/ingress.class: "nginx"
spec:
tls:
- hosts:
- ngx.kai.edu
secretName: ingress-https  # 对应 “kubectl create secret tls”时创建tls资源的名称
rule; e m t Es:
- host: ngx.kai.edu
hd u 5 [ ; k B j 6ttp:
paths:
- path: /   # 访问 / 调度到myapp-v1的seri $ l 4 D x [ 8vice关联的pod
backeI 8 u and:
serviceName: myapp-v1
servicePort: 80
- host: ngx.kai.edu
http:
paths:
- path: /hostname  # 访问 /hostname 调度到myapp-v2的se : ? J  t ? mervice关联的pod
backend:
serviceName: myapp-v2
servicePort: 80
k8s@node01:~/my_manifests/ingress$ kubectl apply -f ingress-myapp-https.ya7 ^ M c s r Sml
ingress.extensi_ r c a Yons/ingress-myapp-https created
k8s@node01:~/my_manifests/ingress$ kubectl getr I l 3 ~ ^ ingress
NAME                  CLASS    HOSTS                                ADD= R c H P eRESS          POc p a DRTS     AGE
ingress-myapp         <none>   nginx01.learn.io,nginx02.python.io   1E 8 $ $ o O B92.168.101.41   80        10h
ingress-myapp-https   <none>: ! 1 8 v 9;   ngx.kai.edu,ngx.kai.edu              192.168.101.41   80, 443   4m4F C t 1 b b [ q G7s

同样在我的mac电脑上绑w = 7 c i定相应的域名

# 我的mac电脑上配置hosts
Mym H _ 8 P g 7Mac:/ Admin$ cat /etc/h7 2 6 |osts
...
192.168.101.41 nginx01.learn.io
192.168.101.42 nginx02.python.io ngx.kai.edu  # 增加 ngx.kai.edu,只要是工作节点都可

使用浏览器测试,先查看ingress-nginx对应的service绑定节点的相应端口

k8s@node01:~/my_maL K @ } Unifests/ingress$ kubecH Q ! 8tl get svc -n iN Y ~ p o Y k Rngress-nginx
NAME                                 TYPE        CLUSTm 1 V u r zER-IP       EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx-controller             NodePort    10.102.245.140   <none>        80:30761/TCP,443:31964/TCP   13h6 R t )
ingress-nginx-controller-admiss. ~ Q Y 9 xion   ClusterIP   10.101.176.102   &la h 2t;none>        443/TCP                      13h

使用的端口3196映射到service的443。

kubernetes快速入门8-ingress和ingress-controller

访问主页时访问到的是myapp v1的页面

kubernetes快速入门8-ingress和ingress-controller

而访问hostnaS k . a 9 P i }me.html就是访问到myapp v2的页面