kubernetes1.20版本启用ipvs模式

Myth丶恋晨 2022-10-14 00:56 208阅读 0赞

在1.19版本之前,kubeadm部署方式启用ipvs模式时,初始化配置文件需要添加以下内容:

---
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    kind: KubeProxyConfiguration
    featureGates:
      SupportIPVSProxyMode: true
    mode: ipvs

本次在1.20.2版本中,使用kubeadm进行集群初始化时,虽然可以正常部署,但是查看pod情况的时候可以看到kube-proxy无法运行成功,报错部分内容如下:

]# kubectl get pod -A
    NAMESPACE NAME READY STATUS RESTARTS AGE
    
    ....#省略其他输出内容
    kube-system kube-proxy-7vrbv 0/1 CrashLoopBackOff 9 43m
    kube-system kube-proxy-ghs7h 0/1 CrashLoopBackOff 9 43m
    kube-system kube-proxy-l9twb 0/1 CrashLoopBackOff 1 7s
    kube-system kube-proxy-mzfrf 0/1 CrashLoopBackOff 9 42m
    kube-system kube-proxy-nxpls 0/1 CrashLoopBackOff 9 3h4m
    kube-system kube-proxy-pmmtq 0/1 CrashLoopBackOff 8 42m

#查看日志信息
    ]# kubectl  logs kube-proxy-l9twb -n kube-system
    F0114 12:58:34.042769       1 server.go:488] failed complete: unrecognized feature gate: SupportIPVSProxyMode
    goroutine 1 [running]:
    k8s.io/kubernetes/vendor/k8s.io/klog/v2.stacks(0xc00000e001, 0xc0004b6000, 0x6e, 0xc0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:1026 +0xb9
    k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).output(0x29b65c0, 0xc000000003, 0x0, 0x0, 0xc0003d8230, 0x28edbc9, 0x9, 0x1e8, 0x0)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:975 +0x19b
    k8s.io/kubernetes/vendor/k8s.io/klog/v2.(*loggingT).printf(0x29b65c0, 0xc000000003, 0x0, 0x0, 0x0, 0x0, 0x1b3a573, 0x13, 0xc000431310, 0x1, ...)
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/k8s.io/klog/v2/klog.go:750 +0x191
    k8s.io/kubernetes/vendor/k8s.io/klog/v2.Fatalf(...)

通过报错可以看到kube-proxy无法识别SupportIPVSProxyMode这个字段,于是访问官方查看最新版本ipvs开启的正确配置,通过[https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md][https_github.com_kubernetes_kubernetes_blob_master_pkg_proxy_ipvs_README.md]可以看到官方说明:

Cluster Created by Kubeadm

If you are using kubeadm with a configuration file, you have to add mode: ipvs below the kubeProxy field as part of the kubeadm configuration.

...
    kubeProxy:
      config:
        mode: ipvs
    ...

由于集群已经初始化成功了,所以现在改kubeadm初始化配置文件没有意义,因为我们需要直接修改kube-proxy的启动配置

通过查看kube-pxory的资源清单可以知道, kube-proxy的配置文件是通过configmap方式挂载到容器中的,因此我们只需要对应修改configmap中的配置内容,就可以将无效字段删除

]# kubectl -n kube-system get pod kube-proxy-24tkb -o yaml
    apiVersion: v1
    kind: Pod
    metadata:
    .....　　#其他内容省略
      containers:
      - command:
        - /usr/local/bin/kube-proxy
        - --config=/var/lib/kube-proxy/config.conf
        - --hostname-override=$(NODE_NAME)
    .....　　#其他内容省略
        volumeMounts:
        - mountPath: /var/lib/kube-proxy
          name: kube-proxy
    .....　　#其他内容省略
      volumes:
      - configMap:
          defaultMode: 420
          name: kube-proxy
        name: kube-proxy

]# kubectl get cm -n kube-system
    NAME                                 DATA   AGE
    coredns                              1      5h18m
    extension-apiserver-authentication   6      5h18m
    kube-proxy                           2      5h18m
    kube-root-ca.crt                     1      5h18m
    kubeadm-config                       2      5h18m
    kubelet-config-1.20                  1      5h18m

]# kubectl edit cm kube-proxy -n kube-system　　　　
    #在编辑模式中找到以下字段,删除后保存退出
    featureGates: 
      SupportIPVSProxyMode: true

然后将删除所有kube-proxy进行重启,查看pod运行情况

]# kubectl get pod -n kube-system
    NAME                                          READY   STATUS    RESTARTS   AGE
    ...　　#其他内容省略
    kube-proxy-24tkb                              1/1     Running   0          122m
    kube-proxy-9cl7j                              1/1     Running   0          123m
    kube-proxy-cxbg5                              1/1     Running   0          123m
    kube-proxy-cxgqk                              1/1     Running   0          123m
    kube-proxy-hkq54                              1/1     Running   0          123m
    kube-proxy-ttdqb                              1/1     Running   0          122m

在服务器上安装ipvsadm,查看ipvs模式是否启用成功

]# yum install ipvsadm -y
    ]# ipvsadm -Ln
    IP Virtual Server version 1.2.1 (size=4096)
    Prot LocalAddress:Port Scheduler Flags
      -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
    TCP  10.96.0.1:443 rr
      -> 和谐ip:6443          Masq    1      0          0         
      -> 和谐ip:6443          Masq    1      0          0         
      -> 和谐ip:6443          Masq    1      0          0         
    TCP  10.96.0.10:53 rr
    TCP  10.96.0.10:9153 rr
    UDP  10.96.0.10:53 rr

[https_github.com_kubernetes_kubernetes_blob_master_pkg_proxy_ipvs_README.md]: https://github.com/kubernetes/kubernetes/blob/master/pkg/proxy/ipvs/README.md