kubernetes pod NodeLost 分析

ゝ一世哀愁。 2022-06-01 14:28 326阅读 0赞

#### 问题描述 ####

[root@master-64 ~]# kubectl get pods -n kube-system -owide| grep -v Running
    NAME                                        READY     STATUS             RESTARTS   AGE       IP                NODE
    calico-node-cnrvm                           2/2       NodeLost           0          48d       10.39.1.44        proxy-44
    filebeat-k7svm                              1/1       NodeLost           0          15d       192.168.159.0     proxy-44
    kube-proxy-oop2g                            1/1       NodeLost           0          62d       10.39.1.44        proxy-44
    kubectl-qlj27                               1/1       NodeLost           0          15d       192.168.159.3     proxy-44
    prometheus-tim-3864503240-ng4h0             0/1       CrashLoopBackOff   485        1d        192.168.88.70     slave-95
    service-proxy-clq2t                         1/1       NodeLost           0          2h        10.39.1.44        proxy-44

[root@master ~]# kubectl describe pod calico-node-cnrvm -n kube-system
    ...
    Node-Selectors: calico=2.5.1
    Tolerations:    node.alpha.kubernetes.io/notReady:NoExecute
            node.alpha.kubernetes.io/unreachable:NoExecute

/proc/2005/net/dev: no such file or directory
    I0119 13:53:02.039754    2029 helpers.go:101] Unable to get network stats from pid 16385: couldn't read network stats: failure opening /proc/16385/net/dev: open /proc/16385/net/dev: no such file or directory
    
    ...
    I0119 13:53:10.273923    2029 reconciler.go:299] MountVolume operation started for volume "kubernetes.io/secret/aee483dc-f6c3-11e7-9d36-5254b24cbf5e-default-token-qu72f" (spec.Name: "default-token-qu72f") to pod "aee483dc-f6c3-11e7-9d36-5254b24cbf5e" (UID: "aee483dc-f6c3-11e7-9d36-5254b24cbf5e"). Volume is already mounted to pod, but remount was requested.
    
    ...
    I0119 13:54:33.846820    2029 conversion.go:133] failed to handle multiple devices for container. Skipping Filesystem stats
    I0119 13:54:33.846949    2029 conversion.go:133] failed to handle multiple devices for container. Skipping Filesystem stats

#### 解决问题 ####

执行重启命令就可以了

docker restart containerID

END