k8s etcd-2 Unhealthy HTTP probe failed with statuscode: 503

今天药忘吃喽~ 2024-04-17 06:01 45阅读 0赞

\[root@vlin121 bin\]\# kubectl get componentstatuses --server=192.168.133.121:8080      k8s验证错误  
NAME                 STATUS      MESSAGE                                  ERROR  
scheduler            Healthy     ok                                         
controller-manager   Healthy     ok                                         
etcd-2               Unhealthy   HTTP probe failed with statuscode: 503     
etcd-0               Healthy     \{"health":"true"\}                          
etcd-1               Healthy     \{"health":"true"\}

cat /var/log/messge

Aug 25 18:37:21 vlin121 etcd: health check for peer 5e2c50125f89e6fc could not connect: dial tcp 1  
92.168.133.122:2380: connect: connection refused (prober "ROUND\_TRIPPER\_SNAPSHOT")Aug 25 18:37:21 vlin121 etcd: health check for peer ad0d52b54b7da12 could not connect: dial tcp 19  
2.168.133.123:2380: connect: connection refused (prober "ROUND\_TRIPPER\_RAFT\_MESSAGE")Aug 25 18:37:21 vlin121 etcd: health check for peer 5e2c50125f89e6fc could not connect: dial tcp 1  
92.168.133.122:2380: connect: connection refused (prober "ROUND\_TRIPPER\_RAFT\_MESSAGE")Aug 25 18:37:21 vlin121 etcd: health check for peer ad0d52b54b7da12 could not connect: dial tcp 19  
2.168.133.123:2380: connect: connection refused (prober "ROUND\_TRIPPER\_SNAPSHOT")Aug 25 18:37:22 vlin121 etcd: e8c545a5e7757ebe \[logterm: 250, index: 318, vote: 5e2c50125f89e6fc\]   
ignored MsgVote from ad0d52b54b7da12 \[logterm: 249, index: 14\] at term 250: lease is not expired (remaining ticks: 10)

\[root@vlin121 bin\]\# etcdctl --endpoints=http://192.168.133.121:2379 cluster-health      验证etcd有个节点出现错误  
member ad0d52b54b7da12 is unhealthy: got unhealthy result from http://192.168.133.123:2379  
member 5e2c50125f89e6fc is healthy: got healthy result from http://192.168.133.122:2379  
member e8c545a5e7757ebe is healthy: got healthy result from http://192.168.133.121:2379

\[root@vlin121 bin\]\# scp /etc/etcd/etcd.conf 192.168.133.123:/etc/etcd/

\[root@vlin123 docker\]\# vim /etc/etcd/etcd.conf                上面四处 加主机名 改本机的

ETCD\_DATA\_DIR="/var/lib/etcd/default.etcd"  
\#ETCD\_WAL\_DIR=""  
ETCD\_LISTEN\_PEER\_URLS="http://192.168.133.123:2380"  
ETCD\_LISTEN\_CLIENT\_URLS="http://192.168.133.123:2379"  
\#ETCD\_MAX\_SNAPSHOTS="5"  
\#ETCD\_MAX\_WALS="5"  
ETCD\_NAME="vlin123.ywz.com"  
\#ETCD\_SNAPSHOT\_COUNT="100000"  
\#ETCD\_HEARTBEAT\_INTERVAL="100"  
\#ETCD\_ELECTION\_TIMEOUT="1000"  
\#ETCD\_QUOTA\_BACKEND\_BYTES="0"  
\#ETCD\_MAX\_REQUEST\_BYTES="1572864"  
\#ETCD\_GRPC\_KEEPALIVE\_MIN\_TIME="5s"  
\#ETCD\_GRPC\_KEEPALIVE\_INTERVAL="2h0m0s"  
\#ETCD\_GRPC\_KEEPALIVE\_TIMEOUT="20s"  
\#  
\#\[Clustering\]  
ETCD\_INITIAL\_ADVERTISE\_PEER\_URLS="http://192.168.133.123:2380"  
ETCD\_ADVERTISE\_CLIENT\_URLS="http://192.168.133.123:2379"  
\#ETCD\_DISCOVERY=""  
\#ETCD\_DISCOVERY\_FALLBACK="proxy"  
\#ETCD\_DISCOVERY\_PROXY=""  
\#ETCD\_DISCOVERY\_SRV=""  
ETCD\_INITIAL\_CLUSTER="vlin121.ywz.com=http://192.168.133.121:2380,vlin122.ywz.com=http://192.168.133.122:238  
0,vlin123.ywz.com=http://192.168.133.123:2380"ETCD\_INITIAL\_CLUSTER\_TOKEN="etcd-cluster"  
ETCD\_INITIAL\_CLUSTER\_STATE="new"

\[root@vlin123 docker\]\# systemctl restart etcd.service ; systemctl status etcd.service

\[root@vlin121 bin\]\# etcdctl --endpoints=http://192.168.133.121:2379 cluster-health  
member ad0d52b54b7da12 is healthy: got healthy result from http://192.168.133.123:2379  
member 5e2c50125f89e6fc is healthy: got healthy result from http://192.168.133.122:2379  
member e8c545a5e7757ebe is healthy: got healthy result from http://192.168.133.121:2379

至此 全好了

\[root@vlin121 bin\]\# kubectl get componentstatuses --server=192.168.133.121:8080  
NAME                 STATUS    MESSAGE             ERROR  
controller-manager   Healthy   ok                    
scheduler            Healthy   ok                    
etcd-1               Healthy   \{"health":"true"\}     
etcd-2               Healthy   \{"health":"true"\}     
etcd-0               Healthy   \{"health":"true"\}