Kubernetes 实战教学，手把手教您用 Helm 在 K8s 平台上部署 Prometheus

忘是亡心i 2022-02-12 23:25 364阅读 0赞

![screenshot][]

出品丨Docker公司（ID：docker-cn）  
编译丨小东  
每周一、三、五，与您不见不散！

--------------------

随着单片系统变得越来越庞大以至于无法处理，许多企业开始将其分解为微服务体系结构。无论何时我们从单片架构转向微服务架构，应用程序都包含多个组件，这些组件之间的服务需要进行相互通信。每个组件都有自己的资源，可以单独扩展。如果我们谈论 Kubernetes，它会变得非常复杂，除了需要管理版本数量之外，还需要处理所有对象，例如 ConfigMaps、services、pods 以及 Persistent Volumes。 可能会出现以下挑战：

*  管理、编辑和更新多个 k8s 配置；
 *  将多个 K8s 配置部署为单个应用程序；
 *  共享和重用 K8s 配置和应用程序；
 *  参数化并支持多个环境；
 *  管理应用程序版本：rollout、rollback、diff、history；
 *  定义部署生命周期（将在不同阶段运行的控制操作）；
 *  部署后验证发布状态；

这些都可以通过 Kubernetes Helm 来管理，它提供了一种简单的方法来将所有东西打包到一个简单的应用程序中，并突出显示您可以配置的内容。

--------------------

Helm 是 Kubernetes 的部署管理（不仅仅是包管理器）。它对可重复部署、依赖项管理(重用和共享)、多配置管理、更新、回滚和测试应用程序部署（版本）等方面做了大量工作。

今天，我们将在 Play with Kubernetes 平台上尝试使用 Helm。

首先，打开 [https://labs.play-with-k8s.com/][https_labs.play-with-k8s.com] 访问 Kubernetes Playground。

![screenshot][screenshot 1]

点击“Login”按钮以使用 Docker Hub 或 GitHub ID 进行身份验证。

![screenshot][screenshot 2]

一旦开始会话，您将拥有一个自己的实验室环境。

--------------------

### 添加第一个 Kubernetes 节点 ###

![screenshot][screenshot 3]

单击左侧的“Add New Instance” 来建您的第一个 Kubernetes 集群节点。它会自动将其命名为“node1”。每个实例都预装了Docker Community Edition（CE）和Kubeadm。该节点将被视为我们群集的主节点。

--------------------

### 引导主节点 ###

您可以通过使用以下脚本初始化主节点（node1）来引导 Kubernetes 集群。将此脚本内容复制到 bootstrap.sh 文件中，并使用“chmod + x bootstrap.sh”命令使其可执行。

当您执行此脚本时，作为初始化的一部分，kubeadm 将会编写所需的几个配置文件，设置 RBAC 并部署Kubernetes控制平面组件（如 kube-apiserver、kube-dns、kube-proxy 和 etcd等等）。控制平面组件以 Docker 容器的形式进行部署。

![screenshot][screenshot 4]

复制上面的 kubeadm 连接令牌命令，并将其保存以供下一步使用。此命令将用于将其他节点连接到集群。

--------------------

### 添加工作节点 ###

![screenshot][screenshot 5]

点击“Add New Node”来添加一个新的工作节点。

--------------------

### 检查集群状态 ###

[node1 ~]$ kubectl get nodes
    NAME     STATUS    ROLES    AGE      VERSION
    node1    Ready     master   18m      v1.11.3
    node2    Ready        4m       v1.11.3
    node3    Ready        39s      v1.11.3
    node4    NotReady     22s      v1.11.3
    node5    NotReady     4s       v1.11.3
    [node1 ~]$
    [node1 ]$ kubectl get po
    No resources found.
    [node1 ]$ kubectl get svc
    NAME        TYPE       CLUSTER-IP  EXTERNAL-IP  PORT(S)  AGE
    kubernetes  ClusterIP  10.96.0.1          443/TCP  1h
    [node1]$

--------------------

### 验证正在运行的 Pod ###

[node1 ~]$ kubectl get nodes -o json |
    >      jq ".items[] | {name:.metadata.name} + .status.capacity"
     
    {
     "name": "node1",
     "cpu": "8",
     "ephemeral-storage": "10Gi",
     "hugepages-1Gi": "0",
     "hugepages-2Mi": "0",
     "memory": "32929612Ki",
     "pods": "110"
    }
    {
     "name": "node2",
     "cpu": "8",
     "ephemeral-storage": "10Gi",
     "hugepages-1Gi": "0",
     "hugepages-2Mi": "0",
     "memory": "32929612Ki",
     "pods": "110"
    }
    {
     "name": "node3",
     "cpu": "8",
     "ephemeral-storage": "10Gi",
     "hugepages-1Gi": "0",
     "hugepages-2Mi": "0",
     "memory": "32929612Ki",
     "pods": "110"
    }
    {
     "name": "node4",
     "cpu": "8",
     "ephemeral-storage": "10Gi",
     "hugepages-1Gi": "0",
     "hugepages-2Mi": "0",
     "memory": "32929612Ki",
     "pods": "110"
    }
    {
     "name": "node5",
     "cpu": "8",
     "ephemeral-storage": "10Gi",
     "hugepages-1Gi": "0",
     "hugepages-2Mi": "0",
     "memory": "32929612Ki",
     "pods": "110"
    }

--------------------

### 安装 OpenSSL ###

[node1 ~]$ yum install -y openssl

--------------------

### 安装 Helm ###

$ url https://raw.githubusercontent.com/helm/helm/master/scripts/get > get_helm.sh
    $ chmod 700 get_helm.sh
    $ ./get_helm.sh
    [node1 ~]$ sh get_helm.sh
    Downloading https://kubernetes-helm.storage.googleapis.com/helm-v2.11.0-linux-amd64.tar.gz
    Preparing to install helm and tiller into /usr/local/bin
    helm installed into /usr/local/bin/helm
    tiller installed into /usr/local/bin/tiller
    get_helm.sh: line 177: which: command not found
    Run 'helm init' to configure helm.
    [node1 ~]$ helm init
    Creating /root/.helm
    Creating /root/.helm/repository
    Creating /root/.helm/repository/cache
    Creating /root/.helm/repository/local
    Creating /root/.helm/plugins
    Creating /root/.helm/starters
    Creating /root/.helm/cache/archive
    Creating /root/.helm/repository/repositories.yaml
    Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
    Adding local repo with URL: http://127.0.0.1:8879/charts
    $HELM_HOME has been configured at /root/.helm.
     
    Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
     
    Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
    To prevent this, run `helm init` with the --tiller-tls-verify flag.
    For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
    Happy Helming

--------------------

### 安装 Prometheus ###

让我们尝试使用 Helm 在拥有 5 个节点 K8s 群集上安装 Prometheus Stack。

首先，可以使用“helm search ”选项搜索应用程序堆栈。

[node1 ~]$ helm search prometheus
    NAME                                   CHART VERSION  APP VERSION    DESCRIPTION
    stable/prometheus                      7.3.4          2.4.3          Prometheus is a monitoring system and time series database.
    stable/prometheus-adapter              v0.2.0         v0.2.1         A Helm chart for k8s prometheus adapter
    stable/prometheus-blackbox-exporter    0.1.3          0.12.0         Prometheus Blackbox Exporter
    stable/prometheus-cloudwatch-exporter  0.2.1          0.5.0          A Helm chart for prometheus cloudwatch-exporter
    stable/prometheus-couchdb-exporter     0.1.0          1.0            A Helm chart to export the metrics from couchdb in Promet...
    stable/prometheus-mysql-exporter       0.2.1          v0.11.0        A Helm chart for prometheus mysql exporter with cloudsqlp...
    stable/prometheus-node-exporter        0.5.0          0.16.0         A Helm chart for prometheus node-exporter
    stable/prometheus-operator             0.1.7          0.24.0         Provides easy monitoring definitions for Kubernetes servi...
    stable/prometheus-postgres-exporter    0.5.0          0.4.6          A Helm chart for prometheus postgres-exporter
    stable/prometheus-pushgateway          0.1.3          0.6.0          A Helm chart for prometheus pushgateway
    stable/prometheus-rabbitmq-exporter    0.1.4          v0.28.0        Rabbitmq metrics exporter for prometheus
    stable/prometheus-redis-exporter       0.3.2          0.21.1         Prometheus exporter for Redis metrics
    stable/prometheus-to-sd                0.1.1          0.2.2          Scrape metrics stored in prometheus format and push them ...
    stable/elasticsearch-exporter          0.4.0          1.0.2          Elasticsearch stats exporter for Prometheus
    stable/karma                           1.1.2          v0.14          A Helm chart for Karma - an UI for Prometheus Alertmanager
    stable/stackdriver-exporter            0.0.4          0.5.1          Stackdriver exporter for Prometheus
    stable/weave-cloud                     0.3.0          1.1.0          Weave Cloud is a add-on to Kubernetes which provides Cont...
    stable/kube-state-metrics              0.9.0          1.4.0          Install kube-state-metrics to generate and expose cluster...
    stable/mariadb                         5.2.2          10.1.36        Fast, reliable, scalable, and easy to use open-source rel...
    [node1 ~]$

--------------------

### 更新镜像仓库 ###

[node1 ~]$ helm repo update
    Hang tight while we grab the latest from your chart repositories...
    ...Skip local chart repository
    ...Successfully got an update from the "stable" chart repository
    Update Complete. ⎈ Happy Helming!⎈

--------------------

### 安装 Prometheus ###

$helm install stable/prometheus

出现错误提示：“default” is forbidden: User “system:serviceaccount:kube-system:default” cannot get namespaces in the namespace “default”

--------------------

### 如何修复？ ###

要解决该问题，您需要按照以下步骤操作：

kubectl --namespace kube-system create serviceaccount tiller
    kubectl create clusterrolebinding tiller --clusterrole cluster-admin --serviceaccount=kube-system:tiller
    helm init --service-account tiller --upgrade

--------------------

### Helm 清单 ###

[node1 ~]$ helm list
    NAME           REVISION       UPDATED                        STATUS         CHART                  APP VERSION    NAMESPACE
    excited-elk    1              Sun Oct 28 10:00:02 2018       DEPLOYED       prometheus-7.3.4       2.4.3          default
    [node1 ~]$ helm repo update
    Hang tight while we grab the latest from your chart repositories...
    ...Skip local chart repository
    ...Successfully got an update from the "stable" chart repository
    Update Complete. ⎈ Happy Helming!⎈
    [node1 ~]$ helm install stable/prometheus
    NAME:  excited-elk
    LAST DEPLOYED: Sun Oct 28 10:00:02 2018
    NAMESPACE: default
    STATUS: DEPLOYED
     
    RESOURCES:
    ==> v1beta1/DaemonSet
    NAME                                 AGE
    excited-elk-prometheus-node-exporter 1s
     
    ==> v1/Pod(related)
     
    NAME                                                       READY STATUS            RESTARTS AGE
    excited-elk-prometheus-node-exporter-7bjqc                 0/1   ContainerCreating 0        1s
    excited-elk-prometheus-node-exporter-gbcd7                 0/1   ContainerCreating 0        1s
    excited-elk-prometheus-node-exporter-tk56q                 0/1   ContainerCreating 0        1s
    excited-elk-prometheus-node-exporter-tkk9b                 0/1   ContainerCreating 0        1s
    excited-elk-prometheus-alertmanager-68f4f57c97-wrfjz       0/2   Pending           0        1s
    excited-elk-prometheus-kube-state-metrics-858d44dfdc-vt4wj 0/1   ContainerCreating 0        1s
    excited-elk-prometheus-pushgateway-58bfd54d6d-m4n69        0/1   ContainerCreating 0        1s
    excited-elk-prometheus-server-5958586794-b97xn             0/2   Pending           0        1s
     
    ==> v1/ConfigMap
     
    NAME                                AGE
    excited-elk-prometheus-alertmanager 1s
    excited-elk-prometheus-server       1s
     
    ==> v1/ServiceAccount
    excited-elk-prometheus-alertmanager       1s
    excited-elk-prometheus-kube-state-metrics 1s
    excited-elk-prometheus-node-exporter      1s
    excited-elk-prometheus-pushgateway        1s
    excited-elk-prometheus-server             1s
     
    ==> v1beta1/ClusterRole
    excited-elk-prometheus-kube-state-metrics 1s
    excited-elk-prometheus-server             1s
     
    ==> v1beta1/Deployment
    excited-elk-prometheus-alertmanager       1s
    excited-elk-prometheus-kube-state-metrics 1s
    excited-elk-prometheus-pushgateway        1s
    excited-elk-prometheus-server             1s
     
    ==> v1/PersistentVolumeClaim
    excited-elk-prometheus-alertmanager 1s
    excited-elk-prometheus-server       1s
     
    ==> v1beta1/ClusterRoleBinding
    excited-elk-prometheus-kube-state-metrics 1s
    excited-elk-prometheus-server             1s
     
    ==> v1/Service
    excited-elk-prometheus-alertmanager       1s
    excited-elk-prometheus-kube-state-metrics 1s
    excited-elk-prometheus-node-exporter      1s
    excited-elk-prometheus-pushgateway        1s
    excited-elk-prometheus-server             1s
    NOTES:
    The Prometheus server can be accessed via port 80 on the following DNS name from within your cluster:
    excited-elk-prometheus-server.default.svc.cluster.local
     
     
    Get the Prometheus server URL by running these commands in the same shell:
     export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}")
     kubectl --namespace default port-forward $POD_NAME 9090
     
     
    The Prometheus alertmanager can be accessed via port 80 on the following DNS name from within your cluster:
    excited-elk-prometheus-alertmanager.default.svc.cluster.local
     
     
    Get the Alertmanager URL by running these commands in the same shell:
     export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=alertmanager" -o jsonpath="{.items[0].metadata.name}")
     kubectl --namespace default port-forward $POD_NAME 9093
     
     
    The Prometheus PushGateway can be accessed via port 9091 on the following DNS name from within your cluster:
    excited-elk-prometheus-pushgateway.default.svc.cluster.local
     
     
    Get the PushGateway URL by running these commands in the same shell:
     export POD_NAME=$(kubectl get pods --namespace default -l "app=prometheus,component=pushgateway" -o jsonpath="{.items[0].metadata.name}")
     kubectl --namespace default port-forward $POD_NAME 9091
     
    For more information on running Prometheus, visit:
    https://prometheus.io/
    [node1 ~]$ kubectl get all
    NAME                                                            READY    STATUS   RESTARTS  AGE
    pod/excited-elk-prometheus-alertmanager-68f4f57c97-wrfjz        0/2      Pending  0         3m
    pod/excited-elk-prometheus-kube-state-metrics-858d44dfdc-vt4wj  1/1      Running  0         3m
    pod/excited-elk-prometheus-node-exporter-7bjqc                  1/1      Running  0         3m
    pod/excited-elk-prometheus-node-exporter-gbcd7                  1/1      Running  0         3m
    pod/excited-elk-prometheus-node-exporter-tk56q                  1/1      Running  0         3m
    pod/excited-elk-prometheus-node-exporter-tkk9b                  1/1      Running  0         3m
    pod/excited-elk-prometheus-pushgateway-58bfd54d6d-m4n69         1/1      Running  0         3m
    pod/excited-elk-prometheus-server-5958586794-b97xn              0/2      Pending  0         3m
     
    NAME                                               TYPE       CLUSTER-IP     EXTERNAL-IP  PORT(S)   AGE
    service/excited-elk-prometheus-alertmanager        ClusterIP  10.106.159.46         80/TCP    3m
    service/excited-elk-prometheus-kube-state-metrics  ClusterIP  None                  80/TCP    3m
    service/excited-elk-prometheus-node-exporter       ClusterIP  None                  9100/TCP  3m
    service/excited-elk-prometheus-pushgateway         ClusterIP  10.106.88.15          9091/TCP  3m
    service/excited-elk-prometheus-server              ClusterIP  10.107.15.64          80/TCP    3m
    service/kubernetes                                 ClusterIP  10.96.0.1             443/TCP   37m
     
    NAME                                                 DESIRED  CURRENT  READY    UP-TO-DATE  AVAILABLE  NODE SELECTOR  AGE
    daemonset.apps/excited-elk-prometheus-node-exporter  4        4        4        4           4                   3m
     
    NAME                                                       DESIRED  CURRENT  UP-TO-DATE  AVAILABLE  AGE
    deployment.apps/excited-elk-prometheus-alertmanager        1        1        1           0          3m
    deployment.apps/excited-elk-prometheus-kube-state-metrics  1        1        1           1          3m
    deployment.apps/excited-elk-prometheus-pushgateway         1        1        1           1          3m
    deployment.apps/excited-elk-prometheus-server              1        1        1           0          3m
     
    NAME                                                                  DESIRED  CURRENT  READY    AGE
    replicaset.apps/excited-elk-prometheus-alertmanager-68f4f57c97        1        1        0        3m
    replicaset.apps/excited-elk-prometheus-kube-state-metrics-858d44dfdc  1        1        1        3m
    replicaset.apps/excited-elk-prometheus-pushgateway-58bfd54d6d         1        1        1        3m
    replicaset.apps/excited-elk-prometheus-server-5958586794              1        1        0        3m
    [node1 ~]$

等待几分钟，您就可以使用 https://:9090 访问 Prometheus UI了。

--------------------

在下一篇 Kubernetes 实战教学系列文章中，我将为您带来更多关于 Helm on PWD Playground 的有趣内容。

[screenshot]: /images/20220213/6da8b5b7681b438e9a8150a45ba72166.png
[https_labs.play-with-k8s.com]: https://labs.play-with-k8s.com/
[screenshot 1]: /images/20220213/81f30f2fcbaa4bd0a581c7edf4a329cd.png
[screenshot 2]: /images/20220213/3b6159739ffc439f8c09d1884acebc8a.png
[screenshot 3]: /images/20220213/dc0dccc7b5644a5688a2e3527bfdb4e1.png
[screenshot 4]: /images/20220213/dfd2918141a3478186dd435fde8b0d51.png
[screenshot 5]: /images/20220213/40399dc1c7e24ae78ea5e9c5f58dbfd7.png