k8s集群搭建DNS服务

ゝ一世哀愁。 2022-12-04 01:16 461阅读 0赞

## 一、k8s集群中DNS服务介绍  ##

作为服务发现机制的基本功能，在集群内需要能够通过服务名对服务进行访问，这就需要一个集群范围的DNS服务来完成服务名到ClusterIP的解析。

DNS在k8s集群中发展的三个阶段

**1. k8s版本 < 1.4 ，DNS由SkyDNS提供**

SkyDNS有4个容器组成

kube2sky: 监控k8s集群中Service资源的变化

skydns: 从etcd中读取DNS记录，并为客户端容器应用提供DNS查询服务

healthz: 提供对skydns服务的健康检查功能

etcd:  用于保存DNS记录

skydns架构图如下：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70][]

**2. k8s版本 1.4~1.11**

从k8s1.4版本以后SkyDNS组件便被KubeDNS替换,KubeDNS去掉了etcd直接保存在内存中，以提高查询性能。

KubeDNS由3个容器组成

kubedns: 监控k8s集群中Service资源的变化，根据Service的名称和IP地址生成DNS记录

dnsmasq：从kubedns中获取DNS记录，提供DNS缓存，为客户端容器应用提供DNS查询服务

sidecar: 提供对kubedns和dnsmasq服务的健康检查功能。

KubeDNS架构图

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70 1][]

**3. k8s版本大于1.11以后(至今1.19)**

从k8s1.11版本开始，DNS服务由CoreDNS提供。CoreDNS是CNCF基金会的一个项目，是用Go语言实现的高性能、插件式、易扩展的DNS服务器。

CoreDNS只使用了一个容器coredns

架构图如下：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70 2][]

## 二、CoreDNS服务搭建 ##

**1.在创建DNS服务之前修改每个Node上kubelet的启动参数**

修改每个kubelet的启动参数，加上以下两个参数：

\--cluster-dns=clusterIP  \#clusterIP是具体配置的k8s可用的ip地址

\--cluster-domain=cluster.local \#配置DNS服务中设置的域名

修改启动参数可在：10-kubeadm.conf 或在kubelet其他加载的配置文件中添加配置(比如1.18的, /etc/sysconfig/kubelet)

配置完之后重启kubelet

**2.搭建DNS服务**

项目源码：[https://github.com/coredns/coredns][https_github.com_coredns_coredns]

官网中文介绍：[https://kubernetes.io/zh/docs/tasks/administer-cluster/dns-custom-nameservers/][https_kubernetes.io_zh_docs_tasks_administer-cluster_dns-custom-nameservers]

对应的yaml文件：[https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/coredns/coredns.yaml.base][https_github.com_kubernetes_kubernetes_blob_master_cluster_addons_dns_coredns_coredns.yaml.base]

将coredns对应的yaml文件下载，需要将配置文件中的大写属性进行修改

如： \_\_PILLAR\_\_DNS\_\_DOMAIN\_\_ 改成刚才配置的--cluster-domain的值

\_\_PILLAR\_\_DNS\_\_MEMORY\_\_LIMIT\_\_、\_\_PILLAR\_\_DNS\_\_SERVER\_\_

如果因为防火墙的问题拉取不到image: k8s.gcr.io/coredns:1.7.0，建议将coredns的镜像配置修改，

coredns的参考yaml文件(coredns.yaml)：

# __MACHINE_GENERATED_WARNING__
    
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: coredns
      namespace: kube-system
      labels:
          kubernetes.io/cluster-service: "true"
          addonmanager.kubernetes.io/mode: Reconcile
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      labels:
        kubernetes.io/bootstrapping: rbac-defaults
        addonmanager.kubernetes.io/mode: Reconcile
      name: system:coredns
    rules:
    - apiGroups:
      - ""
      resources:
      - endpoints
      - services
      - pods
      - namespaces
      verbs:
      - list
      - watch
    - apiGroups:
      - ""
      resources:
      - nodes
      verbs:
      - get
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      annotations:
        rbac.authorization.kubernetes.io/autoupdate: "true"
      labels:
        kubernetes.io/bootstrapping: rbac-defaults
        addonmanager.kubernetes.io/mode: EnsureExists
      name: system:coredns
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: system:coredns
    subjects:
    - kind: ServiceAccount
      name: coredns
      namespace: kube-system
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: coredns
      namespace: kube-system
      labels:
          addonmanager.kubernetes.io/mode: EnsureExists
    data:
      Corefile: |
        .:53 {
            errors
            health {
                lameduck 5s
            }
            ready
            kubernetes cluster.local in-addr.arpa ip6.arpa {
                pods insecure
                fallthrough in-addr.arpa ip6.arpa
                ttl 30
            }
            prometheus :9153
            forward . /etc/resolv.conf {
                max_concurrent 1000
            }
            cache 30
            loop
            reload
            loadbalance
        }
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: coredns
      namespace: kube-system
      labels:
        k8s-app: kube-dns
        kubernetes.io/cluster-service: "true"
        addonmanager.kubernetes.io/mode: Reconcile
        kubernetes.io/name: "CoreDNS"
    spec:
      # replicas: not specified here:
      # 1. In order to make Addon Manager do not reconcile this replicas parameter.
      # 2. Default is 1.
      # 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
      strategy:
        type: RollingUpdate
        rollingUpdate:
          maxUnavailable: 1
      selector:
        matchLabels:
          k8s-app: kube-dns
      template:
        metadata:
          labels:
            k8s-app: kube-dns
          annotations:
            seccomp.security.alpha.kubernetes.io/pod: 'runtime/default'
        spec:
          priorityClassName: system-cluster-critical
          serviceAccountName: coredns
          affinity:
            podAntiAffinity:
              preferredDuringSchedulingIgnoredDuringExecution:
              - weight: 100
                podAffinityTerm:
                  labelSelector:
                    matchExpressions:
                      - key: k8s-app
                        operator: In
                        values: ["kube-dns"]
                  topologyKey: kubernetes.io/hostname
          tolerations:
            - key: "CriticalAddonsOnly"
              operator: "Exists"
          nodeSelector:
            kubernetes.io/os: linux
          containers:
          - name: coredns
            image: coredns/coredns:1.7.0
            imagePullPolicy: IfNotPresent
            resources:
              limits:
                memory: 170Mi
              requests:
                cpu: 100m
                memory: 70Mi
            args: [ "-conf", "/etc/coredns/Corefile" ]
            volumeMounts:
            - name: config-volume
              mountPath: /etc/coredns
              readOnly: true
            ports:
            - containerPort: 53
              name: dns
              protocol: UDP
            - containerPort: 53
              name: dns-tcp
              protocol: TCP
            - containerPort: 9153
              name: metrics
              protocol: TCP
            livenessProbe:
              httpGet:
                path: /health
                port: 8080
                scheme: HTTP
              initialDelaySeconds: 60
              timeoutSeconds: 5
              successThreshold: 1
              failureThreshold: 5
            readinessProbe:
              httpGet:
                path: /ready
                port: 8181
                scheme: HTTP
            securityContext:
              allowPrivilegeEscalation: false
              capabilities:
                add:
                - NET_BIND_SERVICE
                drop:
                - all
              readOnlyRootFilesystem: true
          dnsPolicy: Default
          volumes:
            - name: config-volume
              configMap:
                name: coredns
                items:
                - key: Corefile
                  path: Corefile
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: kube-dns
      namespace: kube-system
      annotations:
        prometheus.io/port: "9153"
        prometheus.io/scrape: "true"
      labels:
        k8s-app: kube-dns
        kubernetes.io/cluster-service: "true"
        addonmanager.kubernetes.io/mode: Reconcile
        kubernetes.io/name: "CoreDNS"
    spec:
      selector:
        k8s-app: kube-dns
      clusterIP: 10.96.0.100
      ports:
      - name: dns
        port: 53
        protocol: UDP
      - name: dns-tcp
        port: 53
        protocol: TCP
      - name: metrics
        port: 9153
        protocol: TCP

$  kubectl create -f coredns.yaml

创建CoreDNS服务

$ kubectl get deployment --namespace=kube-system

查看对应的coredns Deployment是否正常启动

$ kubectl get pods --namespace=kube-system

查看对应的coredns Pod是否正常启动

$  kubectl get services --namespace=kube-system

查看对应的core-dns service是否正常启动

**创建busybox的pod来验证dns功能**

文件名：busybox.yaml

apiVersion: v1
    kind: Pod
    metadata:
     name: busybox
     namespace: default
    spec:
     containers:
     - name: busybox
       image: busybox:1.28.4
       imagePullPolicy: IfNotPresent
       command:
       - sleep
       - "3600"

创建该busybox的Pod

$  kubectl create busybox.yaml

验证dns服务是否正常

$ kubectl exec busybox -- nslookup redis-master

![20200904181110366.png][]

看到系统可以根据服务名进行解析

问题与解决：

如果执行以上指令出现了dns can't resolve kubernetes.default and/or cluster.local错误

[https://github.com/kubernetes/kubernetes/issues/66924][https_github.com_kubernetes_kubernetes_issues_66924]

则很可能是因为busybox的镜像版本导致的，busybox不能使用最新版本，要使用1.28.4及以下版本。

![20200904181305436.png][]

best wish!

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70]: /images/20221123/724cac9330184136995727b2167c333c.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70 1]: /images/20221123/da75a4082ed140fabfba348f4fbc4191.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3NraDIwMTVqYXZh_size_16_color_FFFFFF_t_70 2]: /images/20221123/a6ac75c596ed41b59265f8a80ec35291.png
[https_github.com_coredns_coredns]: https://github.com/coredns/coredns
[https_kubernetes.io_zh_docs_tasks_administer-cluster_dns-custom-nameservers]: https://kubernetes.io/zh/docs/tasks/administer-cluster/dns-custom-nameservers/
[https_github.com_kubernetes_kubernetes_blob_master_cluster_addons_dns_coredns_coredns.yaml.base]: https://github.com/kubernetes/kubernetes/blob/master/cluster/addons/dns/coredns/coredns.yaml.base
[20200904181110366.png]: /images/20221123/24f4cc58ee724816a22e8f96193dcd0c.png
[https_github.com_kubernetes_kubernetes_issues_66924]: https://github.com/kubernetes/kubernetes/issues/66924
[20200904181305436.png]: /images/20221123/7f6313346b03461393f88ae8e0ed141e.png