kubebuilder实战之二：初次体验kubebuilder

迈不过友情╰ 2023-01-10 13:23 35阅读 0赞

### 欢迎访问我的GitHub ###

> 这里分类和汇总了欣宸的全部原创(含配套源码)：[https://github.com/zq2599/blog\_demos][https_github.com_zq2599_blog_demos]

### 系列文章链接 ###

1.  [kubebuilder实战之一：准备工作][kubebuilder]
2.  [kubebuilder实战之二：初次体验kubebuilder][kubebuilder_kubebuilder]
3.  [kubebuilder实战之三：基础知识速览][kubebuilder 1]
4.  [kubebuilder实战之四：operator需求说明和设计][kubebuilder_operator]
5.  [kubebuilder实战之五：operator编码][kubebuilder_operator 1]
6.  [kubebuilder实战之六：构建部署运行][kubebuilder 2]
7.  [kubebuilder实战之七：webhook][kubebuilder_webhook]
8.  [kubebuilder实战之八：知识点小记][kubebuilder 3]

### 本篇概览 ###

本文是《kubebuilder实战》系列的第二篇，前文将kubebuilder环境准备完毕，今天咱们在此环境创建CRD和Controller，再部署到kubernetes环境并且验证是否生效，整篇文章由以下内容组成：

1.  创建API(CRD和Controller)
2.  构建和部署CRD
3.  编译和运行controller
4.  创建CRD对应的实例
5.  删除实例并停止controller
6.  将controller制作成docker镜像
7.  卸载和清理

### 创建helloworld项目 ###

1.  执行以下命令，创建helloworld项目：

mkdir -p $GOPATH/src/helloworld
    cd $GOPATH/src/helloworld
    kubebuilder init --domain com.bolingcavalry

1.  控制台输出类似以下内容：

[root@kubebuilder helloworld]# kubebuilder init --domain com.bolingcavalry
    Writing scaffold for you to edit...
    Get controller runtime:
    $ go get sigs.k8s.io/controller-runtime@v0.5.0
    Update go.mod:
    $ go mod tidy
    Running make:
    $ make
    /root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
    go fmt ./...
    go vet ./...
    go build -o bin/manager main.go
    Next: define a resource with:
    $ kubebuilder create api

1.  等待数分钟后创建完成，在$GOPATH/src/helloworld目录下新增以下内容，可见这是个标准的go module工程：

[root@kubebuilder ~]# tree $GOPATH/src/helloworld
    /root/gopath/src/helloworld
    ├── bin
    │   └── manager
    ├── config
    │   ├── certmanager
    │   │   ├── certificate.yaml
    │   │   ├── kustomization.yaml
    │   │   └── kustomizeconfig.yaml
    │   ├── default
    │   │   ├── kustomization.yaml
    │   │   ├── manager_auth_proxy_patch.yaml
    │   │   ├── manager_webhook_patch.yaml
    │   │   └── webhookcainjection_patch.yaml
    │   ├── manager
    │   │   ├── kustomization.yaml
    │   │   └── manager.yaml
    │   ├── prometheus
    │   │   ├── kustomization.yaml
    │   │   └── monitor.yaml
    │   ├── rbac
    │   │   ├── auth_proxy_client_clusterrole.yaml
    │   │   ├── auth_proxy_role_binding.yaml
    │   │   ├── auth_proxy_role.yaml
    │   │   ├── auth_proxy_service.yaml
    │   │   ├── kustomization.yaml
    │   │   ├── leader_election_role_binding.yaml
    │   │   ├── leader_election_role.yaml
    │   │   └── role_binding.yaml
    │   └── webhook
    │       ├── kustomization.yaml
    │       ├── kustomizeconfig.yaml
    │       └── service.yaml
    ├── Dockerfile
    ├── go.mod
    ├── go.sum
    ├── hack
    │   └── boilerplate.go.txt
    ├── main.go
    ├── Makefile
    └── PROJECT
    
    9 directories, 30 files

### 创建API(CRD和Controller) ###

1.  接下来要要创建资源相关的内容了，group/version/kind这三部分可以确定资源的唯一身份，命令如下：

cd $GOPATH/src/helloworld
    kubebuilder create api \
    --group webapp \
    --version v1 \
    --kind Guestbook

1.  控制台会提醒是否创建资源(Create Resource \[y/n\])，输入y
2.  接下来控制台会提醒是否创建控制器(Create Controller \[y/n\])，输入y
3.  kubebuilder会根据上述命令新增多个文件，如下图红框所示：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70]

### 构建和部署CRD ###

1.  kubebuilder提供的Makefile将构建和部署工作大幅度简化，执行以下命令会将最新构建的CRD部署在kubernetes上：

cd $GOPATH/src/helloworld
    make install

1.  控制台输出如下内容，提示部署成功：

[root@kubebuilder helloworld]# make install
    /root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    kustomize build config/crd | kubectl apply -f -
    Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
    customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.com.bolingcavalry created

### 编译和运行controller ###

1.  kubebuilder自动生成的controller源码地址是：$GOPATH/src/helloworld/controllers/guestbook\_controller.go ， 内容如下：

package controllers
    
    import (
    	"context"
    
    	"github.com/go-logr/logr"
    	"k8s.io/apimachinery/pkg/runtime"
    	ctrl "sigs.k8s.io/controller-runtime"
    	"sigs.k8s.io/controller-runtime/pkg/client"
    
    	webappv1 "helloworld/api/v1"
    )
    
    // GuestbookReconciler reconciles a Guestbook object
    type GuestbookReconciler struct { 
    	client.Client
    	Log    logr.Logger
    	Scheme *runtime.Scheme
    }
    
    // +kubebuilder:rbac:groups=webapp.com.bolingcavalry,resources=guestbooks,verbs=get;list;watch;create;update;patch;delete
    // +kubebuilder:rbac:groups=webapp.com.bolingcavalry,resources=guestbooks/status,verbs=get;update;patch
    
    func (r *GuestbookReconciler) Reconcile(req ctrl.Request) (ctrl.Result, error) { 
    	_ = context.Background()
    	_ = r.Log.WithValues("guestbook", req.NamespacedName)
    
    	// your logic here
    
    	return ctrl.Result{ }, nil
    }
    
    func (r *GuestbookReconciler) SetupWithManager(mgr ctrl.Manager) error { 
    	return ctrl.NewControllerManagedBy(mgr).
    		For(&webappv1.Guestbook{ }).
    		Complete(r)
    }

1.  本文以体验基本流程为主，不深入研究源码，所以对上面的代码仅做少量修改，用于验证是否能生效，改动如下图红框所示：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 1]
2.  执行以下命令，会编译并启动刚才修改的controller：

cd $GOPATH/src/helloworld
    make run

1.  此时控制台输出以下内容，这里要注意，controller是在kubebuilder电脑上运行的，一旦使用Ctrl+c中断控制台，就会导致controller停止：

[root@kubebuilder helloworld]# make run
    /root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
    go fmt ./...
    go vet ./...
    /root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    go run ./main.go
    2021-01-23T20:58:35.107+0800	INFO	controller-runtime.metrics	metrics server is starting to listen	{ "addr": ":8080"}
    2021-01-23T20:58:35.108+0800	INFO	setup	starting manager
    2021-01-23T20:58:35.108+0800	INFO	controller-runtime.manager	starting metrics server	{ "path": "/metrics"}
    2021-01-23T20:58:35.108+0800	INFO	controller-runtime.controller	Starting EventSource	{ "controller": "guestbook", "source": "kind source: /, Kind="}
    2021-01-23T20:58:35.208+0800	INFO	controller-runtime.controller	Starting Controller	{ "controller": "guestbook"}
    2021-01-23T20:58:35.209+0800	INFO	controller-runtime.controller	Starting workers	{ "controller": "guestbook", "worker count": 1}

### 创建Guestbook资源的实例 ###

1.  现在kubernetes已经部署了Guestbook类型的CRD，而且对应的controller也已正在运行中，可以尝试创建Guestbook类型的实例了(相当于有了pod的定义后，才可以创建pod)；
2.  kubebuilder已经自动创建了一个类型的部署文件：$GOPATH/src/helloworld/config/samples/webapp\_v1\_guestbook.yaml ，内容如下，很简单，接下来咱们就用这个文件来创建Guestbook实例：

apiVersion: webapp.com.bolingcavalry/v1
    kind: Guestbook
    metadata:
      name: guestbook-sample
    spec:
      # Add fields here
      foo: bar

1.  重新打开一个控制台，登录kubebuilder电脑，执行以下命令即可创建Guestbook类型的实例：

cd $GOPATH/src/helloworld
    kubectl apply -f config/samples/

1.  如下所示，控制台提示资源创建成功：

[root@kubebuilder helloworld]# kubectl apply -f config/samples/
    guestbook.webapp.com.bolingcavalry/guestbook-sample created

1.  用kubectl get命令可以看到实例已经创建：

[root@kubebuilder helloworld]# kubectl get Guestbook
    NAME               AGE
    guestbook-sample   112s

1.  用命令kubectl edit Guestbook guestbook-sample编辑该实例，修改的内容如下图红框所示：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 2]
2.  此时去controller所在控制台，可以看到新增和修改的操作都有日志输出，咱们新增的日志都在里面，代码调用栈一目了然：

2021-01-24T09:51:50.418+0800	INFO	controllers.Guestbook	1. default/guestbook-sample
    2021-01-24T09:51:50.418+0800	INFO	controllers.Guestbook	2. goroutine 188 [running]:
    runtime/debug.Stack(0xc0002a1808, 0xc0002fc600, 0x1b)
    	/root/go/src/runtime/debug/stack.go:24 +0x9f
    helloworld/controllers.(*GuestbookReconciler).Reconcile(0xc0003c9dd0, 0xc0002d02f9, 0x7, 0xc0002d02e0, 0x10, 0x12f449647b, 0xc000456f30, 0xc000456ea8, 0xc000456ea0)
    	/root/gopath/src/helloworld/controllers/guestbook_controller.go:49 +0x1a9
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00022a480, 0x1430e00, 0xc0003e7560, 0x0)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:256 +0x166
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00022a480, 0xc000469600)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232 +0xb0
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00022a480)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211 +0x2b
    k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000292980)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152 +0x5f
    k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000292980, 0x3b9aca00, 0x0, 0x1609101, 0xc000102480)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153 +0x105
    k8s.io/apimachinery/pkg/util/wait.Until(0xc000292980, 0x3b9aca00, 0xc000102480)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88 +0x4d
    created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:193 +0x32d
    
    2021-01-24T09:51:50.418+0800	DEBUG	controller-runtime.controller	Successfully Reconciled	{ "controller": "guestbook", "request": "default/guestbook-sample"}
    
    
    2021-01-24T09:52:33.632+0800	INFO	controllers.Guestbook	1. default/guestbook-sample
    2021-01-24T09:52:33.633+0800	INFO	controllers.Guestbook	2. goroutine 188 [running]:
    runtime/debug.Stack(0xc0002a1808, 0xc0003fa5e0, 0x1b)
    	/root/go/src/runtime/debug/stack.go:24 +0x9f
    helloworld/controllers.(*GuestbookReconciler).Reconcile(0xc0003c9dd0, 0xc0002d02f9, 0x7, 0xc0002d02e0, 0x10, 0x1d0410fe42, 0xc000456f30, 0xc000456ea8, 0xc000456ea0)
    	/root/gopath/src/helloworld/controllers/guestbook_controller.go:49 +0x1a9
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc00022a480, 0x1430e00, 0xc0003d24c0, 0x0)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:256 +0x166
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc00022a480, 0xc000469600)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:232 +0xb0
    sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker(0xc00022a480)
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:211 +0x2b
    k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1(0xc000292980)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152 +0x5f
    k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc000292980, 0x3b9aca00, 0x0, 0x1609101, 0xc000102480)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153 +0x105
    k8s.io/apimachinery/pkg/util/wait.Until(0xc000292980, 0x3b9aca00, 0xc000102480)
    	/root/gopath/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88 +0x4d
    created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func1
    	/root/gopath/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.0/pkg/internal/controller/controller.go:193 +0x32d
    
    2021-01-24T09:52:33.633+0800	DEBUG	controller-runtime.controller	Successfully Reconciled	{ "controller": "guestbook", "request": "default/guestbook-sample"}

### 删除实例并停止controller ###

1.  不再需要Guestbook实例的时候，执行以下命令即可删除：

cd $GOPATH/src/helloworld
    kubectl delete -f config/samples/

1.  不再需要controller的时候，去它的控制台使用Ctrl+c中断即可；

### 将controller制作成docker镜像 ###

1.  至此，咱们已经体验过了kubebuilder的基本功能，不过实际生产环境中controller一般都会运行在kubernetes环境内，像上面这种运行在kubernetes之外的方式就不合适了，咱们来试试将其做成docker镜像然后在kubernetes环境运行；
2.  这里有个要求，就是您要有个kubernetes可以访问的镜像仓库，例如局域网内的Harbor，或者公共的hub.docker.com，我这为了操作方便选择了hub.docker.com，使用它的前提是拥有hub.docker.com的注册帐号；
3.  在kubebuilder电脑上，打开一个控制台，执行docker login命令登录，根据提示输入hub.docker.com的帐号和密码，这样就可以在当前控制台上执行docker push命令将镜像推送到hub.docker.com上了（这个网站的网络很差，可能要登录好几次才能成功）；
4.  执行以下命令构建docker镜像并推送到hub.docker.com，镜像名为bolingcavalry/guestbook:002：

cd $GOPATH/src/helloworld
    make docker-build docker-push IMG=bolingcavalry/guestbook:002

1.  hub.docker.com的网络状况不是一般的差，kubebuilder电脑上的docker一定要设置镜像加速，上述命令如果遭遇超时失败，请重试几次，此外，构建过程中还会下载诸多go模块的依赖，也需要您耐心等待，也很容易遇到网络问题，需要多次重试，所以，最好是使用局域网内搭建的Habor服务；
2.  最终，命令执行成功后输出如下：

[root@kubebuilder helloworld]# make docker-build docker-push IMG=bolingcavalry/guestbook:002
    /root/gopath/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
    go fmt ./...
    go vet ./...
    /root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    go test ./... -coverprofile cover.out
    ?   	helloworld	[no test files]
    ?   	helloworld/api/v1	[no test files]
    ok  	helloworld/controllers	8.604s	coverage: 0.0% of statements
    docker build . -t bolingcavalry/guestbook:002
    Sending build context to Docker daemon  40.27MB
    Step 1/14 : FROM golang:1.13 as builder
     ---> d6f3656320fe
    Step 2/14 : WORKDIR /workspace
     ---> Using cache
     ---> 83d05ead1041
    Step 3/14 : COPY go.mod go.mod
     ---> Using cache
     ---> ae3e15a529f4
    Step 4/14 : COPY go.sum go.sum
     ---> Using cache
     ---> 082223532ccc
    Step 5/14 : RUN go mod download
     ---> Using cache
     ---> bcdcfa1d65ca
    Step 6/14 : COPY main.go main.go
     ---> Using cache
     ---> 81d6a629ca98
    Step 7/14 : COPY api/ api/
     ---> Using cache
     ---> 75f99b174e97
    Step 8/14 : COPY controllers/ controllers/
     ---> b130d9f47903
    Step 9/14 : RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 GO111MODULE=on go build -a -o manager main.go
     ---> Running in 768880aca19f
    Removing intermediate container 768880aca19f
     ---> bb4a494d3b43
    Step 10/14 : FROM gcr.io/distroless/static:nonroot
     ---> 947e6f3ed7c1
    Step 11/14 : WORKDIR /
     ---> Using cache
     ---> 22cc43cef8fb
    Step 12/14 : COPY --from=builder /workspace/manager .
     ---> 2137778f22c0
    Step 13/14 : USER nonroot:nonroot
     ---> Running in 18295673073d
    Removing intermediate container 18295673073d
     ---> f7545379ab1f
    Step 14/14 : ENTRYPOINT ["/manager"]
     ---> Running in 550c47dd61dc
    Removing intermediate container 550c47dd61dc
     ---> 31cb31a6b03f
    Successfully built 31cb31a6b03f
    Successfully tagged bolingcavalry/guestbook:002
    docker push bolingcavalry/guestbook:002
    The push refers to repository [docker.io/bolingcavalry/guestbook]
    99035107a955: Pushed 
    728501c5607d: Layer already exists 
    002: digest: sha256:54f8ec88511cce5b04c5d65cc15e0f7a7b4a8afb6b235904a638bff79e3c5784 size: 739

1.  去hub.docker.com网站看看，如下图，新镜像已经上传，这样只要任何机器只要能上网就能pull此镜像到本地使用了：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 3]
2.  镜像准备好之后，执行以下命令即可在kubernetes环境部署controller：

cd $GOPATH/src/helloworld
    make deploy IMG=bolingcavalry/guestbook:002

1.  控制台会提示各类资源被创建（rbac居多）：

[root@kubebuilder ~]# cd $GOPATH/src/helloworld
    [root@kubebuilder helloworld]# make deploy IMG=bolingcavalry/guestbook:002
    /root/gopath/bin/controller-gen "crd:trivialVersions=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
    cd config/manager && kustomize edit set image controller=bolingcavalry/guestbook:002
    kustomize build config/default | kubectl apply -f -
    namespace/helloworld-system created
    Warning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinition
    customresourcedefinition.apiextensions.k8s.io/guestbooks.webapp.com.bolingcavalry configured
    role.rbac.authorization.k8s.io/helloworld-leader-election-role created
    clusterrole.rbac.authorization.k8s.io/helloworld-manager-role created
    clusterrole.rbac.authorization.k8s.io/helloworld-proxy-role created
    Warning: rbac.authorization.k8s.io/v1beta1 ClusterRole is deprecated in v1.17+, unavailable in v1.22+; use rbac.authorization.k8s.io/v1 ClusterRole
    clusterrole.rbac.authorization.k8s.io/helloworld-metrics-reader created
    rolebinding.rbac.authorization.k8s.io/helloworld-leader-election-rolebinding created
    clusterrolebinding.rbac.authorization.k8s.io/helloworld-manager-rolebinding created
    clusterrolebinding.rbac.authorization.k8s.io/helloworld-proxy-rolebinding created
    service/helloworld-controller-manager-metrics-service created
    deployment.apps/helloworld-controller-manager created

1.  此时去看kubernetes环境的pod，发现确实已经新增了controller，如下图红框：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 4]

11.细心的您应该会发现上图黄框中显示这个pod实际上有两个容器，用kubectl describe命令细看，分别是kube-rbac-proxy和manager，如下图：  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 5]  
11. 由于有两个容器，那么查看日志时就要指定其中一个了，咱们的controller对应的是manager容器，因此查看日志的命令是：

kubectl logs -f \
    helloworld-controller-manager-689d4b6f5b-h9pzg \
    -n helloworld-system \
    -c manager

1.  再次创建Guestbook资源的实例，依旧是kubectl apply -f config/samples/命令，再去看manager容器的日志，可见咱们修改的内容已经打印出来了：  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 6]

### 卸载和清理 ###

*  体验完毕后，如果想把前面创建的资源和CRD全部清理掉，可以执行以下命令：

cd $GOPATH/src/helloworld
    make uninstall

*  至此，通过kubebuilder创建Operator相关资源的基本流程，咱们已经体验过一遍了，本篇以熟悉工具和流程为主，并未体验到Operator实质性的强大功能，这些都留待后面的章节吧，咱们逐步深入学习实践；

### 你不孤单，欣宸原创一路相伴 ###

1.  [Java系列][Java]
2.  [Spring系列][Spring]
3.  [Docker系列][Docker]
4.  [kubernetes系列][kubernetes]
5.  [数据库+中间件系列][Link 1]
6.  [DevOps系列][DevOps]

### 欢迎关注公众号：程序员欣宸 ###

> 微信搜索「程序员欣宸」，我是欣宸，期待与您一同畅游Java世界…

[https_github.com_zq2599_blog_demos]: https://github.com/zq2599/blog_demos
[kubebuilder]: https://xinchen.blog.csdn.net/article/details/113035349
[kubebuilder_kubebuilder]: https://xinchen.blog.csdn.net/article/details/113089414
[kubebuilder 1]: https://xinchen.blog.csdn.net/article/details/113815479
[kubebuilder_operator]: https://xinchen.blog.csdn.net/article/details/113822065
[kubebuilder_operator 1]: https://xinchen.blog.csdn.net/article/details/113836090
[kubebuilder 2]: https://xinchen.blog.csdn.net/article/details/113840999
[kubebuilder_webhook]: https://xinchen.blog.csdn.net/article/details/113922328
[kubebuilder 3]: https://xinchen.blog.csdn.net/article/details/114215218
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70]: /images/20221119/5a570fe959214cd4b1211fe4dd5fa304.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 1]: /images/20221119/eea11f9a0dad4aa896cfa47cf626cb04.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 2]: /images/20221119/abe42ead49eb46da9646009799f07361.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 3]: /images/20221119/96d49653feb24864893eac93d5f147a1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 4]: /images/20221119/0705367d6a064f3cba2921645b8ca3bd.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 5]: /images/20221119/1e0adae372cc4fc9a684e772ff41cbba.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2JvbGluZ19jYXZhbHJ5_size_16_color_FFFFFF_t_70 6]: /images/20221119/ae27e57e110740d4a78e6af0f8450fbf.png
[Java]: https://xinchen.blog.csdn.net/article/details/105068742
[Spring]: https://xinchen.blog.csdn.net/article/details/105086498
[Docker]: https://xinchen.blog.csdn.net/article/details/105086732
[kubernetes]: https://xinchen.blog.csdn.net/article/details/105086794
[Link 1]: https://xinchen.blog.csdn.net/article/details/105086850
[DevOps]: https://xinchen.blog.csdn.net/article/details/105086920