SpringCloud Gateway与k8s

悠悠 2023-02-09 11:47 8阅读 0赞

### 本文大纲 ###

接下来的内容由以下几部分组成：

1.  什么是SpringCloud Gateway
2.  SpringCloud Gateway实战参考
3.  kubernetes上的SpringCloud Gateway
4.  实战环境信息
5.  实战源码下载
6.  开发webdemo
7.  开发k8sgatewaydemo
8.  解决权限问题
9.  最后一个疑问

### 什么是SpringCloud Gateway ###

SpringCloud Gateway是SpringCloud技术栈下的网关服务框架，在基于SpringCloud的微服务环境中，外部请求会到达SpringCloud Gateway应用，该应用对请求做转发、过滤、鉴权、熔断等前置操作，一个典型的请求响应流程如下所示：

![20200518170336757.png][]

### SpringCloud Gateway实战参考 ###

如果您之前没有使用过SpringCloud Gateway，推荐您阅读[《速体验SpringCloud Gateway》][SpringCloud Gateway]，有时间的话动手实战效果更佳，只需编写少量代码就能快速熟悉这个SpringCloud技术栈中非常重要的功能；

### kubernetes上的SpringCloud Gateway ###

注意以下两个知识点：

1.  SpringCloud Gateway之所以能将外部请求路由到正确的后台服务上，是因为注册中心的存在，SpringCloud Gateway可以在注册中心取得所有服务的信息，因此它可以根据路径和服务的对应关系，将请求转发到对应的服务上；
2.  如果您看过本系列的上一篇 [《spring-cloud-kubernetes的服务发现和轮询实战(含熔断)》][spring-cloud-kubernetes]，您就知道spring-cloud-kubernetes框架可以获取kubernetes环境内的所有服务（这里说的服务就是kubernetes的service）；

将以上两个知识点结合起来，于是可以推测：运行在kubernetes环境的SpringCloud Gateway应用，如果使用了spring-cloud-kubernetes框架就能得到kubernetes的service列表，因此可以承担网关的角色，将外部请求转发至kubernetes内的service上，最终到达对应的Pod；

架构如下图所示，请注意黄色背景的对话框，里面标识了关键操作：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nanVubGk_size_16_color_FFFFFF_t_70][]

至此，理论分析已经完成，我们来实战验证这个理论，接下来我们开发两个java应用：

1.  先开发一个普通的web服务，名为webdemo，提供一个http接口；
2.  再开发一个SpringCloud Gateway应用，名为k8sgatewaydemo；

### 环境信息 ###

本次实战的环境和版本信息如下：

1.  操作系统：CentOS Linux release 7.6.1810
2.  minikube：1.1.1
3.  Java：1.8.0\_191
4.  Maven：3.6.0
5.  fabric8-maven-plugin插件：3.5.37
6.  spring-cloud-kubernetes：1.0.1.RELEASE
7.  spring cloud：Greenwich.SR2
8.  springboot：2.1.6.RELEASE

### 源码下载 ###

如果您不打算写代码，也可以从GitHub上下载本次实战的源码，地址和链接信息如下表所示：

<table> 
 <thead> 
  <tr> 
   <th> <p>名称</p> </th> 
   <th> <p>链接</p> </th> 
   <th> <p>备注</p> </th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td> <p>项目主页</p> </td> 
   <td> <p>https://github.com/zq2599/blog_demos</p> <p>&nbsp;</p> </td> 
   <td> <p>该项目在GitHub上的主页</p> </td> 
  </tr> 
  <tr> 
   <td> <p>git仓库地址(https)</p> </td> 
   <td> <p>https://github.com/zq2599/blog_demos.git</p> </td> 
   <td> <p>该项目源码的仓库地址，https协议</p> </td> 
  </tr> 
  <tr> 
   <td> <p>git仓库地址(ssh)</p> </td> 
   <td> <p>git@github.com:zq2599/blog_demos.git</p> </td> 
   <td> <p>该项目源码的仓库地址，ssh协议</p> </td> 
  </tr> 
 </tbody> 
</table>

这个git项目中有多个文件夹，本章的两个应用分别在webdemo和k8sgatewaydemo文件夹下； 下图红框中是webdemo应用的源码：

![20200518170626837.png][]

下图红框中是k8sgatewaydemo应用的源码：

![20200518170649580.png][]

下面是详细的编码过程；

### 开发webdemo ###

webdemo是个极其普通的spring boot应用，和SpringCloud没有任何关系；

1.  webdemo提供一个http接口，将请求header中名为extendtag的参数返回给请求方，controller类如下：

@RestController
    @RequestMapping("/hello")
    public class HelloController {
    
        @RequestMapping(value = "time", method = RequestMethod.GET)
        public String hello(HttpServletRequest request){
    
            return "hello, "
                    + new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").format(new Date())
                    + ", extendtag ["
                    + request.getHeader("extendtag")
                    + "]";
        }
    }

1.  启动类WebdemoApplication.java：

@SpringBootApplication
    public class WebdemoApplication {
        public static void main(String[] args) {
            SpringApplication.run(WebdemoApplication.class, args);
        }
    }

1.  要注意的是pom.xml，里面通过名为fabric8-maven-plugin的maven插件，将webdemo快速部署到minikube环境：

1.  以上就是webdemo应用的内容了，接下来要编译、构建、部署到minikube环境，在pom.xml执行以下命令即可：

mvn clean install fabric8:deploy -Dfabric8.generator.from=fabric8/java-jboss-openjdk8-jdk -Pkubernetes

部署完成后终端输出类似如下成功信息：

[INFO] 
    [INFO] <<< fabric8-maven-plugin:3.5.37:deploy (default-cli) < install @ webdemo <<<
    [INFO] 
    [INFO] 
    [INFO] --- fabric8-maven-plugin:3.5.37:deploy (default-cli) @ webdemo ---
    [INFO] F8: Using Kubernetes at https://192.168.121.133:8443/ in namespace default with manifest /usr/local/work/k8s/webdemo/target/classes/META-INF/fabric8/kubernetes.yml 
    [INFO] Using namespace: default
    [INFO] Updating a Service from kubernetes.yml
    [INFO] Updated Service: target/fabric8/applyJson/default/service-webdemo.json
    [INFO] Using namespace: default
    [INFO] Updating Deployment from kubernetes.yml
    [INFO] Updated Deployment: target/fabric8/applyJson/default/deployment-webdemo.json
    [INFO] F8: HINT: Use the command `kubectl get pods -w` to watch your pods start up
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  11.804 s
    [INFO] Finished at: 2019-07-07T21:32:26+08:00
    [INFO] ------------------------------------------------------------------------

1.  查看service和pod，确认一切正常：

[root@minikube webdemo]# kubectl get service
    NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
    kubernetes       ClusterIP   10.96.0.1       <none>        443/TCP          29d
    webdemo          NodePort    10.106.98.137   <none>        8080:30160/TCP   115m
    [root@minikube webdemo]# kubectl get pod
    NAME                              READY   STATUS    RESTARTS   AGE
    webdemo-c9f774b9-gsbgx            1/1     Running   0          3m13s

1.  使用minikube命令取得webdemo服务对外暴露的地址：

[root@minikube webdemo]# minikube service webdemo --url
    http://192.168.121.133:30160

可见外部通过地址：[http://192.168.121.133:30160][http_192.168.121.133_30160] 即可访问到webdemo应用； 7. 在浏览器输入地址：[http://192.168.121.133:30160/hello/time][http_192.168.121.133_30160_hello_time] ，即可验证webdemo的http接口是否正常，如下图，由于header中没有extendtag属性，因此返回的extendtag为null：

![20200518170744364.png][]

至此，webdemo在minikue上已经正常运行，该开发gateway应用了；

### 开发k8sgatewaydemo ###

1.  基于maven创建一个名为k8sgatewaydemo的springboot应用，pom.xml内容如下：

<?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
        <modelVersion>4.0.0</modelVersion>
        <parent>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-parent</artifactId>
            <version>2.1.6.RELEASE</version>
            <relativePath/> 
        </parent>
        <groupId>com.bolingcavalry</groupId>
        <artifactId>k8sgatewaydemo</artifactId>
        <version>0.0.1-SNAPSHOT</version>
        <name>k8sgatewaydemo</name>
        <description>Demo project for Spring Boot</description>
    
        <properties>
            <java.version>1.8</java.version>
            <spring-boot.version>2.1.6.RELEASE</spring-boot.version>
            <maven-checkstyle-plugin.failsOnError>false</maven-checkstyle-plugin.failsOnError>
            <maven-checkstyle-plugin.failsOnViolation>false</maven-checkstyle-plugin.failsOnViolation>
            <maven-checkstyle-plugin.includeTestSourceDirectory>false</maven-checkstyle-plugin.includeTestSourceDirectory>
            <maven-compiler-plugin.version>3.5</maven-compiler-plugin.version>
            <maven-deploy-plugin.version>2.8.2</maven-deploy-plugin.version>
            <maven-failsafe-plugin.version>2.18.1</maven-failsafe-plugin.version>
            <maven-surefire-plugin.version>2.21.0</maven-surefire-plugin.version>
            <fabric8.maven.plugin.version>3.5.37</fabric8.maven.plugin.version>
            <springcloud.kubernetes.version>1.0.1.RELEASE</springcloud.kubernetes.version>
            <spring-cloud.version>Greenwich.SR2</spring-cloud.version>
        </properties>
    
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-gateway</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter-test</artifactId>
                <scope>test</scope>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-kubernetes-core</artifactId>
                <version>${springcloud.kubernetes.version}</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-kubernetes-discovery</artifactId>
                <version>${springcloud.kubernetes.version}</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
                <version>${springcloud.kubernetes.version}</version>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-commons</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-starter</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
            </dependency>
    
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-starter-netflix-hystrix</artifactId>
            </dependency>
        </dependencies>
    
        <dependencyManagement>
            <dependencies>
                <dependency>
                    <groupId>org.springframework.cloud</groupId>
                    <artifactId>spring-cloud-dependencies</artifactId>
                    <version>${spring-cloud.version}</version>
                    <type>pom</type>
                    <scope>import</scope>
                </dependency>
            </dependencies>
        </dependencyManagement>
    
        <build>
            <plugins>
                <plugin>
                    <groupId>org.springframework.boot</groupId>
                    <artifactId>spring-boot-maven-plugin</artifactId>
                    <version>${spring-boot.version}</version>
                    <executions>
                        <execution>
                            <goals>
                                <goal>repackage</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
    
                <plugin>
                    
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-deploy-plugin</artifactId>
                    <version>${maven-deploy-plugin.version}</version>
                    <configuration>
                        <skip>true</skip>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>${maven-surefire-plugin.version}</version>
                    <configuration>
                        <skipTests>true</skipTests>
                        
                        <useSystemClassLoader>false</useSystemClassLoader>
                    </configuration>
                </plugin>
                <plugin>
                    <groupId>io.fabric8</groupId>
                    <artifactId>fabric8-maven-plugin</artifactId>
                    <version>${fabric8.maven.plugin.version}</version>
                    <executions>
                        <execution>
                            <id>fmp</id>
                            <goals>
                                <goal>resource</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>
            </plugins>
        </build>
    
        <profiles>
            <profile>
                <id>kubernetes</id>
                <build>
                    <plugins>
                        <plugin>
                            <groupId>io.fabric8</groupId>
                            <artifactId>fabric8-maven-plugin</artifactId>
                            <version>${fabric8.maven.plugin.version}</version>
                            <executions>
                                <execution>
                                    <id>fmp</id>
                                    <goals>
                                        <goal>resource</goal>
                                        <goal>build</goal>
                                    </goals>
                                </execution>
                            </executions>
                            <configuration>
                                <enricher>
                                    <config>
                                        <fmp-service>
                                            <type>NodePort</type>
                                        </fmp-service>
                                    </config>
                                </enricher>
                            </configuration>
                        </plugin>
                    </plugins>
                </build>
            </profile>
        </profiles>
    </project>

上述pom文件中有以下几点需要注意： 第一、 依赖spring-cloud-kubernetes-core和spring-cloud-kubernetes-discovery，这样能用到spring-cloud-kubernetes提供的服务发现能力； 第二、依赖spring-cloud-starter-gateway，这样能用上SpringCloud的gateway能力； 第三、不要依赖spring-boot-starter-web，会和spring-cloud-starter-gateway冲突，启动时抛出以下异常：

2019-07-06 08:12:09.188  WARN 1 --- [           main] ConfigServletWebServerApplicationContext : 
    Exception encountered during context initialization - 
    cancelling refresh attempt: 
    org.springframework.beans.factory.UnsatisfiedDependencyException: 
    Error creating bean with name 'routeDefinitionRouteLocator' defined in class path resource [org/springframework/cloud/gateway/config/GatewayAutoConfiguration.class]: 
    Unsatisfied dependency expressed through method 'routeDefinitionRouteLocator' parameter 1; 
    nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: 
    Error creating bean with name 'modifyRequestBodyGatewayFilterFactory' defined in class path resource [org/springframework/cloud/gateway/config/GatewayAutoConfiguration.class]: 
    Unsatisfied dependency expressed through method 'modifyRequestBodyGatewayFilterFactory' parameter 0; 
    nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.http.codec.ServerCodecConfigurer' available: 
    expected at least 1 bean which qualifies as autowire candidate. Dependency annotations: {}

1.  开发SpringCloud Gateway的启动类K8sgatewaydemoApplication.java，里面也包含了网关路由配置的实例化，除了配置路径和转发服务的关系，还在请求的header中添加了extendtag属性，请注意注释的内容：

@SpringBootApplication
    @EnableDiscoveryClient
    public class K8sgatewaydemoApplication {
    
        public static void main(String[] args) {
            SpringApplication.run(K8sgatewaydemoApplication.class, args);
        }
    
        @Bean
        public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
            return builder.routes()
                    //增加一个path匹配，以"/customize/hello/"开头的请求都在此路由
                    .route(r -> r.path("/customize/hello/**")
                            //表示将路径中的第一级参数删除，用剩下的路径与webdemo的路径做拼接，
                            //这里就是"lb://webdemo/hello/"，能匹配到webdemo的HelloController的路径
                            .filters(f -> f.stripPrefix(1)
                                    //在请求的header中添加一个key&value
                                    .addRequestHeader("extendtag", "geteway-" + System.currentTimeMillis()))
                            //指定匹配服务webdemo，lb是load balance的意思
                            .uri("lb://webdemo")
                    ).build();
        }
    }

从上述代码可见，K8sgatewaydemoApplication与普通环境下的SpringCloud Gateway并无差别，都是通过EnableDiscoveryClient注解获取服务列表，配置RouteLocator实现路由逻辑； 3. 配置文件application.yml的内容：

spring:
      application:
        name: gateway
      cloud:
        gateway:
          discovery:
            locator:
              enabled: true
              lowerCaseServiceId: true

1.  以上就是k8sgatewaydemo应用的内容了，接下来要编译、构建、部署到minikube环境，在pom.xml执行以下命令即可：

mvn clean install fabric8:deploy -Dfabric8.generator.from=fabric8/java-jboss-openjdk8-jdk -Pkubernetes

部署完成后终端输出类似如下成功信息：

[INFO] 
    [INFO] <<< fabric8-maven-plugin:3.5.37:deploy (default-cli) < install @ k8sgatewaydemo <<<
    [INFO] 
    [INFO] 
    [INFO] --- fabric8-maven-plugin:3.5.37:deploy (default-cli) @ k8sgatewaydemo ---
    [INFO] F8: Using Kubernetes at https://192.168.121.133:8443/ in namespace default with manifest /usr/local/work/k8s/k8sgatewaydemo/target/classes/META-INF/fabric8/kubernetes.yml 
    [INFO] Using namespace: default
    [INFO] Updating a Service from kubernetes.yml
    [INFO] Updated Service: target/fabric8/applyJson/default/service-k8sgatewaydemo.json
    [INFO] Using namespace: default
    [INFO] Updating Deployment from kubernetes.yml
    [INFO] Updated Deployment: target/fabric8/applyJson/default/deployment-k8sgatewaydemo.json
    [INFO] F8: HINT: Use the command `kubectl get pods -w` to watch your pods start up
    [INFO] ------------------------------------------------------------------------
    [INFO] BUILD SUCCESS
    [INFO] ------------------------------------------------------------------------
    [INFO] Total time:  16.538 s
    [INFO] Finished at: 2019-07-07T22:04:48+08:00
    [INFO] ------------------------------------------------------------------------

1.  查看service和pod，确认一切正常：

[root@minikube k8sgatewaydemo]# clear
    [root@minikube k8sgatewaydemo]# kubectl get service
    NAME             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
    k8sgatewaydemo   NodePort    10.97.94.238    <none>        8080:31352/TCP   129m
    kubernetes       ClusterIP   10.96.0.1       <none>        443/TCP          29d
    webdemo          NodePort    10.106.98.137   <none>        8080:30160/TCP   145m
    [root@minikube k8sgatewaydemo]# kubectl get pod
    NAME                              READY   STATUS    RESTARTS   AGE
    k8sgatewaydemo-6fbb79885c-r2jfn   1/1     Running   0          33s
    webdemo-c9f774b9-gsbgx            1/1     Running   0          32m

1.  使用minikube命令取得webdemo服务对外暴露的地址：

[root@minikube k8sgatewaydemo]# minikube service k8sgatewaydemo --url
    http://192.168.121.133:31352

可见外部通过地址：[http://192.168.121.133:31352][http_192.168.121.133_31352] 即可访问到k8sgatewaydemo应用； 7. 在浏览器输入地址：[http://192.168.121.133:31352/customize/hello/time][http_192.168.121.133_31352_customize_hello_time] ，即可验证k8sgatewaydemo作为网关应用，能否将路径中带有customize的请求转发到webdemo应用，并且在请求header中添加名为entendtag的属性，如下图，浏览器展示的内容是webdemo的http接口返回的，并且extendtag的内容也不为空了，而是k8sgatewaydemo在转发前写入的：

![20200518170824585.png][]

上述结果表明已可以证明我们之前的推测是正确的：SpringCloud Gateway应用在使用了spring-cloud-kubernetes提供的注册发现能力后，可以将请求转发到kubernetes环境中的服务上； 也就是说，借助spring-cloud-kubernetes框架，你在SpringCloud环境开发的SpringCloud Gateway应用，可以以很小的代价迁移到kubernetes环境，与kubernetes环境中的service可以很好的交互，而原有的eureka注册中心也可以不用了；

### 解决权限问题 ###

如果您的spring-cloud-kubernetes在向webdemo转发请求时抛出以下错误，那是因为遇到了kubernetes的权限问题：

2019-07-06 04:46:40.042  WARN 1 --- [erListUpdater-1] c.n.l.PollingServerListUpdater           : Failed one update cycle
    io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: GET at: https://10.96.0.1/api/v1/namespaces/default/endpoints/account-service. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. endpoints "account-service" is forbidden: User "system:serviceaccount:default:default" cannot get resource "endpoints" in API group "" in the namespace "default".
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:476) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:413) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:381) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:344) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:313) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleGet(OperationSupport.java:296) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleGet(BaseOperation.java:794) ~[kubernetes-client-4.1.0.jar!/:na]
            at io.fabric8.kubernetes.client.dsl.base.BaseOperation.getMandatory(BaseOperation.java:210) ~[kubernetes-client-4.1.0.jar!/:na]
    
    
            at io.fabric8.kubernetes.client.dsl.base.BaseOperation.get(BaseOperation.java:177) ~[kubernetes-client-4.1.0.jar!/:na]
            at org.springframework.cloud.kubernetes.ribbon.KubernetesServerList
    
    
    .getUpdatedListOfServers(KubernetesServerList.java:75) ~[spring-cloud-kubernetes-ribbon-1.0.1.RELEASE.jar!/:1.0.1.RELEASE]
            at com.netflix.loadbalancer.DynamicServerListLoadBalancer.updateListOfServers(DynamicServerListLoadBalancer.java:240) ~[ribbon-loadbalancer-2.3.0.jar!/:2.3.0]
            at com.netflix.loadbalancer.DynamicServerListLoadBalancer$1.doUpdate(DynamicServerListLoadBalancer.java:62) ~[ribbon-loadbalancer-2.3.0.jar!/:2.3.0]
            at com.netflix.loadbalancer.PollingServerListUpdater$1.run(PollingServerListUpdater.java:116) ~[ribbon-loadbalancer-2.3.0.jar!/:2.3.0]
            at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_191]
            at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_191]
            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_191]
            at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_191]
            at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191]
            at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191]
            at java.lang.Thread.run(Thread.java:748) [na:1.8.0_191]

处理方法是创建ServiceAccount对象，步骤如下：

1.  创建名为fabric8-rbac.yaml的文件，内容如下：

# NOTE: The service account `default:default` already exists in k8s cluster.
    # You can create a new account following like this:
    #---
    #apiVersion: v1
    #kind: ServiceAccount
    #metadata:
    #  name: <new-account-name>
    #  namespace: <namespace>
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1beta1
    kind: ClusterRoleBinding
    metadata:
      name: fabric8-rbac
    subjects:
      - kind: ServiceAccount
        # Reference to upper's `metadata.name`
        name: default
        # Reference to upper's `metadata.namespace`
        namespace: default
    roleRef:
      kind: ClusterRole
      name: cluster-admin
      apiGroup: rbac.authorization.k8s.io

1.  执行以下命令即可创建ServiceAccount对象：

kubectl apply -f fabric8-rbac.yaml

1.  再在浏览器上继续刚才的验证，可以操作成功；

### 最后一个疑问 ###

再回顾一下k8sgatewaydemo的开发过程，您会发现除了依赖spring-cloud-kubernetes对应的maven库，我们并没有显式调用spring-cloud-kubernetes相关的API或者做相关配置，就获取了所在kubernetes环境的原生服务，这是怎么回事呢？为何成本如此的低？ 答案就在[《spring-cloud-kubernetes背后的三个关键知识点》][spring-cloud-kubernetes 1]一文中，推荐您回顾一下此文。

至此，spring-cloud-kubernetes框架下的SpringCloud Gateway开发实战就完成了，希望本文能帮助您更好的理解和使用spring-cloud-kubernetes，更加高效的将应用向容器化迁移。

[20200518170336757.png]: https://img-blog.csdnimg.cn/20200518170336757.png
[SpringCloud Gateway]: https://xinchen.blog.csdn.net/article/details/94907172
[spring-cloud-kubernetes]: https://xinchen.blog.csdn.net/article/details/92394559
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nanVubGk_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20200518170451109.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3poYW5nanVubGk=,size_16,color_FFFFFF,t_70
[20200518170626837.png]: https://img-blog.csdnimg.cn/20200518170626837.png
[20200518170649580.png]: https://img-blog.csdnimg.cn/20200518170649580.png
[http_192.168.121.133_30160]: http://192.168.121.133:30160/
[http_192.168.121.133_30160_hello_time]: http://192.168.121.133:30160/hello/time
[20200518170744364.png]: https://img-blog.csdnimg.cn/20200518170744364.png
[http_192.168.121.133_31352]: http://192.168.121.133:31352/
[http_192.168.121.133_31352_customize_hello_time]: http://192.168.121.133:31352/customize/hello/time
[20200518170824585.png]: https://img-blog.csdnimg.cn/20200518170824585.png
[spring-cloud-kubernetes 1]: https://blog.csdn.net/boling_cavalry/article/details/92069486