生产环境下sentinel规则持久化方案

短命女 2021-09-21 23:36 461阅读 0赞

参考文档：[https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel][https_github.com_alibaba_Sentinel_wiki_E5_9C_A8_E7_94_9F_E4_BA_A7_E7_8E_AF_E5_A2_83_E4_B8_AD_E4_BD_BF_E7_94_A8-Sentinel]

### **一，背景** ###

sentinel 官方规则管理 提供三种模式

<table> 
 <thead> 
  <tr> 
   <th>推送模式</th> 
   <th>说明</th> 
   <th>优点</th> 
   <th>缺点</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td><a href="https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel#%E5%8E%9F%E5%A7%8B%E6%A8%A1%E5%BC%8F" rel="nofollow">原始模式</a></td> 
   <td>API 将规则推送至客户端并直接更新到内存中，扩展写数据源（<a href="https://github.com/alibaba/Sentinel/wiki/%E5%8A%A8%E6%80%81%E8%A7%84%E5%88%99%E6%89%A9%E5%B1%95" rel="nofollow"><code>WritableDataSource</code></a>）</td> 
   <td>简单，无任何依赖</td> 
   <td>不保证一致性；规则保存在内存中，重启即消失。严重不建议用于生产环境</td> 
  </tr> 
  <tr> 
   <td><a href="https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel#Pull%E6%A8%A1%E5%BC%8F" rel="nofollow">Pull 模式</a></td> 
   <td>扩展写数据源（<a href="https://github.com/alibaba/Sentinel/wiki/%E5%8A%A8%E6%80%81%E8%A7%84%E5%88%99%E6%89%A9%E5%B1%95" rel="nofollow"><code>WritableDataSource</code></a>）， 客户端主动向某个规则管理中心定期轮询拉取规则，这个规则中心可以是 RDBMS、文件 等</td> 
   <td>简单，无任何依赖；规则持久化</td> 
   <td>不保证一致性；实时性不保证，拉取过于频繁也可能会有性能问题。</td> 
  </tr> 
  <tr> 
   <td><strong><a href="https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel#Push%E6%A8%A1%E5%BC%8F" rel="nofollow">Push 模式</a></strong></td> 
   <td>扩展读数据源（<a href="https://github.com/alibaba/Sentinel/wiki/%E5%8A%A8%E6%80%81%E8%A7%84%E5%88%99%E6%89%A9%E5%B1%95" rel="nofollow"><code>ReadableDataSource</code></a>），规则中心统一推送，客户端通过注册监听器的方式时刻监听变化，比如使用 Nacos、Zookeeper 等配置中心。这种方式有更好的实时性和一致性保证。<strong>生产环境下一般采用 push 模式的数据源。</strong></td> 
   <td>规则持久化；一致性；快速</td> 
   <td>引入第三方依赖</td> 
  </tr> 
 </tbody> 
</table>

默认我们引入

<dependency>
                <groupId>com.alibaba.cloud</groupId>
                <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
            </dependency>

各种规则是默认保存到内存中的，每次项目重启之后规则都会消失不见。在这我们提供两种解决方案。[Pull 模式][Pull] 和 **[Push 模式][Push]**

### **二，方案** ###

**1，**[Pull 模式][Pull] (将各种规则添加进不同的文件中)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70][]

package com.example.demo.config;
    
    import com.alibaba.csp.sentinel.command.handler.ModifyParamFlowRulesCommandHandler;
    import com.alibaba.csp.sentinel.datasource.FileRefreshableDataSource;
    import com.alibaba.csp.sentinel.datasource.FileWritableDataSource;
    import com.alibaba.csp.sentinel.datasource.ReadableDataSource;
    import com.alibaba.csp.sentinel.datasource.WritableDataSource;
    import com.alibaba.csp.sentinel.init.InitFunc;
    import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRule;
    import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRuleManager;
    import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRule;
    import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRuleManager;
    import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
    import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
    import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRule;
    import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRuleManager;
    import com.alibaba.csp.sentinel.slots.system.SystemRule;
    import com.alibaba.csp.sentinel.slots.system.SystemRuleManager;
    import com.alibaba.csp.sentinel.transport.util.WritableDataSourceRegistry;
    import com.alibaba.fastjson.JSON;
    import com.alibaba.fastjson.TypeReference;
    
    import java.io.File;
    import java.io.IOException;
    import java.util.List;
    
    /**
     * 拉模式规则持久化
     *
     * @author lwc
     */
    public class FileDataSourceInit implements InitFunc {
    
        @Override
        public void init() throws Exception {
    
            String ruleDir = System.getProperty("user.dir") + "/src/main/resources/sentinel/rules";
            String flowRulePath = ruleDir + "/flow-rule.json";
            String degradeRulePath = ruleDir + "/degrade-rule.json";
            String systemRulePath = ruleDir + "/system-rule.json";
            String authorityRulePath = ruleDir + "/authority-rule.json";
            String paramFlowRulePath = ruleDir + "/param-flow-rule.json";
    
            this.mkdirIfNotExits(ruleDir);
            this.createFileIfNotExits(flowRulePath);
            this.createFileIfNotExits(degradeRulePath);
            this.createFileIfNotExits(systemRulePath);
            this.createFileIfNotExits(authorityRulePath);
            this.createFileIfNotExits(paramFlowRulePath);
    
            //流控规则
            ReadableDataSource<String, List<FlowRule>> flowRuleRds = new FileRefreshableDataSource<>(flowRulePath, source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {
            })
            );
            //将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时，就会更新规则到内存
            FlowRuleManager.register2Property(flowRuleRds.getProperty());
            WritableDataSource<List<FlowRule>> flowRuleWds = new FileWritableDataSource<>(flowRulePath, JSON::toJSONString);
            //将可写数据源注册至transport模块的WritableDataSourceRegistry中 这样收到控制台推送的规则时，Sentinel会先更新到内存，然后将规则写入到文件中
            WritableDataSourceRegistry.registerFlowDataSource(flowRuleWds);
    
            //降级规则
            ReadableDataSource<String, List<DegradeRule>> degradeRuleRds = new FileRefreshableDataSource<>(degradeRulePath, source -> JSON.parseObject(source, new TypeReference<List<DegradeRule>>() {
            }));
            DegradeRuleManager.register2Property(degradeRuleRds.getProperty());
            WritableDataSource<List<DegradeRule>> degradeRuleWds = new FileWritableDataSource<>(degradeRulePath, JSON::toJSONString);
            WritableDataSourceRegistry.registerDegradeDataSource(degradeRuleWds);
    
            //系统规则
            ReadableDataSource<String, List<SystemRule>> systemRuleRds = new FileRefreshableDataSource<>(systemRulePath, source -> JSON.parseObject(source, new TypeReference<List<SystemRule>>() {
            }));
            SystemRuleManager.register2Property(systemRuleRds.getProperty());
            WritableDataSource<List<SystemRule>> systemRuleWds = new FileWritableDataSource<>(systemRulePath, JSON::toJSONString);
            WritableDataSourceRegistry.registerSystemDataSource(systemRuleWds);
    
            //授权规则
            ReadableDataSource<String, List<AuthorityRule>> authorityRuleRds = new FileRefreshableDataSource<>(authorityRulePath, source -> JSON.parseObject(source, new TypeReference<List<AuthorityRule>>() {
            }));
            AuthorityRuleManager.register2Property(authorityRuleRds.getProperty());
            WritableDataSource<List<AuthorityRule>> authorityRuleWds = new FileWritableDataSource<>(authorityRulePath, JSON::toJSONString);
            WritableDataSourceRegistry.registerAuthorityDataSource(authorityRuleWds);
    
            //热点参数规则
            ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleRds = new FileRefreshableDataSource<>(paramFlowRulePath, source -> JSON.parseObject(source, new TypeReference<List<ParamFlowRule>>() {
            }));
            ParamFlowRuleManager.register2Property(paramFlowRuleRds.getProperty());
            WritableDataSource<List<ParamFlowRule>> paramFlowRuleWds = new FileWritableDataSource<>(paramFlowRulePath, JSON::toJSONString);
            ModifyParamFlowRulesCommandHandler.setWritableDataSource(paramFlowRuleWds);
        }
    
        private void mkdirIfNotExits(String filePath) {
            File file = new File(filePath);
            if (!file.exists()) {
                boolean mkdirs = file.mkdirs();
            }
        }
    
        private void createFileIfNotExits(String filePath) throws IOException {
            File file = new File(filePath);
            if (!file.exists()) {
                boolean newFile = file.createNewFile();
            }
        }
    }

配置加载，这里使用的是  SPI 加载，  SPI全称Service Provider Interface，是Java提供的一套用来被第三方实现或者扩展的接口，它可以用来启用框架扩展和替换组件。 SPI的作用就是为这些被扩展的API寻找服务实现。感兴趣的可以去学习学习。

在resources 目录创建 META-INF\\services，创建加载文件，文件名为接口名 com.alibaba.csp.sentinel.init.InitFunc ,内容为实现类路径 com.example.demo.config.FileDataSourceInit

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 1][]

重启项目，添加规则

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 2][]

我们发现文件已经创建

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 3][]

2，**[Push 模式][Push] ** (将各种规则添加进nacos)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 4][]

引入依赖

<dependency>
                <groupId>com.alibaba.csp</groupId>
                <artifactId>sentinel-datasource-nacos</artifactId>
            </dependency>

package com.example.demo.config;
    
    //
    // Source code recreated from a .class file by IntelliJ IDEA
    // (powered by FernFlower decompiler)
    //
    
    import com.alibaba.csp.sentinel.concurrent.NamedThreadFactory;
    import com.alibaba.csp.sentinel.datasource.AbstractDataSource;
    import com.alibaba.csp.sentinel.datasource.Converter;
    import com.alibaba.csp.sentinel.log.RecordLog;
    import com.alibaba.csp.sentinel.util.AssertUtil;
    import com.alibaba.csp.sentinel.util.StringUtil;
    import com.alibaba.nacos.api.NacosFactory;
    import com.alibaba.nacos.api.config.ConfigService;
    import com.alibaba.nacos.api.config.listener.Listener;
    
    import java.util.Properties;
    import java.util.concurrent.*;
    import java.util.concurrent.ThreadPoolExecutor.DiscardOldestPolicy;
    
    /**
     * @author lwc
     */
    public class CustomNacosDataSource<T> extends AbstractDataSource<String, T> {
        private static final int DEFAULT_TIMEOUT = 3000;
        private final ExecutorService pool;
        private final Listener configListener;
        protected final String groupId;
        protected final String dataId;
        private final Properties properties;
        protected ConfigService configService;
    
        public CustomNacosDataSource(String serverAddr, String groupId, String dataId, Converter<String, T> parser) {
            this(buildProperties(serverAddr), groupId, dataId, parser);
        }
    
        public CustomNacosDataSource(final Properties properties, final String groupId, final String dataId, Converter<String, T> parser) {
            super(parser);
            this.pool = new ThreadPoolExecutor(1, 1, 0L, TimeUnit.MILLISECONDS, new ArrayBlockingQueue(1), new NamedThreadFactory("sentinel-nacos-ds-update"), new DiscardOldestPolicy());
            this.configService = null;
            if (!StringUtil.isBlank(groupId) && !StringUtil.isBlank(dataId)) {
                AssertUtil.notNull(properties, "Nacos properties must not be null, you could put some keys from PropertyKeyConst");
                this.groupId = groupId;
                this.dataId = dataId;
                this.properties = properties;
                this.configListener = new Listener() {
                    @Override
                    public Executor getExecutor() {
                        return CustomNacosDataSource.this.pool;
                    }
    
                    @Override
                    public void receiveConfigInfo(String configInfo) {
                        RecordLog.info(String.format("[CustomNacosDataSource] New property value received for (properties: %s) (dataId: %s, groupId: %s): %s", properties, dataId, groupId, configInfo), new Object[0]);
                        T newValue = CustomNacosDataSource.this.parser.convert(configInfo);
                        CustomNacosDataSource.this.getProperty().updateValue(newValue);
                    }
                };
                this.initNacosListener();
                this.loadInitialConfig();
            } else {
                throw new IllegalArgumentException(String.format("Bad argument: groupId=[%s], dataId=[%s]", groupId, dataId));
            }
        }
    
        private void loadInitialConfig() {
            try {
                T newValue = this.loadConfig();
                if (newValue == null) {
                    RecordLog.warn("[CustomNacosDataSource] WARN: initial config is null, you may have to check your data source", new Object[0]);
                }
    
                this.getProperty().updateValue(newValue);
            } catch (Exception var2) {
                RecordLog.warn("[CustomNacosDataSource] Error when loading initial config", var2);
            }
    
        }
    
        private void initNacosListener() {
            try {
                this.configService = NacosFactory.createConfigService(this.properties);
                this.configService.addListener(this.dataId, this.groupId, this.configListener);
            } catch (Exception var2) {
                RecordLog.warn("[CustomNacosDataSource] Error occurred when initializing Nacos data source", var2);
                var2.printStackTrace();
            }
    
        }
    
        @Override
        public String readSource() throws Exception {
            if (this.configService == null) {
                throw new IllegalStateException("Nacos config service has not been initialized or error occurred");
            } else {
                return this.configService.getConfig(this.dataId, this.groupId, 3000L);
            }
        }
    
        @Override
        public void close() {
            if (this.configService != null) {
                this.configService.removeListener(this.dataId, this.groupId, this.configListener);
            }
    
            this.pool.shutdownNow();
        }
    
        private static Properties buildProperties(String serverAddr) {
            Properties properties = new Properties();
            properties.setProperty("serverAddr", serverAddr);
            return properties;
        }
    }

package com.example.demo.config;
    
    import com.alibaba.csp.sentinel.datasource.Converter;
    import com.alibaba.csp.sentinel.datasource.WritableDataSource;
    
    import java.util.concurrent.locks.Lock;
    import java.util.concurrent.locks.ReentrantLock;
    
    /**
     * @author lwc
     */
    public class NacosWritableDataSource<T> extends CustomNacosDataSource implements WritableDataSource<T> {
    
        private final Lock lock = new ReentrantLock(true);
    
        private final Converter<T, String> configEncoder;
    
        public NacosWritableDataSource(String serverAddr, String groupId, String dataId, Converter<T, String> configEncoder) {
            super(serverAddr, groupId, dataId, configEncoder);
            this.configEncoder = configEncoder;
        }
    
        @Override
        public void write(T value) throws Exception {
            lock.lock();
            try {
                String convertResult = configEncoder.convert(value);
                configService.publishConfig(dataId, groupId, convertResult);
            } finally {
                lock.unlock();
            }
        }
    
        @Override
        public void close() {
            super.close();
        }
    }

package com.example.demo.config;
    
    import com.alibaba.csp.sentinel.command.handler.ModifyParamFlowRulesCommandHandler;
    import com.alibaba.csp.sentinel.datasource.ReadableDataSource;
    import com.alibaba.csp.sentinel.datasource.WritableDataSource;
    import com.alibaba.csp.sentinel.init.InitFunc;
    import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRule;
    import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRuleManager;
    import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRule;
    import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRuleManager;
    import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
    import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
    import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRule;
    import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRuleManager;
    import com.alibaba.csp.sentinel.slots.system.SystemRule;
    import com.alibaba.csp.sentinel.slots.system.SystemRuleManager;
    import com.alibaba.csp.sentinel.transport.util.WritableDataSourceRegistry;
    import com.alibaba.fastjson.JSON;
    import com.alibaba.fastjson.TypeReference;
    
    import java.util.List;
    
    /**
     * @author lwc
     */
    public class NacosDataSourceInit implements InitFunc {
    
        private static final String SERVER_ADDR = "localhost:8848";
        private static final String GROUP_ID = "RENREN_CLOUD_GROUP";
        private static final String FLOW_DATA_ID = "sentinel-flow-rule.json";
        private static final String AUTHORITY_DATA_ID = "sentinel-authority-rule.json";
        private static final String DEGRADE_DATA_ID = "sentinel-degrade-rule.json";
        private static final String PARAM_DATA_ID = "sentinel-param-flow-rule.json";
        private static final String SYSTEM_DATA_ID = "sentinel-system-rule.json";
    
        @Override
        public void init() {
    
            //流控规则
            ReadableDataSource<String, List<FlowRule>> flowRuleDataSource = new CustomNacosDataSource<>(SERVER_ADDR, GROUP_ID, FLOW_DATA_ID, source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {
            }));
            FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
    
            WritableDataSource<List<FlowRule>> writableFlowRuleDataSource = new NacosWritableDataSource<>(SERVER_ADDR, GROUP_ID, FLOW_DATA_ID, JSON::toJSONString);
            WritableDataSourceRegistry.registerFlowDataSource(writableFlowRuleDataSource);
    
            //降级规则
            ReadableDataSource<String, List<DegradeRule>> degradeRuleDataSource = new CustomNacosDataSource<>(SERVER_ADDR, GROUP_ID, DEGRADE_DATA_ID, source -> JSON.parseObject(source, new TypeReference<List<DegradeRule>>() {
            }));
            DegradeRuleManager.register2Property(degradeRuleDataSource.getProperty());
    
            WritableDataSource<List<DegradeRule>> writableDegradeRuleDataSource = new NacosWritableDataSource<>(SERVER_ADDR, GROUP_ID, DEGRADE_DATA_ID, JSON::toJSONString);
            WritableDataSourceRegistry.registerDegradeDataSource(writableDegradeRuleDataSource);
    
            //系统规则
            ReadableDataSource<String, List<SystemRule>> systemRuleDataSource = new CustomNacosDataSource<>(SERVER_ADDR, GROUP_ID, SYSTEM_DATA_ID, source -> JSON.parseObject(source, new TypeReference<List<SystemRule>>() {
            }));
            SystemRuleManager.register2Property(systemRuleDataSource.getProperty());
    
            WritableDataSource<List<SystemRule>> writableSystemRuleDataSource = new NacosWritableDataSource<>(SERVER_ADDR, GROUP_ID, SYSTEM_DATA_ID, JSON::toJSONString);
            WritableDataSourceRegistry.registerSystemDataSource(writableSystemRuleDataSource);
    
            //授权规则
            ReadableDataSource<String, List<AuthorityRule>> authorityRuleDataSource = new CustomNacosDataSource<>(SERVER_ADDR, GROUP_ID, AUTHORITY_DATA_ID, source -> JSON.parseObject(source, new TypeReference<List<AuthorityRule>>() {
            }));
            AuthorityRuleManager.register2Property(authorityRuleDataSource.getProperty());
    
            WritableDataSource<List<AuthorityRule>> writableAuthorityRuleDataSource = new NacosWritableDataSource<>(SERVER_ADDR, GROUP_ID, FLOW_DATA_ID, JSON::toJSONString);
            WritableDataSourceRegistry.registerAuthorityDataSource(writableAuthorityRuleDataSource);
    
            //热点参数规则
            ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleDataSource = new CustomNacosDataSource<>(SERVER_ADDR, GROUP_ID, PARAM_DATA_ID, source -> JSON.parseObject(source, new TypeReference<List<ParamFlowRule>>() {
            }));
            ParamFlowRuleManager.register2Property(paramFlowRuleDataSource.getProperty());
    
            WritableDataSource<List<ParamFlowRule>> writableParamFlowRuleDataSource = new NacosWritableDataSource<>(SERVER_ADDR, GROUP_ID, PARAM_DATA_ID, JSON::toJSONString);
            ModifyParamFlowRulesCommandHandler.setWritableDataSource(writableParamFlowRuleDataSource);
    
        }
    }

指定加载

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 5][]

去nacos 创建文件

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 6][]

去sentinel 添加规则

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 7][]

去nacos查看，发现数据已经更新

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 8][]

> 项目地址：[https://gitee.com/g7go/sentinel-demo-test.git][https_gitee.com_g7go_sentinel-demo-test.git]

3，直接修改sentinel 面板端源码，添加按按钮或者直接修改保存按钮，把数据同步到nacos等地方 源码重新打包

这种方法网上解决方案较多，这里不再赘述。

[https_github.com_alibaba_Sentinel_wiki_E5_9C_A8_E7_94_9F_E4_BA_A7_E7_8E_AF_E5_A2_83_E4_B8_AD_E4_BD_BF_E7_94_A8-Sentinel]: https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel
[Pull]: https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel#Pull%E6%A8%A1%E5%BC%8F
[Push]: https://github.com/alibaba/Sentinel/wiki/%E5%9C%A8%E7%94%9F%E4%BA%A7%E7%8E%AF%E5%A2%83%E4%B8%AD%E4%BD%BF%E7%94%A8-Sentinel#Push%E6%A8%A1%E5%BC%8F
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70]: /images/20210920/6fd45e6b337d4cba909535496273eb24.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 1]: /images/20210920/97b19f55f6d54f5ba1b83e6c4fb46ab1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 2]: /images/20210920/fc3b77a9ef2d49848f83658348d23e54.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 3]: /images/20210920/bfcfbea60e444748b99a32667cfe241d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 4]: /images/20210920/e774567007dd4189a47fdd9ba15b158e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 5]: /images/20210920/09ba334b59c8452f9de67bd1a448b7bd.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 6]: /images/20210920/e9fdace87d1d4fcbba4b5abf8f3f5a11.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 7]: /images/20210920/1022321a15fa4b30abdfe712a9f4cd20.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2VjaGl6YW8xODM5_size_16_color_FFFFFF_t_70 8]: /images/20210920/982db5fadcc64112ab0b0d7d94b1b9e3.png
[https_gitee.com_g7go_sentinel-demo-test.git]: https://gitee.com/g7go/sentinel-demo-test.git