【kubernetes/k8s源码分析】calico node felix policy manager源码分析

亦凉 2021-11-09 05:26 213阅读 0赞

[Network Policy][] 提供了基于策略的网络控制，用于隔离应用并减少攻击面。使用标签选择器模拟传统的分段网络，并通过策略控制流量以及外部流量。Network Policy 需要网络插件来监测策略和 Pod 的变更，并为 Pod 配置流量控制

policy 和 profile 都是做策略管理的，但是policy 更加细致，优先级也更高(相对于profile)。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3pob25nbGluemhhbmc_size_16_color_FFFFFF_t_70][]

*  CNI 网络插件：负责给 Pod 配置网络接口
 *  Policy controller：监听 Network Policy 的变化，并将 Policy 应用到相应的网络接口

参考： [https://www.bookstack.cn/read/feiskyer-kubernetes-handbook/plugins-network-policy.md][https_www.bookstack.cn_read_feiskyer-kubernetes-handbook_plugins-network-policy.md]

## 结构体 policyManager： ##

这要是提交 policy / profile 到数据平面层来更新 iptables 规则链

>     // policyManager simply renders policy/profile updates into iptables.Chain objects and sends
>     // them to the dataplane layer.
>     type policyManager struct {
>        rawTable     iptablesTable
>        mangleTable  iptablesTable
>        filterTable  iptablesTable
>        ruleRenderer policyRenderer
>        ipVersion    uint8
>        callbacks    policyManagerCallbacks
>     }

> NewIntDataplaneDriver
> 
>        -->  newPolicyManager

# 1. newPolicyManager 函数 #

实例化 policyManager，raw mangle filter 表都是实现了 update remove iptables 规则链

func newPolicyManager(rawTable, mangleTable, filterTable iptablesTable, ruleRenderer policyRenderer, ipVersion uint8, callbacks *callbacks) *policyManager {
    	return &policyManager{
    		rawTable:     rawTable,
    		mangleTable:  mangleTable,
    		filterTable:  filterTable,
    		ruleRenderer: ruleRenderer,
    		ipVersion:    ipVersion,
    		callbacks:    newPolicyManagerCallbacks(callbacks, ipVersion),
    	}
    }

# 2. OnUpdate 函数 #

如果消息类型为 ActivePolicyUpdate，

PolicyToIptablesChains，将 policy 换成 iptables 规则链，第 3 章节讲解

func (m *policyManager) OnUpdate(msg interface{}) {
    	switch msg := msg.(type) {
    	case *proto.ActivePolicyUpdate:
    		log.WithField("id", msg.Id).Debug("Updating policy chains")
    		chains := m.ruleRenderer.PolicyToIptablesChains(msg.Id, msg.Policy, m.ipVersion)
    		m.rawTable.UpdateChains(chains)
    		m.mangleTable.UpdateChains(chains)
    		m.filterTable.UpdateChains(chains)
    		m.callbacks.InvokeUpdatePolicy(*msg.Id, msg.Policy)

> NewIntDataplaneDriver
> 
>         -->  rules.NewRenderer
> 
>                -->  DefaultRuleRenderer 实例化

#  3. PolicyToIptablesChains 函数 #

inbound 的 iptables 规则链名为 cali-pi-XXXX

outbound 的 iptables 规则链名为 cali-po-xxxx

func (r *DefaultRuleRenderer) PolicyToIptablesChains(policyID *proto.PolicyID, policy *proto.Policy, ipVersion uint8) []*iptables.Chain {
    	inbound := iptables.Chain{
    		Name:  PolicyChainName(PolicyInboundPfx, policyID),
    		Rules: r.ProtoRulesToIptablesRules(policy.InboundRules, ipVersion),
    	}
    	outbound := iptables.Chain{
    		Name:  PolicyChainName(PolicyOutboundPfx, policyID),
    		Rules: r.ProtoRulesToIptablesRules(policy.OutboundRules, ipVersion),
    	}
    	return []*iptables.Chain{&inbound, &outbound}
    }

###     3.1 ProtoRulesToIptablesRules ###

把 protobuf 中的 rules 转换为 iptables 规则

ProtoRuleToIptablesRules 函数第 4 章节讲解

func (r *DefaultRuleRenderer) ProtoRulesToIptablesRules(protoRules []*proto.Rule, ipVersion uint8) []iptables.Rule {
    	var rules []iptables.Rule
    	for _, protoRule := range protoRules {
    		rules = append(rules, r.ProtoRuleToIptablesRules(protoRule, ipVersion)...)
    	}
    	return rules
    }

# 4. ProtoRuleToIptablesRules #

定义在 fules/policy.go

func (r *DefaultRuleRenderer) ProtoRuleToIptablesRules(pRule *proto.Rule, ipVersion uint8) []iptables.Rule {
    	// Filter the CIDRs to the IP version that we're rendering.  In general, we should have an
    	// explicit IP version in the rule and all CIDRs should match it (and calicoctl, for
    	// example, enforces that).  However, we try to handle a rule gracefully if it's missing a
    	// version.
    	//
    	// We do that by rendering the rule, filtered to only have CIDRs of the right version,
    	// unless filtering the rule would completely remove one of its match fields.
    	//
    	// That handles the mainline case well, where the IP version is missing but the rule is
    	// otherwise consistent since we'll render the rule only for the matching version.
    	//
    	// It also handles rules like "allow from 10.0.0.1,feed::beef" in an intuitive way.  Only
    	// rules of the form "allow from 10.0.0.1,feed::beef to 10.0.0.2" will get filtered out,
    	// and only for IPv6, where there's no obvious meaning to the rule.

###     4.1  filterNets 函数滤出 src dst  ###

ruleCopy := *pRule
    var filteredAll bool
    ruleCopy.SrcNet, filteredAll = filterNets(pRule.SrcNet, ipVersion)
    if filteredAll {
    	return nil
    }
    ruleCopy.NotSrcNet, filteredAll = filterNets(pRule.NotSrcNet, ipVersion)
    if filteredAll {
    	return nil
    }
    ruleCopy.DstNet, filteredAll = filterNets(pRule.DstNet, ipVersion)
    if filteredAll {
    	return nil
    }
    ruleCopy.NotDstNet, filteredAll = filterNets(pRule.NotDstNet, ipVersion)
    if filteredAll {
    	return nil
    }

###     4.2 设置 ipset 的版本，ipv4 或者 ipv6 ###

// Port matches.  We only need to render blocks of ports if, in total, there's more than one
    // source or more than one destination match that needs to be or-ed together.
    //
    // Split the port list into blocks of 15, as per iptables limit and add in the number of
    // named ports.
    var ipSetConfig *ipsets.IPVersionConfig
    if ipVersion == 4 {
    	ipSetConfig = r.IPSetConfigV4
    } else {
    	ipSetConfig = r.IPSetConfigV6
    }

### raw 表 ###

\-A cali-pi-\_cFFazTSiRen9aUwjRK- -m comment --comment "cali:v5SbhhO1MPyqLDVI" -m set --match-set cali40s:jgMzSJZJuvIT4or8GqiyLzi src -j MARK --set-xmark 0x10000/0x10000  
\-A cali-pi-\_cFFazTSiRen9aUwjRK- -m comment --comment "cali:OAf2bqXafsnqxifk" -m mark --mark 0x10000/0x10000 -j RETURN

### mangle 表 ###

### filter 表 ###

\-A cali-tw-calia9114c9a515 -m comment --comment "cali:Sgd0lasnpaikkPpF" -m mark --mark 0x0/0x20000 -j cali-pi-\_cFFazTSiRen9aUwjRK-

### 参考： ###

**   Network Policies**

[https://kubernetes.io/docs/concepts/services-networking/network-policies/][https_kubernetes.io_docs_concepts_services-networking_network-policies]

**  Securing Kubernetes Cluster Networking**

[https://ahmet.im/blog/kubernetes-network-policy/][https_ahmet.im_blog_kubernetes-network-policy]

### network policy 示例 ###

podSelector: 定义了哪些组的 pod 需要应用到该 policy，空的则是所有 namespace 下的 pod

policyTypes: `Ingress`, `Egress`

ingress: ingress 白名单规则

> apiVersion: projectcalico.org/v3  
> kind: NetworkPolicy  
> metadata:  
>   creationTimestamp: 2019-08-05T02:41:44Z  
>   name: knp.default.api-allow  
>   namespace: default  
>   resourceVersion: /2950231  
>   uid: 903c2678-b72a-11e9-a1f8-080027603363  
> spec:  
>   ingress:  
>   - action: Allow  
>     destination: \{\}  
>     source:  
>       selector: projectcalico.org/orchestrator == 'k8s' && app == 'bookstore'  
>   order: 1000  
>   selector: projectcalico.org/orchestrator == 'k8s' && app == 'iperf'  
>   types:  
>   - Ingress

> apiVersion: projectcalico.org/v3  
> kind: Profile  
> metadata:  
>   creationTimestamp: 2019-06-14T02:11:10Z  
>   name: kns.default  
>   resourceVersion: 1125917/  
>   uid: ad9acab7-8e49-11e9-bee1-080027603363  
> spec:  
>   egress:  
>   - action: Allow  
>     destination: \{\}  
>     source: \{\}  
>   ingress:  
>   - action: Allow  
>     destination: \{\}  
>     source: \{\}  
>   labelsToApply:  
>     pcns.sidecar-injector: enabled

[Network Policy]: https://www.bookstack.cn/read/feiskyer-kubernetes-handbook/concepts-network-policy.md
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3pob25nbGluemhhbmc_size_16_color_FFFFFF_t_70]: /images/20211109/7124abd8beb8409e9cfd0efab7ee45d8.png
[https_www.bookstack.cn_read_feiskyer-kubernetes-handbook_plugins-network-policy.md]: https://www.bookstack.cn/read/feiskyer-kubernetes-handbook/plugins-network-policy.md
[https_kubernetes.io_docs_concepts_services-networking_network-policies]: https://kubernetes.io/docs/concepts/services-networking/network-policies/
[https_ahmet.im_blog_kubernetes-network-policy]: https://ahmet.im/blog/kubernetes-network-policy/