所有文档

          容器引擎 CCE

          在CCE集群中使用NetworkPolicy

          NetworkPolicy 是 K8S 提供的一种资源,用于定义基于 Pod 的网络隔离策略。它描述了一组 Pod 能否与其它组 Pod 及其它 Endpoints 进行通信。本文主要演示如何使用开源工具 kube-router 在 CCE 上实现 NetworkPolicy 功能.

          kube-router

          kube-router 是一个 kubernetes 的容器网络解决方案,它的官网和代码地址如下:

          kube-router 有三大功能:

          • Pod Networking;
          • IPVS/LVS based service proxy;
          • Network Policy Controller.

          CCE 有自己的容器网络实现方案,本文主要使用 kube-router 的 Network Policy Controller 的功能.

          部署 kube-router

          在 CCE K8S 集群上部署 kube-router ,YAML 如下:

          apiVersion: v1
          kind: ConfigMap
          metadata:
            name: kube-router-cfg
            namespace: kube-system
            labels:
              tier: node
              k8s-app: kube-router
          data:
            cni-conf.json: |
              {
                "name":"kubernetes",
                "type":"bridge",
                "bridge":"kube-bridge",
                "isDefaultGateway":true,
                "ipam": {
                  "type":"host-local"
                }
              }
          ---
          apiVersion: apps/v1
          kind: DaemonSet
          metadata:
            name: kube-router
            namespace: kube-system
            labels:
              k8s-app: kube-router
          spec:
            selector:
              matchLabels:
                k8s-app: kube-router
            template:
              metadata:
                labels:
                  k8s-app: kube-router
                annotations:
                  scheduler.alpha.kubernetes.io/critical-pod: ''
              spec:
                containers:
                - name: kube-router
                  image: docker.io/cloudnativelabs/kube-router
                  args: ["--run-router=false", "--run-firewall=true", "--run-service-proxy=false", "--kubeconfig=/root/.kube/config"]
                  securityContext:
                    privileged: true
                  imagePullPolicy: Always
                  env:
                  - name: NODE_NAME
                    valueFrom:
                      fieldRef:
                        fieldPath: spec.nodeName
                  livenessProbe:
                    httpGet:
                      path: /healthz
                      port: 20244
                    initialDelaySeconds: 10
                    periodSeconds: 3
                  volumeMounts:
                  - name: lib-modules
                    mountPath: /lib/modules
                    readOnly: true
                  - name: cni-conf-dir
                    mountPath: /etc/cni/net.d
                  - name: kubeconfig
                    mountPath: /root/.kube/config
                    readOnly: true
                initContainers:
                - name: install-cni
                  image: busybox
                  imagePullPolicy: Always
                  command:
                  - /bin/sh
                  - -c
                  - set -e -x;
                    if [ ! -f /etc/cni/net.d/10-kuberouter.conf ]; then
                      TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
                      cp /etc/kube-router/cni-conf.json ${TMP};
                      mv ${TMP} /etc/cni/net.d/10-kuberouter.conf;
                    fi
                  volumeMounts:
                  - name: cni-conf-dir
                    mountPath: /etc/cni/net.d
                  - name: kube-router-cfg
                    mountPath: /etc/kube-router
                hostNetwork: true
                tolerations:
                - key: CriticalAddonsOnly
                  operator: Exists
                - effect: NoSchedule
                  key: node-role.kubernetes.io/master
                  operator: Exists
                - effect: NoSchedule
                  key: node.kubernetes.io/not-ready
                  operator: Exists
                volumes:
                - name: lib-modules
                  hostPath:
                    path: /lib/modules
                - name: cni-conf-dir
                  hostPath:
                    path: /etc/cni/net.d
                - name: kube-router-cfg
                  configMap:
                    name: kube-router-cfg
                - name: kubeconfig
                  hostPath:
                    path: /root/.kube/config

          例子说明

          1 创建namespaces

          $kubectl create namespace production
          $kubectl create namespace staging

          2 启动 nginx 服务

          在不同的 namespace 中创建 nginx deployment.

          $kubectl apply -f nginx.yaml --namespace=production
          $kubectl apply -f nginx.yaml --namespace=staging

          nginx.yaml 的 YAML 如下:

          apiVersion: apps/v1
          kind: Deployment
          metadata:
            name: nginx-deployment
            labels:
              app: nginx
          spec:
            replicas: 3
            selector:
              matchLabels:
                app: nginx
            template:
              metadata:
                labels:
                  app: nginx
              spec:
                containers:
                - name: nginx
                  image: hub.baidubce.com/cce/nginx-alpine-go:latest
                  ports:
                  - containerPort: 80

          验证 Pod 启动成功:

          # staging 环境
          $kubectl get pods -n staging
          NAME                                READY     STATUS    RESTARTS   AGE
          nginx-deployment-7fbd5f4c55-2xgd4   1/1       Running   0          45s
          nginx-deployment-7fbd5f4c55-5xr75   1/1       Running   0          45s
          nginx-deployment-7fbd5f4c55-fn6lr   1/1       Running   0          20m
          
          # productionn 环境
          $kubectl get pods -n production
          NAME                                READY     STATUS    RESTARTS   AGE
          nginx-deployment-7fbd5f4c55-m764f   1/1       Running   0          10s
          nginx-deployment-7fbd5f4c55-pdhhz   1/1       Running   0          10s
          nginx-deployment-7fbd5f4c55-r98w5   1/1       Running   0          20m

          没有设置 NetworkPolicy 的时候,所有的 Pod 是可以相互访问的,可以直接 ping PodIP.

          Network Policy 策略测试

          1. Default deny all ingress traffic

          禁止 namespace=staging 中 pod 被访问.

          apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: default-deny
            namespace: staging
          spec:
            podSelector: {}
            policyTypes:
            - Ingress

          各个字段含义说明:

          • PodSelector:选中需要隔离的 Pod;
          • policyTypes: 策略类型,NetworkPolicy 将流量分为 ingress 和 egress,即入方向和出方向。如果没有指定则表示不闲置;
          • ingress:入方向,白名单,需要指定 from、ports,即来源、目的端口号,from有三种类型,ipBlock/namespaceSelector/podSelector;
          • egress:出方向,白名单,类似 ingress,egress 需要指定 to、ports,即目的、目的端口号。

          上述 NetworkPolicy 创建完成后,可以在任意 Pod 中访问 namespace=staging 下的 PodIP,发现是无法访问,比如从 production 中的 pod 进行访问 :

          $kubectl exec -it nginx-deployment-7fbd5f4c55-m764f /bin/sh -n production
          / # ping 172.16.0.92
          PING 172.16.0.92 (172.16.0.92): 56 data bytes

          2. Default allow all ingress traffic

          允许 namespace=staging 中 pod 被访问.

          apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: allow-all
            namespace: staging
          spec:
            podSelector: {}
            ingress:
            - {}
            policyTypes:
            - Ingress

          3. Default deny all egress traffic

          禁止 namespace=production 中 pod 对外访问.

          apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: default-deny
            namespace: production
          spec:
            podSelector: {}
            policyTypes:
            - Egress

          4. Default allow all egress traffic

          允许 namespace=production 中 pod 对外访问.

          apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: allow-all
            namespace: production
          spec:
            podSelector: {}
            egress:
            - {}
            policyTypes:
            - Egress

          5. Default deny all ingress and all egress traffic

          禁止所有 pod 的入和出的流量:

          apiVersion: networking.k8s.io/v1
          kind: NetworkPolicy
          metadata:
            name: default-deny
          spec:
            podSelector: {}
            policyTypes:
            - Ingress
            - Egress
          上一篇
          流量接入
          下一篇
          创建VPC-CNI模式集群