容器引擎CCE

    在CCE集群中使用NetworkPolicy

    NetworkPolicy 是 K8S 提供的一种资源,用于定义基于 Pod 的网络隔离策略。它描述了一组 Pod 能否与其它组 Pod 及其它 Endpoints 进行通信。本文主要演示如何使用开源工具 kube-router 在 CCE 上实现 NetworkPolicy 功能.

    kube-router

    kube-router 是一个 kubernetes 的容器网络解决方案,它的官网和代码地址如下:

    kube-router 有三大功能:

    • Pod Networking;
    • IPVS/LVS based service proxy;
    • Network Policy Controller.

    CCE 有自己的容器网络实现方案,本文主要使用 kube-router 的 Network Policy Controller 的功能.

    部署 kube-router

    在 CCE K8S 集群上部署 kube-router ,YAML 如下:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: kube-router-cfg
      namespace: kube-system
      labels:
        tier: node
        k8s-app: kube-router
    data:
      cni-conf.json: |
        {
          "name":"kubernetes",
          "type":"bridge",
          "bridge":"kube-bridge",
          "isDefaultGateway":true,
          "ipam": {
            "type":"host-local"
          }
        }
    ---
    apiVersion: extensions/v1beta1
    kind: DaemonSet
    metadata:
      name: kube-router
      namespace: kube-system
      labels:
        k8s-app: kube-router
    spec:
      template:
        metadata:
          labels:
            k8s-app: kube-router
          annotations:
            scheduler.alpha.kubernetes.io/critical-pod: ''
        spec:
          containers:
          - name: kube-router
            image: docker.io/cloudnativelabs/kube-router
            args: ["--run-router=false", "--run-firewall=true", "--run-service-proxy=false", "--kubeconfig=/root/.kube/config"]
            securityContext:
              privileged: true
            imagePullPolicy: Always
            env:
            - name: NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            livenessProbe:
              httpGet:
                path: /healthz
                port: 20244
              initialDelaySeconds: 10
              periodSeconds: 3
            volumeMounts:
            - name: lib-modules
              mountPath: /lib/modules
              readOnly: true
            - name: cni-conf-dir
              mountPath: /etc/cni/net.d
            - name: kubeconfig
              mountPath: /root/.kube/config
              readOnly: true
          initContainers:
          - name: install-cni
            image: busybox
            imagePullPolicy: Always
            command:
            - /bin/sh
            - -c
            - set -e -x;
              if [ ! -f /etc/cni/net.d/10-kuberouter.conf ]; then
                TMP=/etc/cni/net.d/.tmp-kuberouter-cfg;
                cp /etc/kube-router/cni-conf.json ${TMP};
                mv ${TMP} /etc/cni/net.d/10-kuberouter.conf;
              fi
            volumeMounts:
            - name: cni-conf-dir
              mountPath: /etc/cni/net.d
            - name: kube-router-cfg
              mountPath: /etc/kube-router
          hostNetwork: true
          tolerations:
          - key: CriticalAddonsOnly
            operator: Exists
          - effect: NoSchedule
            key: node-role.kubernetes.io/master
            operator: Exists
          - effect: NoSchedule
            key: node.kubernetes.io/not-ready
            operator: Exists
          volumes:
          - name: lib-modules
            hostPath:
              path: /lib/modules
          - name: cni-conf-dir
            hostPath:
              path: /etc/cni/net.d
          - name: kube-router-cfg
            configMap:
              name: kube-router-cfg
          - name: kubeconfig
            hostPath:
              path: /root/.kube/config

    例子说明

    1 创建namespaces

    $kubectl create namespace production
    $kubectl create namespace staging

    2 启动 nginx 服务

    在不同的 namespace 中创建 nginx deployment.

    $kubectl apply -f nginx.yaml --namespace=production
    $kubectl apply -f nginx.yaml --namespace=staging

    nginx.yaml 的 YAML 如下:

    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: nginx-deployment
      labels:
        app: nginx
    spec:
      replicas: 3
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: hub.baidubce.com/cce/nginx-alpine-go:latest
            ports:
            - containerPort: 80

    验证 Pod 启动成功:

    # staging 环境
    $kubectl get pods -n staging
    NAME                                READY     STATUS    RESTARTS   AGE
    nginx-deployment-7fbd5f4c55-2xgd4   1/1       Running   0          45s
    nginx-deployment-7fbd5f4c55-5xr75   1/1       Running   0          45s
    nginx-deployment-7fbd5f4c55-fn6lr   1/1       Running   0          20m
    
    # productionn 环境
    $kubectl get pods -n production
    NAME                                READY     STATUS    RESTARTS   AGE
    nginx-deployment-7fbd5f4c55-m764f   1/1       Running   0          10s
    nginx-deployment-7fbd5f4c55-pdhhz   1/1       Running   0          10s
    nginx-deployment-7fbd5f4c55-r98w5   1/1       Running   0          20m

    没有设置 NetworkPolicy 的时候,所有的 Pod 是可以相互访问的,可以直接 ping PodIP.

    Network Policy 策略测试

    1. Default deny all ingress traffic

    禁止 namespace=staging 中 pod 被访问.

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: default-deny
      namespace: staging
    spec:
      podSelector: {}
      policyTypes:
      - Ingress

    各个字段含义说明:

    • PodSelector:选中需要隔离的 Pod;
    • policyTypes: 策略类型,NetworkPolicy 将流量分为 ingress 和 egress,即入方向和出方向。如果没有指定则表示不闲置;
    • ingress:入方向,白名单,需要指定 from、ports,即来源、目的端口号,from有三种类型,ipBlock/namespaceSelector/podSelector;
    • egress:出方向,白名单,类似 ingress,egress 需要指定 to、ports,即目的、目的端口号。

    上述 NetworkPolicy 创建完成后,可以在任意 Pod 中访问 namespace=staging 下的 PodIP,发现是无法访问,比如从 production 中的 pod 进行访问 :

    $kubectl exec -it nginx-deployment-7fbd5f4c55-m764f /bin/sh -n production
    / # ping 172.16.0.92
    PING 172.16.0.92 (172.16.0.92): 56 data bytes

    2. Default allow all ingress traffic

    允许 namespace=staging 中 pod 被访问.

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: allow-all
      namespace: staging
    spec:
      podSelector: {}
      ingress:
      - {}
      policyTypes:
      - Ingress

    3. Default deny all egress traffic

    禁止 namespace=production 中 pod 对外访问.

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: default-deny
      namespace: production
    spec:
      podSelector: {}
      policyTypes:
      - Egress

    4. Default allow all egress traffic

    允许 namespace=production 中 pod 对外访问.

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: allow-all
      namespace: production
    spec:
      podSelector: {}
      egress:
      - {}
      policyTypes:
      - Egress

    5. Default deny all ingress and all egress traffic

    禁止所有 pod 的入和出的流量:

    apiVersion: networking.k8s.io/v1
    kind: NetworkPolicy
    metadata:
      name: default-deny
    spec:
      podSelector: {}
      policyTypes:
      - Ingress
      - Egress
    上一篇
    流量接入
    下一篇
    创建VPC-CNI模式集群