使用 NGINX Ingress
所有文档

          容器引擎 CCE

          使用 NGINX Ingress

          本文介绍使用 NGINX Ingress 作为 Ingress 的实现方式。

          使用场景

          Kubernetes Ingress 除了 CCE Ingress 之外,也可以使用社区的 NGINX Ingress

          使用限制

          集群版本
          新版 CCE 集群(集群 ID 以 cce- 为前缀)集群支持此功能。
          旧版 CCE 集群(集群 ID 以 c- 为前缀)集群不支持此功能。

          操作步骤

          首先部署 NGINX Ingress 相关组件,然后部署 NGINX Ingress 服务。

          部署 NGINX Ingress 相关组件

          部署 NGINX Ingress 相关服务需要使用Kubectl连接集群,并使用以下命令进行部署:

          # yaml文件内容见附录 ingress-nginx.yaml
          kubectl apply -f ingress-nginx.yaml

          部署 NGINX Ingress 服务

          对于未使用 IPVLAN 容器网卡类型的集群,按如下方式部署:

          1. 部署 CCE-LB-Controller。登录百度云控制台,进入『容器引擎CCE - Helm - Helm模板 - 百度云模板』页面,搜索cce-lb-controller,可看到如下页面

            helm.png

            点击右侧的『安装』,输入实例名称和部署集群,并点击确定,即可完成 CCE-LB-Controller 的部署。

            helminstall.png

          2. 部署 NGINX Ingress 服务

            # yaml文件内容见附录 ingress-nginx-service-normal.yaml
            kubectl apply -f ingress-nginx-service-normal.yaml

          对于使用 IPVLAN 容器网卡类型的集群,按如下方式部署:

          1. 部署 NGINX Ingress 服务

            # yaml文件内容见附录 ingress-nginx-service-ipvlan.yaml
            kubectl apply -f ingress-nginx-service-ipvlan.yaml

          部署完成之后即可使用 NGINX Ingress 。官方文档请参考 NGINX Ingress User Guide。示例 Ingress 配置如下:

          apiVersion: networking.k8s.io/v1beta1
          kind: Ingress
          metadata:
            name: ingress-demo-service
            annotations:
              # use the shared ingress-nginx
              kubernetes.io/ingress.class: "nginx"
          spec:
            rules:
            - host: demo-service.foo.org
              http:
                paths:
                - path: /
                  backend:
                    serviceName: demo-service
                    servicePort: 80

          附录:NGINX Ingress 相关 Yaml 文件

          • ingress-nginx.yaml
          apiVersion: v1
          kind: Namespace
          metadata:
            name: ingress-nginx
            labels:
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
          
          ---
          # Source: ingress-nginx/templates/controller-serviceaccount.yaml
          apiVersion: v1
          kind: ServiceAccount
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx
            namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/controller-configmap.yaml
          apiVersion: v1
          kind: ConfigMap
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx-controller
            namespace: ingress-nginx
          data:
          ---
          # Source: ingress-nginx/templates/clusterrole.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRole
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
            name: ingress-nginx
          rules:
            - apiGroups:
                - ''
              resources:
                - configmaps
                - endpoints
                - nodes
                - pods
                - secrets
              verbs:
                - list
                - watch
            - apiGroups:
                - ''
              resources:
                - nodes
              verbs:
                - get
            - apiGroups:
                - ''
              resources:
                - services
              verbs:
                - get
                - list
                - update
                - watch
            - apiGroups:
                - extensions
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingresses
              verbs:
                - get
                - list
                - watch
            - apiGroups:
                - ''
              resources:
                - events
              verbs:
                - create
                - patch
            - apiGroups:
                - extensions
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingresses/status
              verbs:
                - update
            - apiGroups:
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingressclasses
              verbs:
                - get
                - list
                - watch
          ---
          # Source: ingress-nginx/templates/clusterrolebinding.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRoleBinding
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
            name: ingress-nginx
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: ClusterRole
            name: ingress-nginx
          subjects:
            - kind: ServiceAccount
              name: ingress-nginx
              namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/controller-role.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: Role
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx
            namespace: ingress-nginx
          rules:
            - apiGroups:
                - ''
              resources:
                - namespaces
              verbs:
                - get
            - apiGroups:
                - ''
              resources:
                - configmaps
                - pods
                - secrets
                - endpoints
              verbs:
                - get
                - list
                - watch
            - apiGroups:
                - ''
              resources:
                - services
              verbs:
                - get
                - list
                - update
                - watch
            - apiGroups:
                - extensions
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingresses
              verbs:
                - get
                - list
                - watch
            - apiGroups:
                - extensions
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingresses/status
              verbs:
                - update
            - apiGroups:
                - networking.k8s.io   # k8s 1.14+
              resources:
                - ingressclasses
              verbs:
                - get
                - list
                - watch
            - apiGroups:
                - ''
              resources:
                - configmaps
              resourceNames:
                - ingress-controller-leader-nginx
              verbs:
                - get
                - update
            - apiGroups:
                - ''
              resources:
                - configmaps
              verbs:
                - create
            - apiGroups:
                - ''
              resources:
                - endpoints
              verbs:
                - create
                - get
                - update
            - apiGroups:
                - ''
              resources:
                - events
              verbs:
                - create
                - patch
          ---
          # Source: ingress-nginx/templates/controller-rolebinding.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx
            namespace: ingress-nginx
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: Role
            name: ingress-nginx
          subjects:
            - kind: ServiceAccount
              name: ingress-nginx
              namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/controller-service-webhook.yaml
          apiVersion: v1
          kind: Service
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx-controller-admission
            namespace: ingress-nginx
          spec:
            type: ClusterIP
            ports:
              - name: https-webhook
                port: 443
                targetPort: webhook
            selector:
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/component: controller
          ---
          # Source: ingress-nginx/templates/controller-deployment.yaml
          apiVersion: apps/v1
          kind: Deployment
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx-controller
            namespace: ingress-nginx
          spec:
            selector:
              matchLabels:
                app.kubernetes.io/name: ingress-nginx
                app.kubernetes.io/instance: ingress-nginx
                app.kubernetes.io/component: controller
            revisionHistoryLimit: 10
            minReadySeconds: 0
            template:
              metadata:
                labels:
                  app.kubernetes.io/name: ingress-nginx
                  app.kubernetes.io/instance: ingress-nginx
                  app.kubernetes.io/component: controller
              spec:
                dnsPolicy: ClusterFirst
                containers:
                  - name: controller
                    image: registry.baidubce.com/cce-plugin-pro/ingress-nginx-controller:v0.43
                    imagePullPolicy: IfNotPresent
                    lifecycle:
                      preStop:
                        exec:
                          command:
                            - /wait-shutdown
                    args:
                      - /nginx-ingress-controller
                      - --election-id=ingress-controller-leader
                      - --ingress-class=nginx
                      - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
                      - --validating-webhook=:8443
                      - --validating-webhook-certificate=/usr/local/certificates/cert
                      - --validating-webhook-key=/usr/local/certificates/key
                    securityContext:
                      capabilities:
                        drop:
                          - ALL
                        add:
                          - NET_BIND_SERVICE
                      runAsUser: 101
                      allowPrivilegeEscalation: true
                    env:
                      - name: POD_NAME
                        valueFrom:
                          fieldRef:
                            fieldPath: metadata.name
                      - name: POD_NAMESPACE
                        valueFrom:
                          fieldRef:
                            fieldPath: metadata.namespace
                      - name: LD_PRELOAD
                        value: /usr/local/lib/libmimalloc.so
                    livenessProbe:
                      httpGet:
                        path: /healthz
                        port: 10254
                        scheme: HTTP
                      initialDelaySeconds: 10
                      periodSeconds: 10
                      timeoutSeconds: 1
                      successThreshold: 1
                      failureThreshold: 5
                    readinessProbe:
                      httpGet:
                        path: /healthz
                        port: 10254
                        scheme: HTTP
                      initialDelaySeconds: 10
                      periodSeconds: 10
                      timeoutSeconds: 1
                      successThreshold: 1
                      failureThreshold: 3
                    ports:
                      - name: http
                        containerPort: 80
                        protocol: TCP
                      - name: https
                        containerPort: 443
                        protocol: TCP
                      - name: webhook
                        containerPort: 8443
                        protocol: TCP
                    volumeMounts:
                      - name: webhook-cert
                        mountPath: /usr/local/certificates/
                        readOnly: true
                    resources:
                      requests:
                        cpu: 100m
                        memory: 90Mi
                nodeSelector:
                  kubernetes.io/os: linux
                serviceAccountName: ingress-nginx
                terminationGracePeriodSeconds: 300
                volumes:
                  - name: webhook-cert
                    secret:
                      secretName: ingress-nginx-admission
          ---
          # Source: ingress-nginx/templates/admission-webhooks/validating-webhook.yaml
          # before changing this value, check the required kubernetes version
          # https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites
          apiVersion: admissionregistration.k8s.io/v1
          kind: ValidatingWebhookConfiguration
          metadata:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            name: ingress-nginx-admission
          webhooks:
            - name: validate.nginx.ingress.kubernetes.io
              matchPolicy: Equivalent
              rules:
                - apiGroups:
                    - networking.k8s.io
                  apiVersions:
                    - v1beta1
                  operations:
                    - CREATE
                    - UPDATE
                  resources:
                    - ingresses
              failurePolicy: Fail
              sideEffects: None
              admissionReviewVersions:
                - v1
                - v1beta1
              clientConfig:
                service:
                  namespace: ingress-nginx
                  name: ingress-nginx-controller-admission
                  path: /networking/v1beta1/ingresses
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/serviceaccount.yaml
          apiVersion: v1
          kind: ServiceAccount
          metadata:
            name: ingress-nginx-admission
            annotations:
              helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrole.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRole
          metadata:
            name: ingress-nginx-admission
            annotations:
              helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
          rules:
            - apiGroups:
                - admissionregistration.k8s.io
              resources:
                - validatingwebhookconfigurations
              verbs:
                - get
                - update
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/clusterrolebinding.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: ClusterRoleBinding
          metadata:
            name: ingress-nginx-admission
            annotations:
              helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: ClusterRole
            name: ingress-nginx-admission
          subjects:
            - kind: ServiceAccount
              name: ingress-nginx-admission
              namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/role.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: Role
          metadata:
            name: ingress-nginx-admission
            annotations:
              helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            namespace: ingress-nginx
          rules:
            - apiGroups:
                - ''
              resources:
                - secrets
              verbs:
                - get
                - create
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/rolebinding.yaml
          apiVersion: rbac.authorization.k8s.io/v1
          kind: RoleBinding
          metadata:
            name: ingress-nginx-admission
            annotations:
              helm.sh/hook: pre-install,pre-upgrade,post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            namespace: ingress-nginx
          roleRef:
            apiGroup: rbac.authorization.k8s.io
            kind: Role
            name: ingress-nginx-admission
          subjects:
            - kind: ServiceAccount
              name: ingress-nginx-admission
              namespace: ingress-nginx
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml
          apiVersion: batch/v1
          kind: Job
          metadata:
            name: ingress-nginx-admission-create
            annotations:
              helm.sh/hook: pre-install,pre-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            namespace: ingress-nginx
          spec:
            template:
              metadata:
                name: ingress-nginx-admission-create
                labels:
                  helm.sh/chart: ingress-nginx-3.19.0
                  app.kubernetes.io/name: ingress-nginx
                  app.kubernetes.io/instance: ingress-nginx
                  app.kubernetes.io/version: 0.43.0
                  app.kubernetes.io/managed-by: Helm
                  app.kubernetes.io/component: admission-webhook
              spec:
                containers:
                  - name: create
                    image: registry.baidubce.com/cce-plugin-pro/kube-webhook-certgen:v1.5.0
                    imagePullPolicy: IfNotPresent
                    args:
                      - create
                      - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
                      - --namespace=$(POD_NAMESPACE)
                      - --secret-name=ingress-nginx-admission
                    env:
                      - name: POD_NAMESPACE
                        valueFrom:
                          fieldRef:
                            fieldPath: metadata.namespace
                restartPolicy: OnFailure
                serviceAccountName: ingress-nginx-admission
                securityContext:
                  runAsNonRoot: true
                  runAsUser: 2000
          ---
          # Source: ingress-nginx/templates/admission-webhooks/job-patch/job-patchWebhook.yaml
          apiVersion: batch/v1
          kind: Job
          metadata:
            name: ingress-nginx-admission-patch
            annotations:
              helm.sh/hook: post-install,post-upgrade
              helm.sh/hook-delete-policy: before-hook-creation,hook-succeeded
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: admission-webhook
            namespace: ingress-nginx
          spec:
            template:
              metadata:
                name: ingress-nginx-admission-patch
                labels:
                  helm.sh/chart: ingress-nginx-3.19.0
                  app.kubernetes.io/name: ingress-nginx
                  app.kubernetes.io/instance: ingress-nginx
                  app.kubernetes.io/version: 0.43.0
                  app.kubernetes.io/managed-by: Helm
                  app.kubernetes.io/component: admission-webhook
              spec:
                containers:
                  - name: patch
                    image: registry.baidubce.com/cce-plugin-pro/kube-webhook-certgen:v1.5.0
                    imagePullPolicy: IfNotPresent
                    args:
                      - patch
                      - --webhook-name=ingress-nginx-admission
                      - --namespace=$(POD_NAMESPACE)
                      - --patch-mutating=false
                      - --secret-name=ingress-nginx-admission
                      - --patch-failure-policy=Fail
                    env:
                      - name: POD_NAMESPACE
                        valueFrom:
                          fieldRef:
                            fieldPath: metadata.namespace
                restartPolicy: OnFailure
                serviceAccountName: ingress-nginx-admission
                securityContext:
                  runAsNonRoot: true
                  runAsUser: 2000
          • ingress-nginx-service-normal.yaml
          # Source: ingress-nginx/templates/controller-service.yaml
          apiVersion: v1
          kind: Service
          metadata:
            annotations:
              service.beta.kubernetes.io/cce-load-balancer-backend-type: "eni"
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx-controller
            namespace: ingress-nginx
          spec:
            type: LoadBalancer
            ports:
              - name: http
                port: 80
                protocol: TCP
                targetPort: 80
              - name: https
                port: 443
                protocol: TCP
                targetPort: 443
            selector:
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/component: controller
          • ingress-nginx-service-ipvlan.yaml
          # Source: ingress-nginx/templates/controller-service.yaml
          apiVersion: v1
          kind: Service
          metadata:
            annotations:
            labels:
              helm.sh/chart: ingress-nginx-3.19.0
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/version: 0.43.0
              app.kubernetes.io/managed-by: Helm
              app.kubernetes.io/component: controller
            name: ingress-nginx-controller
            namespace: ingress-nginx
          spec:
            type: LoadBalancer
            ports:
              - name: http
                port: 80
                protocol: TCP
                targetPort: 80
              - name: https
                port: 443
                protocol: TCP
                targetPort: 443
            selector:
              app.kubernetes.io/name: ingress-nginx
              app.kubernetes.io/instance: ingress-nginx
              app.kubernetes.io/component: controller
          上一篇
          CCE 基于 nginx-ingress实现灰度发布
          下一篇
          网络管理