通过PV/PVC方式使用CDS

概述

容器的特性决定了容器本身是非持久化的,容器被删除,其上的数据也一并删除。cds可以解决容器的数据共享和持久化存储问题,适用于大数据分析、媒体、游戏等很多场景。
而PV(PersistentVolume)和PVC(PersistentVolumeClaim)是K8S提供的用于抽象存储细节的API资源。利用cds在集群内创建PV和PVC资源,用户可以直接将cds作为存储卷挂载到容器中,而无需关注底层的实现细节,从而更加便捷地为容器集群提供持久化存储方案。

准备工作

用户做好以下准备,才能在容器中挂载cds实例。

  • 注册百度智能云账号,并完成实名认证

  • 创建一个可用的容器集群。

注意:
需要k8s版本>=1.11

创建容器集群

  1. 创建一个容器集群,操作步骤参考创建集群
  2. 下载命令行客户端kubectl,并连接集群,操作步骤参考通过kubectl连接Kubernetes集群

操作指南

通过PV/PVC方式使用CDS,具体分为两种方式:

  • 静态挂载:需要用户提前在百度智能云中创建好CDS(操作方法参考CDS文档),然后通过CDS的volume id在集群中创建PV和PVC资源。
  • 动态挂载:用户在集群中声明PVC时,自动创建出CDS磁盘并且动态关联至PV。

静态挂载的方式需要用户提供的CDS磁盘处于未挂载状态,并且确保填入正确的volume id。动态挂载在部分场景下可能会更加方便,但是需要注意动态挂载时将会自动创建CDS磁盘并且计费,动态创建的CDS将会随着用户删除PVC时被联动删除(需要将对应storageClasss的reclaimPolicy参数设置为delete),但是如果用户在没有删除PVC的情况下直接删除了CCE集群,则需要手动对这些CDS进行处理。

静态PV/PVC方式挂载cds

1.创建CdsPlugin、Attacher

cds-template.yaml是一个yaml文件模板,包含了需要创建的集群资源信息,使用kubectl apply -f cds-template.yaml进行相关组件部署。

cds-template.yaml文件内容如下(1.11集群使用):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-external-runner
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: external-runner
rules:
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "update", "create", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-binding
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: external-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: kube-system
  name: external-runner-cfg
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "watch", "list", "delete", "update", "create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-cfg
  namespace: kube-system
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: Role
  name: external-runner-cfg
  apiGroup: rbac.authorization.k8s.io
---
kind: DaemonSet
apiVersion: apps/v1beta2
metadata:
  name: csi-cdsplugin
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: csi-cdsplugin
  template:
    metadata:
      labels:
        app: csi-cdsplugin
    spec:
      serviceAccount: csi-external-runner
      hostNetwork: true
      containers:
        - name: driver-registrar
          image: hub.baidubce.com/jpaas-public/driver-registrar:v0.3.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          imagePullPolicy: "Always"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
        - name: csi-cdsplugin
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
            allowPrivilegeEscalation: true
          image: hub.baidubce.com/jpaas-public/cdsplugin:latest
          args :
            - "--nodeid=$(NODE_ID)"
            - "--endpoint=$(CSI_ENDPOINT)"
            - "--v=5"
            - "--drivername=csi-cdsplugin"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix://var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: cloud-config
              mountPath: /etc/kubernetes
            - name: plugin-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
            - name: pods-mount-dir
              mountPath: /var/lib/kubelet/pods
              mountPropagation: "Bidirectional"
            - name: pods-mount-dir-mnt
              mountPath: /mnt/kubelet/pods
              mountPropagation: "Bidirectional"
            - name: pods-mount-dir-data
              mountPath: /data/kubelet/pods
              mountPropagation: "Bidirectional"
            - mountPath: /sys
              name: host-sys
            - name: lib-modules
              mountPath: /lib/modules
              readOnly: true
            - name: host-dev
              mountPath: /dev
      volumes:
        - name: cloud-config
          hostPath:
            path: /etc/kubernetes
        - name: plugin-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate
        - name: pods-mount-dir
          hostPath:
            path: /var/lib/kubelet/pods
            type: Directory
        - name: pods-mount-dir-data
          hostPath:
            path: /data/kubelet/pods
            type: DirectoryOrCreate
        - name: pods-mount-dir-mnt
          hostPath:
            path: /mnt/kubelet/pods
            type: DirectoryOrCreate
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate
        - name: host-sys
          hostPath:
            path: /sys
        - name: lib-modules
          hostPath:
            path: /lib/modules
        - name: host-dev
          hostPath:
            path: /dev
---
kind: Service
apiVersion: v1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
  labels:
    app: csi-attacher-cds
spec:
  selector:
    app: csi-attacher-cds
  clusterIP: None
---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
spec:
  serviceName: "csi-attacher-cds"
  replicas: 2
  template:
    metadata:
      labels:
        app: csi-attacher-cds
    spec:
      serviceAccount: csi-external-runner
      containers:
        - name: csi-attacher-cds
          image: hub.baidubce.com/jpaas-public/csi-attacher:v0.3.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
            - "--leader-election"
            - "--leader-election-namespace=$(MY_NAMESPACE)"
            - "--leader-election-identity=$(MY_NAME)"
          env:
            - name: MY_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate

cds-template.yaml文件内容如下(1.13集群使用):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-external-runner
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: external-runner
rules:
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "update", "create", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["snapshot.storage.k8s.io"]
    resources: ["*"]
    verbs: ["create", "get", "list", "watch", "update", "patch"]
  - apiGroups: ["apiextensions.k8s.io"]
    resources: ["*"]
    verbs: ["create", "get", "list", "watch"]
  - apiGroups: ["csi.storage.k8s.io"]
    resources: ["csidrivers"]
    verbs: ["create", "delete", "get", "list", "watch"]
  - apiGroups: ["csi.storage.k8s.io"]
    resources: ["csinodeinfos"]
    verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-binding
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: external-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: kube-system
  name: external-runner-cfg
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "watch", "list", "delete", "update", "create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-cfg
  namespace: kube-system
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: Role
  name: external-runner-cfg
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: csi-cdsplugin
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: csi-cdsplugin
  template:
    metadata:
      labels:
        app: csi-cdsplugin
    spec:
      containers:
      - args:
        - --v=5
        - --csi-address=$(ADDRESS)
        - --kubelet-registration-path=$(ADDRESS)
        env:
        - name: ADDRESS
          value: /var/lib/kubelet/plugins_registry/csi-cdsplugin/csi.sock
        - name: KUBE_NODE_NAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        image: hub.baidubce.com/jpaas-public/driver-registrar:v1.0.1
        imagePullPolicy: Always
        name: driver-registrar
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /var/lib/kubelet/plugins_registry/csi-cdsplugin
          name: socket-dir
        - mountPath: /registration
          name: reg-dir
      - args:
        - --nodeid=$(NODE_ID)
        - --endpoint=$(CSI_ENDPOINT)
        - --v=5
        - --drivername=csi-cdsplugin
        env:
        - name: NODE_ID
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        - name: CSI_ENDPOINT
          value: unix://var/lib/kubelet/plugins_registry/csi-cdsplugin/csi.sock
        image: hub.baidubce.com/jpaas-public/cdsplugin:v1.1.0.1
        imagePullPolicy: Always
        name: csi-cdsplugin
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - SYS_ADMIN
          privileged: true
        volumeMounts:
        - mountPath: /etc/kubernetes
          name: cloud-config
        - mountPath: /var/lib/kubelet/plugins_registry/csi-cdsplugin
          name: socket-dir
        - mountPath: /var/lib/kubelet/pods
          mountPropagation: Bidirectional
          name: pods-mount-dir
        - mountPath: /data/kubelet/pods
          mountPropagation: Bidirectional
          name: pods-mount-dir-data
        - mountPath: /sys
          name: host-sys
        - mountPath: /lib/modules
          name: lib-modules
          readOnly: true
        - mountPath: /dev
          name: host-dev
      hostNetwork: true
      restartPolicy: Always
      serviceAccount: csi-external-runner
      serviceAccountName: csi-external-runner
      priorityClassName: system-node-critical
      volumes:
      - hostPath:
          path: /etc/kubernetes
          type: ""
        name: cloud-config
      - hostPath:
          path: /var/lib/kubelet/plugins_registry/csi-cdsplugin
          type: DirectoryOrCreate
        name: reg-dir
      - hostPath:
          path: /var/lib/kubelet/pods
          type: Directory
        name: pods-mount-dir
      - hostPath:
          path: /data/kubelet/pods
          type: Directory
        name: pods-mount-dir-data
      - hostPath:
          path: /var/lib/kubelet/plugins_registry/csi-cdsplugin
          type: DirectoryOrCreate
        name: socket-dir
      - hostPath:
          path: /sys
          type: ""
        name: host-sys
      - hostPath:
          path: /lib/modules
          type: ""
        name: lib-modules
      - hostPath:
          path: /dev
          type: ""
        name: host-dev
---
kind: Service
apiVersion: v1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
  labels:
    app: csi-attacher-cds
spec:
  selector:
    app: csi-attacher-cds
  clusterIP: None
---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
spec:
  serviceName: "csi-attacher-cds"
  replicas: 2
  template:
    metadata:
      labels:
        app: csi-attacher-cds
    spec:
      priorityClassName: system-cluster-critical
      serviceAccount: csi-external-runner
      containers:
        - name: csi-attacher-cds
          image: hub.baidubce.com/jpaas-public/csi-attacher:v1.0.1
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
            - "--leader-election"
            - "--leader-election-namespace=$(MY_NAMESPACE)"
            - "--leader-election-identity=$(MY_NAME)"
          env:
            - name: MY_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: ADDRESS
              value: /var/lib/kubelet/plugins_registry/csi-cdsplugin/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins_registry/csi-cdsplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins_registry/csi-cdsplugin
            type: DirectoryOrCreate

2.在集群中创建PV和PVC资源

使用kubectl,执行 kubectl create -f pv-cds.yaml 完成PV的创建

对应的pv-cds.yaml文件如下所示:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-cds
  namespace: "default"
spec:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 5Gi
  csi:
    driver: "csi-cdsplugin"
    volumeHandle: "v-xxxx"
  persistentVolumeReclaimPolicy: Retain

注意: yaml中volumeHandle字段对应的是cds volume id(短 id)

创建PV后,输入kubectl get pv可以看见一个available状态的PV,如下所示:

$ kubectl get pv
NAME                   CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM                     STORAGECLASS   REASON    AGE
pv-cds                 5Gi        RWO            Retain           Available                                                      22s

建立一个能够与该PV绑定的PVC

使用kubectl,执行 kubectl create -f pvc-cds.yaml完成PVC的创建

对应的pvc-cds.yaml文件如下所示:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: csi-cds-pvc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

绑定前,PVC为pending状态

$ kubectl get pvc
NAME      STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
cds-pvc   Pending                                                      2s                                                  2s

绑定后,PV和PVC状态变为Bound

$ kubectl get pv
NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                  STORAGECLASS   REASON    AGE
pv-cds    5Gi        RWO            Retain           Bound     default/csi-cds-pvc                             4s
$ kubectl get pvc
NAME           STATUS    VOLUME    CAPACITY   ACCESS MODES   STORAGECLASS   AGE
csi-cds-pvc    Bound     pv-cds    5Gi        RWO                           36s

有关PV和PVC的更多设置和字段说明,见k8s官方文档

3.在Pod内挂载PVC

在Pod spec内指定相应的PVC名称即可,使用kubectl,执行 kubectl create -f demo-cds-rc.yaml 完成rc的创建

对应的demo-cds-rs.yaml文件如下所示:

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: web-server-rs
  labels:
    test: ws
spec:
  replicas: 1
  selector:
    matchLabels:
      se: ws
  template:
    metadata:
      labels:
        se: ws
      name: web-server
    spec:
      containers:
       - name: web-server
         image: nginx
         volumeMounts:
           - mountPath: /var/lib/www/html
             name: csi-cds-pvc
      volumes:
       - name: csi-cds-pvc
         persistentVolumeClaim:
           claimName: csi-cds-pvc

Pod创建后,可以读写容器内的/cds-volume路径来访问相应的cds存储上的内容。

由于创建PV和PVC时只支持accessModesReadWriteOnce,该PVC可以被一节点上的Pod挂载读写。

4.释放PV和PVC资源

完成存储资源的使用后,可以释放PVC和PV资源

使用以下命令可以释放PVC

$ kubectl delete -f  pvc-cds.yaml

释放PVC后,原来与之绑定的PV状态会变为Release,如下所示:

NAME      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                  STORAGECLASS   REASON    AGE
pv-cds    5Gi        RWO            Retain           Released   default/csi-cds-pvc                             4m

输入以下指令释放PV资源

$ kubectl delete -f  pv-cds.yaml

动态PV/PVC方式挂载cds

1.创建StorageClass、Attacher、Provisioner

dynamic-cds-template.yaml是一个yaml文件模板,包含了需要创建的集群资源信息。

dynamic-cds-template.yaml文件内容如下(1.11集群):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: csi-external-runner
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: external-runner
rules:
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["get", "list", "watch", "update", "create", "patch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "update", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["volumeattachments"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: [""]
    resources: ["namespaces"]
    verbs: ["get", "list"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-binding
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: ClusterRole
  name: external-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: kube-system
  name: external-runner-cfg
rules:
- apiGroups: [""]
  resources: ["configmaps"]
  verbs: ["get", "watch", "list", "delete", "update", "create"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: csi-role-cfg
  namespace: kube-system
subjects:
  - kind: ServiceAccount
    name: csi-external-runner
    namespace: kube-system
roleRef:
  kind: Role
  name: external-runner-cfg
  apiGroup: rbac.authorization.k8s.io
---
kind: DaemonSet
apiVersion: apps/v1beta2
metadata:
  name: csi-cdsplugin
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: csi-cdsplugin
  template:
    metadata:
      labels:
        app: csi-cdsplugin
    spec:
      serviceAccount: csi-external-runner
      hostNetwork: true
      containers:
        - name: driver-registrar
          image: hub.baidubce.com/jpaas-public/driver-registrar:v0.3.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
            - name: KUBE_NODE_NAME
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
          imagePullPolicy: "Always"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
        - name: csi-cdsplugin
          securityContext:
            privileged: true
            capabilities:
              add: ["SYS_ADMIN"]
            allowPrivilegeEscalation: true
          image: hub.baidubce.com/jpaas-public/cdsplugin:latest
          args :
            - "--nodeid=$(NODE_ID)"
            - "--endpoint=$(CSI_ENDPOINT)"
            - "--v=5"
            - "--drivername=csi-cdsplugin"
          env:
            - name: NODE_ID
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: CSI_ENDPOINT
              value: unix://var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: cloud-config
              mountPath: /etc/kubernetes
            - name: plugin-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
            - name: pods-mount-dir
              mountPath: /var/lib/kubelet/pods
              mountPropagation: "Bidirectional"
            - name: pods-mount-dir-data
              mountPath: /data/kubelet/pods
              mountPropagation: "Bidirectional"
            - mountPath: /sys
              name: host-sys
            - name: lib-modules
              mountPath: /lib/modules
              readOnly: true
            - name: host-dev
              mountPath: /dev
      volumes:
        - name: cloud-config
          hostPath:
            path: /etc/kubernetes
        - name: plugin-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate
        - name: pods-mount-dir
          hostPath:
            path: /var/lib/kubelet/pods
            type: Directory
        - name: pods-mount-dir-data
          hostPath:
            path: /data/kubelet/pods
            type: DirectoryOrCreate
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate
        - name: host-sys
          hostPath:
            path: /sys
        - name: lib-modules
          hostPath:
            path: /lib/modules
        - name: host-dev
          hostPath:
            path: /dev
---
kind: Service
apiVersion: v1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
  labels:
    app: csi-attacher-cds
spec:
  selector:
    app: csi-attacher-cds
  clusterIP: None
---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-attacher-cds
  namespace: kube-system
spec:
  serviceName: "csi-attacher-cds"
  replicas: 2
  template:
    metadata:
      labels:
        app: csi-attacher-cds
    spec:
      serviceAccount: csi-external-runner
      containers:
        - name: csi-attacher-cds
          image: hub.baidubce.com/jpaas-public/csi-attacher:v0.3.0
          args:
            - "--v=5"
            - "--csi-address=$(ADDRESS)"
            - "--leader-election"
            - "--leader-election-namespace=$(MY_NAMESPACE)"
            - "--leader-election-identity=$(MY_NAME)"
          env:
            - name: MY_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: MY_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
          imagePullPolicy: "Always"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: DirectoryOrCreate

---

kind: Service
apiVersion: v1
metadata:
  name: csi-provisioner-cds
  namespace: kube-system
  labels:
    app: csi-provisioner-cds
spec:
  selector:
    app: csi-provisioner-cds
  clusterIP: None

---
kind: StatefulSet
apiVersion: apps/v1beta1
metadata:
  name: csi-provisioner-cds
  namespace: kube-system
spec:
  serviceName: "csi-provisioner-cds"
  replicas: 1
  template:
    metadata:
      labels:
        app: csi-provisioner-cds
    spec:
      serviceAccount: csi-external-runner
      containers:
        - name: csi-provisioner-cds
          image: hub.baidubce.com/jpaas-public/csi-provisioner:v0.3.0
          args:
            - "--provisioner=csi-cdsplugin"
            - "--csi-address=$(ADDRESS)"
            - "--v=5"
          env:
            - name: ADDRESS
              value: /var/lib/kubelet/plugins/csi-cdsplugin/csi.sock
          imagePullPolicy: "IfNotPresent"
          volumeMounts:
            - name: socket-dir
              mountPath: /var/lib/kubelet/plugins/csi-cdsplugin
      volumes:
        - name: socket-dir
          hostPath:
            path: /var/lib/kubelet/plugins/csi-cdsplugin
            type: D