Я пытаюсь сделать так, чтобы модуль k8s мог использовать PMEM без использования privileged mode
. Я пытаюсь создать локальный PV поверх каталога fsdax с PVC в k8s и позволить моему модулю использовать его. Однако я всегда получаю ошибку MountVolume.NewMounter initialization failed ... : path does not exist
.
Вот мои файлы yaml и статус PMEM:
Класс хранения yaml:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
PV ямл:
apiVersion: v1
kind: PersistentVolume
metadata:
name: pmem-pv-volume
spec:
capacity:
storage: 50Gi
volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Delete
storageClassName: local-storage
local:
path: /mnt/pmem0/vol1
nodeAffinity:
required:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- pmem
Ямл ПВХ:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pmem-pv-claim
spec:
storageClassName: local-storage
volumeName: pmem-pv-volume
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
Под ямл:
apiVersion: v1
kind: Pod
metadata:
name: daemon
labels:
env: test
spec:
hostNetwork: true
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In
values:
- pmem
containers:
- name: daemon-container
command: ["/usr/bin/bash", "-c", "sleep 3600"]
image: mm:v2
imagePullPolicy: Never
volumeMounts:
- mountPath: /mnt/pmem
name: pmem-pv-storage
- mountPath: /tmp
name: tmp
- mountPath: /var/log/memverge
name: log
- mountPath: /var/memverge/data
name: data
volumes:
- name: pmem-pv-storage
persistentVolumeClaim:
claimName: pmem-pv-claim
- name: tmp
hostPath:
path: /tmp
- name: log
hostPath:
path: /var/log/memverge
- name: data
hostPath:
path: /var/memverge/data
Некоторые выходы status и k8s:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 745.2G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 740G 0 part
├─cl-root 253:0 0 188G 0 lvm /
├─cl-swap 253:1 0 32G 0 lvm [SWAP]
└─cl-home 253:2 0 520G 0 lvm /home
sr0 11:0 1 1024M 0 rom
nvme0n1 259:0 0 7T 0 disk
└─nvme0n1p1 259:1 0 7T 0 part /mnt/nvme
pmem0 259:2 0 100.4G 0 disk /mnt/pmem0
$ kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pmem-pv-volume 50Gi RWO Delete Bound default/pmem-pv-claim local-storage 20h
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
pmem-pv-claim Bound pmem-pv-volume 50Gi RWO local-storage 20h
$ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default daemon 0/1 ContainerCreating 0 20h
kube-system coredns-74ff55c5b-5crgg 1/1 Running 0 20h
kube-system etcd-minikube 1/1 Running 0 20h
kube-system kube-apiserver-minikube 1/1 Running 0 20h
kube-system kube-controller-manager-minikube 1/1 Running 0 20h
kube-system kube-proxy-2m7p6 1/1 Running 0 20h
kube-system kube-scheduler-minikube 1/1 Running 0 20h
kube-system storage-provisioner 1/1 Running 0 20h
$ kubectl get events
LAST SEEN TYPE REASON OBJECT MESSAGE
108s Warning FailedMount pod/daemon MountVolume.NewMounter initialization failed for volume "pmem-pv-volume" : path "/mnt/pmem0/vol1" does not exist
47m Warning FailedMount pod/daemon Unable to attach or mount volumes: unmounted volumes=[pmem-pv-storage], unattached volumes=[tmp log data default-token-4t8sv pmem-pv-storage]: timed out waiting for the condition
37m Warning FailedMount pod/daemon Unable to attach or mount volumes: unmounted volumes=[pmem-pv-storage], unattached volumes=[default-token-4t8sv pmem-pv-storage tmp log data]: timed out waiting for the condition
13m Warning FailedMount pod/daemon Unable to attach or mount volumes: unmounted volumes=[pmem-pv-storage], unattached volumes=[pmem-pv-storage tmp log data default-token-4t8sv]: timed out waiting for the condition
$ ls -l /mnt/pmem0
total 20
drwx------ 2 root root 16384 Jan 20 15:35 lost+found
drwxrwxrwx 2 root root 4096 Jan 21 17:56 vol1
Он жалуется path "/mnt/pmem0/vol1" does not exist
, но он действительно существует:
$ ls -l /mnt/pmem0
total 20
drwx------ 2 root root 16384 Jan 20 15:35 lost+found
drwxrwxrwx 2 root root 4096 Jan 21 17:56 vol1
Помимо использования local-PV, я также пробовал:
PMEM-CSI. Но метод PMEM-CSI заблокирован для меня проблемой с контейнером / ядром: https://github.com/containerd/containerd/issues/3221.
PV. Когда я пытаюсь создать PV, поддерживаемый PMEM, модуль не может правильно требовать хранилище PMEM, но всегда монтируется как оверлейный fs, который находится поверх
/
на хосте.
Может ли кто-нибудь помочь? Большое спасибо!