Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] delete mountpods 后出现多个mountpod ,应用pod写入bad file #1267

Closed
YunhuiChen opened this issue Feb 10, 2025 · 0 comments · Fixed by #1269 or #1268
Closed

[BUG] delete mountpods 后出现多个mountpod ,应用pod写入bad file #1267

YunhuiChen opened this issue Feb 10, 2025 · 0 comments · Fixed by #1269 or #1268
Assignees
Labels
kind/bug Something isn't working

Comments

@YunhuiChen
Copy link

What happened:
k delete pods juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-uusphw
k delete pods juicefs-chaos-k8s-001-pvc-1ad6fe2f-ba3c-413b-aaef-e86353bf40d6-ueyelc
后出现了4个新的mount pod :
root@chaos-k8s-001:~/chenyunhui# k get pods |grep pvc
juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong 1/1 Running 0 2m38s
juicefs-chaos-k8s-001-pvc-1ad6fe2f-ba3c-413b-aaef-e86353bf40d6-mfknus 1/1 Running 0 57s
juicefs-chaos-k8s-001-pvc-1ad6fe2f-ba3c-413b-aaef-e86353bf40d6-tvobac 1/1 Running 0 57s
juicefs-chaos-k8s-001-pvc-1ad6fe2f-ba3c-413b-aaef-e86353bf40d6-wuwaoo 1/1 Running 0 57s
进入应用pod后:
root@dynamic-ee-0:/# echo 1 > /data/2
bash: echo: write error: Bad file descriptor

E0210 10:23:08.870513       7 pod_driver.go:733] "bind mount error" err="function timeout" logger="pod-driver" podName="juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong"
E0210 10:23:08.870588       7 reconciler.go:145] "Driver check pod error, will retry" err="exec cmd: mount -o bind /jfs/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-uusphw/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6 /var/lib/kubelet/pods/43d5de36-65af-48a4-8a22-773ac08243b0/volumes/kubernetes.io~csi/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6/mount err:function timeout" logger="reconciler-controller" name="juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong"
I0210 10:23:13.891017       7 pod.go:260] "Mount point is ready" logger="pod-driver" podName="juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong"
I0210 10:23:14.192817       7 pod.go:260] "Mount point is ready" logger="pod-driver" podName="juicefs-chaos-k8s-001-juicefs-console-pv-yundlg"
I0210 10:23:14.893133       7 pod_driver.go:708] "umount pod target before recover" logger="pod-driver" podName="juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong" target="/var/lib/kubelet/pods/43d5de36-65af-48a4-8a22-773ac08243b0/volumes/kubernetes.io~csi/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6/mount" remain mount count=5
I0210 10:23:15.894852       7 pod_driver.go:726] "recover volPath" logger="pod-driver" podName="juicefs-chaos-k8s-001-pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-vxmong" target="/var/lib/kubelet/pods/43d5de36-65af-48a4-8a22-773ac08243b0/volumes/kubernetes.io~csi/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6/mount" mountPath="/jfs/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6-uusphw/pvc-051d0a21-0983-465b-ac67-ddb79dbd15e6"

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?

Environment:

  • JuiceFS CSI Driver version (which image tag did your CSI Driver use):
  • Kubernetes version (e.g. kubectl version):
  • Object storage (cloud provider and region):
  • Metadata engine info (version, cloud provider managed or self maintained):
  • Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
  • Others:
@YunhuiChen YunhuiChen added the kind/bug Something isn't working label Feb 10, 2025
@zxh326 zxh326 linked a pull request Feb 11, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Something isn't working
Projects
None yet
2 participants