2

I'm running kubernetes on Ubuntu 18.04.1, running some of my pods with mounts via ISCSI. I've noticed recently (maybe due to improperly unmounting the iscsi volume or for other reasons) that at least one of the iscsi extents (ext4 fs) is sometimes corrupt. In an effort to try and fix this, I'd like to change the mount to journaled instead of ordered. I see this in the node's syslog:

Dec 29 11:45:29 mira-b systemd[1]: Started Kubernetes transient mount for /var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/192.168.xx.yy:3260-iqn.2005-10.home.mediastore:container-lun-0.
Dec 29 11:45:29 mira-b kubelet[27086]: I1229 11:45:29.988889   27086 operation_generator.go:506] MountVolume.WaitForAttach succeeded for volume "container-data-folder" (UniqueName: "kubernetes.io/iscsi/192.168.xx.yy:iqn.2005-10.home.mediastore:container:0") pod "container-65d6d65b75-z5l94" (UID: "41ef97fd-0ba2-11e9-ba38-0cc47abd5a26") DevicePath "/dev/disk/by-path/ip-192.168.xx.yy:3260-iscsi-iqn.2005-10.home.mediastore:container-lun-0"
Dec 29 11:45:29 mira-b kubelet[27086]: I1229 11:45:29.988951   27086 operation_generator.go:527] MountVolume.MountDevice succeeded for volume "container-data-folder" (UniqueName: "kubernetes.io/iscsi/192.168.xx.yy:iqn.2005-10.home.mediastore:container:0") pod "container-65d6d65b75-z5l94" (UID: "41ef97fd-0ba2-11e9-ba38-0cc47abd5a26") device mount path "/var/lib/kubelet/plugins/kubernetes.io/iscsi/iface-default/192.168.xx.yy:3260-iqn.2005-10.home.mediastore:container-lun-0"
Dec 29 11:45:29 mira-b kernel: [  389.683464] EXT4-fs (sdi): mounted filesystem with ordered data mode. Opts: (null)

That last line tells me that ext4 fs is being mounted as ordered, but in the k8s docs I don't see any way to specify how to mount the FS, only "fs: ext4" in the iscsi volume options.

inside the pod/container, i see this

/dev/sdi on /config type ext4 (rw,relatime,stripe=256,data=ordered)

Is there a way to explicitly have kubernetes mount the ext4 fs using data=journaled instead of ordered? FWIW, my ISCSI backing store is ZFS, I've already enabled sync writes for the dataset that extent lives on, but to rule anything else out I'd like to have the container also mount journaled to try and ensure any writes are completed before any power-off/unmmounts, etc.

PS: Kubernetes version 1.13.0

Update: Forgot my yaml:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: pod
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: pod
    spec:
      hostNetwork: true
      volumes:
      - name: pod-data-folder
        iscsi:
          targetPortal: 192.168.xx.yy
          iqn: iqn.2005-10.home.mediastore:pod
          lun: 0
          fsType: ext4

looking at @VAS's answer, it seems like mount options aren't valid when specifying iscsi volume inline, and that it needs to be provided as a PV similar to NFS mounts. I'll give that a try.

2

I haven't tested it, but based on the ISCSI examples and code in PR Implement support for mount options in PVs #41906 I can guess that mount options are just added to plugin mount command, so the YAML should be like this:

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: iscsi
provisioner: kubernetes.io/iscsi
mountOptions:
  - rw
  - relatime
  - stripe=256

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv0003
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: iscsi
  mountOptions:
    - data=journaled
  iscsi:
     targetPortal: 10.16.1.10
     iqn: example.server:storage.target00
     lun: 0
     fsType: 'ext4'
     readOnly: false
  • let me give that a try and see if it works. – Evan R. Dec 31 '18 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.