2

I followed this(https://matthewpalmer.net/kubernetes-app-developer/articles/kubernetes-volumes-example-nfs-persistent-volume.html) link and installed an NFS server in my cluster with blow yaml file:

kind: Service
apiVersion: v1
metadata:
  name: nfs-service
spec:
  selector:
    role: nfs
  ports:
    # Open the ports required by the NFS server
    # Port 2049 for TCP
    - name: tcp-2049
      port: 2049
      protocol: TCP

    # Port 111 for UDP
    - name: udp-111
      port: 111
      protocol: UDP

---
kind: Pod
apiVersion: v1
metadata:
  name: nfs-server-pod
  labels:
    role: nfs
spec:
  containers:
    - name: nfs-server-container
      image: cpuguy83/nfs-server
      securityContext:
        privileged: true
      args:
        # Pass the paths to share to the Docker image
        - /exports

I installed the server via below commnad:

kubectl create namespace nfs
kubectl apply -f nfs-server.yaml -n nfs

The server installed smoothly and is up and running.

I installed nfs-common on all nodes then I installed the nfs-client provisioner helm chart via below command:

helm install stable/nfs-client-provisioner --set nfs.server=nfs-service --set nfs.path=/exported/path --name=nfs-client --namespace=nfs

But the nfs-client stuck at ContainerCreating and this is the output of describe:

Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  nfs-client-root:
    Type:      NFS (an NFS mount that lasts the lifetime of a pod)
    Server:    nfs-service
    Path:      /exported/path
    ReadOnly:  false
  nfs-client-nfs-client-provisioner-token-k9n8n:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  nfs-client-nfs-client-provisioner-token-k9n8n
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason       Age   From                      Message
  ----     ------       ----  ----                      -------
  Warning  FailedMount  59m   kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-rdc5765db89a64abf87610ba4d853f48d.scope
mount.nfs: Connection timed out
  Warning  FailedMount  57m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r0b472b9b183240bd8a4a3020bc13a129.scope
mount.nfs: Connection timed out
  Warning  FailedMount  55m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r8f149b9cfb4d414cb15834537f4fd88a.scope
mount.nfs: Connection timed out
  Warning  FailedMount  52m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r3f2e2388086f42f987f0e1475ef95385.scope
mount.nfs: Connection timed out
  Warning  FailedMount  50m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r8db7f40d8a7948598deae1fb07162372.scope
mount.nfs: Connection timed out
  Warning  FailedMount  48m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-rff3d9f71b0e44148ba70c3f9f706d8e4.scope
mount.nfs: Connection timed out
  Warning  FailedMount  46m  kubelet, serflex-argus-1  MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r05af2a5659ad47bea7a0e7010d280444.scope
mount.nfs: Connection timed out
  Warning  FailedMount  20m (x5 over 56m)     kubelet, serflex-argus-1  Unable to attach or mount volumes: unmounted volumes=[nfs-client-root], unattached volumes=[nfs-client-nfs-client-provisioner-token-k9n8n nfs-client-root]: timed out waiting for the condition
  Warning  FailedMount  9m7s (x15 over 59m)   kubelet, serflex-argus-1  Unable to attach or mount volumes: unmounted volumes=[nfs-client-root], unattached volumes=[nfs-client-root nfs-client-nfs-client-provisioner-token-k9n8n]: timed out waiting for the condition
  Warning  FailedMount  3m20s (x15 over 43m)  kubelet, serflex-argus-1  (combined from similar events): MountVolume.SetUp failed for volume "nfs-client-root" : mount failed: exit status 32
Mounting command: systemd-run
Mounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root --scope -- mount -t nfs nfs-service:/exported/path /var/lib/kubelet/pods/811e6fd7-0e48-40a3-ad4d-2f677d49178c/volumes/kubernetes.io~nfs/nfs-client-root
Output: Running scope as unit: run-r27616bcceccc4ff194ac009997604d31.scope
mount.nfs: Connection timed out

I have to mention that I am running k8s v1.16.

How I can solve this problem?

Update

I don't know what was the problem but when I removed the nfs-server from k8s and used a server located on a bare-metal machine the client started to work properly.

1

Make sure you have installed nfs-utils (rpm-base distros) or nfs-common (deb-based distros) packages on all Kubernetes nodes.

| improve this answer | |
  • As I mentioned I already installed nfs-common. nfs-utils is for fedora and centos so I don't need it since my nodes are using Ubuntu. – AVarf Oct 18 '19 at 12:39
  • Yes, you're right. I've just checked it as well and I see that nfs-common for Ubuntu already contains nfs-utils. Could you make sure that there are no firewall rules blocking traffic on 111 and 2049 ( both TCP and UDP ) ? – mario Oct 18 '19 at 12:50
  • I checked 3 out of 5 nodes and there is no rule blocking those ports. – AVarf Oct 18 '19 at 12:58
1

If you use NFS v3, then you need yet another port and daemon to run - mountd. Usually. mountd picks a random port number. Thus you have to pin it in /etc/sysconfig/nfs

MOUNTD_PORT=892

And, as @mario have mentioned, make sure that all ports are TCP.

For client's that use NFSv4.x, probably you need to specify it on the command line:

mount -vers=4.1 ....

or

mountOptions:
 - tcp
 - nfsvers=4.1

info deployment.yaml

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.