I've deployed a Gitlab 12.10 instance on microk8s 1.18 on my Ubuntu 19.10 server. I've noticed repeatedly that some of the pods go into status
Init:CrashBackoffLoop with error messages like the following:
Message: failed to create containerd task: OCI runtime create failed: container_linux.go:345: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/var/snap/microk8s/common/var/lib/kubelet/pods/de572f6b-f738-4b1a-ae02-80fef062cabe/volume-subpaths/sidekiq-secrets/dependencies/2\\\" to rootfs \\\"/var/snap/microk8s/common/run/containerd/io.containerd.runtime.v1.linux/k8s.io/3ae6fc33b02192c3940af4ebc991a47b1fd0afc2533901af50ae5a5f93585d1c/rootfs\\\" at \\\"/var/snap/microk8s/common/run/containerd/io.containerd.runtime.v1.linux/k8s.io/3ae6fc33b02192c3940af4ebc991a47b1fd0afc2533901af50ae5a5f93585d1c/rootfs/srv/gitlab/config/secrets.yml\\\" caused \\\"no such file or directory\\\"\"": unknown
Unfortunately, I'm not sure if this problem is specific to Gitlab or a general issue with my microk8s cluster. It's not always the same component that crashes, the ones that seem to fail most frequently are
unicorn, but I've also seen others with a similar "no such file or directory" error from time to time.
When I delete a crashed pod, the new one that gets created starts without problems, then everything seems to work for a while until one or more of the pods crash again.
Any idea what might be causing this?