How to bind spdk driver to NVMe device if root filesystem is mounted there?

Question

I am trying to use SPDK in Ubuntu 20.04 (Linux kernel 5.4.0) to run storage applications. My NVMe SSD is allocated as below:

$ lsblk
...
nvme0n1                   259:1    0   1.5T  0 disk
├─nvme0n1p1               259:2    0   512M  0 part /boot/efi
├─nvme0n1p2               259:3    0     1G  0 part /boot
└─nvme0n1p3               259:4    0   1.5T  0 part
  └─ubuntu--vg-ubuntu--lv 253:0    0   200G  0 lvm  /

I followed the getting started guide from SPDK docs. It is suggested that any filesystem on the device should be unmounted before binding SPDK driver. Since my root (/) filesystem is mounted in the device, I ran umount / before running scripts/setup.sh. It's likely that the SPDK driver does not bind to the NVMe device properly as the example applications like identify do not run. I am not sure if the filesystem is unmounted properly as I still use the shell (/bin/sh) after that.

When I try to go back to Linux native kernel driver with scripts/setup.sh reset, the nvme device gets a new name nvme0n2 and the whole filesystem remains read-only. I have to then physically restart the machine again.

How can I use SPDK in this case?

Answer 1

You cannot do that. Get second NVMe just for test & development.

Answer 2

Got answer from SPDK Slack channel that I cannot bind the SPDK driver to a device with the root filesystem mounted. I either need to have a separate device to contain the root filesystem or use a VM to run SPDK.