Since yesterday I suddenly stumbled upon a very absurd problem.
My setup is an ESXi 5.1 home server with two VMs: a Debian (for mail, printing, webhosting, telephony, and more) and a ZFSGuru (FreeBSD 10, file server).
The only role of the FreeBSD VM is a ZFS file server.
Everything works great. I mean, worked great. Since yesterday the entire ESXi server locks up if I use the FreeBSD VM.
I found out about this when I was browsing a SMB share hosted by the VM in question and everything froze. I soon found out that the same thing happens if I try to access the web interface or some certain things over SSH.
I'm not sure what exactly triggers it (CPU usage, I/O, network) but it only happens on the FreeBSD VM. While the Debian VM is affected if the system freezes, I can't trigger the problem on the Debian VM.
When I say the server hangs, I mean that I can't access any VM on the server, nor access it via vSphere client or ping it.
ESXi's vmkernel.log contains the following errors:
2014-05-09T11:45:50.456Z cpu1:4550)WARNING: LinNet: netdev_watchdog:3220:NETDEV WATCHDOG: vmnic1: transmit timed out 2014-05-09T12:33:27.066Z cpu1:6063)NetPort: 1586: disabled port 0x2000005 2014-05-09T12:34:09.476Z cpu0:6064)NetPort: 1392: enabled port 0x2000005 with mac 00:0c:29:39:02:c7
There's nothing to be found in the FreeBSD VM's logs.
I have no idea where this problem came from. The only thing that may be related, is that I rebooted the FreeBSD VM to tune a l2arc setting (vfs.zfs.l2arc_noprefetch to be specific) the day before. I already reverted this to rule that out.
I already tried patching ESXi. Didn't help. I can't really update FreeBSD. I tried to update the ports, but that triggered the problem.