I wanted some advice on a file system to use on across 5-10 servers that have a shared block device (iSCSI), GlusterFS and LusterFS seem to be it, but they dont say anything about shared block device support
We are currently using Debian 6 for our projects as well
The key thing for shared-block devices is SCSI-3 Reservation support. Many iSCSI stacks support it, and so does Fibre Channel over Ethernet (FCoE). This is what allows the cluster manager to reserve volumes for specific nodes and keep other nodes out and thus avoid file-system corruption. Without SCSI-3 PR, you can cause corruption by accidentally mounting a volume mounted elsewhere; you don't want that unless the filesystem specifically supports it (like OCFS2, GFS).
All of my shared-block support time has been spent on RPM-based distributions (specifically SLES and CentOS). You need to get the clustering layer up first, which can be a pure network setup. Once you have that, you can leverage Cluster LVM (clvm) to create the shared storage infrastructure between your nodes. At that point, you can use any normal filesystem you want on those volumes since the cluster manager will manage when they move around.
If you're looking for true multi-mount support, you're going to have to use a filesystem that supports it. That would be ocfs2 or gfs2 as I recall. They don't perform nearly as well as regular filesystems since they have to implement file-locks at the filesystem level, but they're right handy when you're trying to set up something like a Xen cluster.
